During the session proposals, Ivan Moore originally suggested a talk on better builds for Java programs. There were later proposals for sessions regarding improvements to Maven, so I proposed my own session “Better Ant Builds” to differentiate it from the Maven sessions.

My main goal was to promote by own contribution to this area: the Ant Script Library. JTF knew about this, and asked some leading questions when I proposed the talk about my library, which I thought was funny. I thought it would have been a bit too cheeky to give a great big marketing pitch, but then again I’m not the best marketer. Thanks for the push, Jeff!

Despite there being no projector available, I was able to talk a fair bit about the ASL, along with some of it’s background and design. People seemed interested, which I’m taking as a good sign. I won’t go into details, the website already has everything I discussed.

Jeff also pointed out the Ant Master Build Scripts, which at a glance seems to be a very similar project to the ASL. Looking at them, I’m pretty impressed. The author of those scripts presented a session at JavaOne called “Object Oriented Ant Scripts for the Enterprise“, which I think nails the leading design principles for reusable Ant scripts. I’d say the the master build scripts have a higher emphasis on J2EE projects, which the ASL leans more to building Java libraries, applications and static analysis. Plenty of room for both projects to grow!

The main question I had for the group was this: “How do you test an Ant script”? Historically, this has not been a problem. You wrote a script for a specific project. You ran the script, and it either worked or it didn’t. However, the Ant Script Library doesn’t have it’s own project – it is reused to build other projects. From my point of view, I really want to be able to set up an “integration test”. The answer is pretty obvious: set up a real project that you can use to run the ant script against. After running the script, you should be able to assert that artifacts are created in the expected locations. Duh! (in hindsight, anyway)

One more good tip: GitHub is a good place to set up an open source project. Another alternative might be Google Code. Currently, I’m running ASL from my own Subversion server. I’m thinking about moving to a location that can handle proper project infrastructure (version control, wiki, mailing lists, etc). Personally, I really like subversion, and I think it’s a pretty nice way to integrate ASL with your own project using svn:externals (assuming you’re using Subversion, too). Google Code uses Subversion, but GitHub uses another version control system I can’t remember (oh, wait: Git). Food for thought; I’ll think about that one for a bit.

For more details of the session, there is a write up on the wiki.

Update: The Build Doctor was actually taking good notes (as opposed to my vague recollection from 2 days later), and has a lot more detail in his own blog post on the discussion.

1. The conference is run in a open space format. This virtually guarantees that the most interesting and important subjects are represented
2. The conference is free to attend, and runs over the weekend. This has the affect that people who attend are extremely engaged with the topic at hand. Pretty much everyone who attends contributes to each discussion. There are not really any people there just to make sure that training budget is used.

A huge thanks to Jeff and PJ for starting this conference, and developing a great community around continuous integration and testing.

There were 5 sessions during the day. I’ll go into more detail about specific sessions in later posts.

JDepend already reports if cycles are present between packages, but it isn’t as easy as it could be to cause the build to break. One suggestion is to write a JUnit test that runs JDepend separately, and fails if a cycle is found (see the JDepend documentation). Although possible, I don’t think it is as elegant as it could be, because:

• I have to get the path to the compiled code into the unit test, either by hard coding the path, or possibly passing it in as a parameter somehow. This is tight coupling between the unit test and the place from which it runs.
• I have to invoke JDepend twice: once to check for cycles, and again to generate the HTML report.

I think that an Ant script is a much better place for this functionality: it already has all the path information, all it needs is a way to find out if there are any cycles. If so, it can fail the build if appropriate. Unfortunately, the current JDepend Ant task doesn’t provide a way to find out. Until now, that is! I’ve just completed a couple of patches: one to JDepend itself, and another to the JDepend Ant task. My patch added an extra attribute to the <jdepend> element called “cyclesdetectedproperty”. This will set a property if JDepend detected a cycle. I expect that this could be used in a manner similar to this:

<target name="report-jdepend" depends="compile"
       description="Creates a package dependency report">
   <mkdir dir="${java-report.jdepend-data-dir}"/>
   <jdepend outputfile="${java-report.jdepend-data-dir}/jdepend.xml"
            format="xml"
            cyclesdetectedproperty="jdepend-cycles-found">
       <exclude name="java.*"/>
       <exclude name="javax.*"/>
       <classespath>
           <pathelement location="${java-build.classes-dir}"/>
       </classespath>
   </jdepend>

   <mkdir dir="${java-report.jdepend-dir}"/>
   <xslt basedir="${java-report.jdepend-data-dir}"
          destdir="${java-report.jdepend-dir}"
          includes="jdepend.xml"
          style="${ant.home}/etc/jdepend.xsl"/>

   <fail if="jdepend-cycles-found">Cyclical package dependencies found</fail>
</target>

I haven’t released these patches yet, but I will soon be adding it to the Ant Script Library. All going well, I’ll try and get the patches rolled back into the JDepend and Ant code bases. Let me know if you’re interested in seeing it earlier.

Update:

I just spotted the verifydesign task in AntContrib, which actually does what I would like, and more. Pretty neat, but I’d still prefer that I could fail the build from JDepend, as it is a much simpler task to configure.

I had a look at the list of registrants, and I was quite pleased to see that there are quite a few people I’ve met before on various jaunts around London and elsewhere. I won’t be completely in an unknown group. Let’s hope it will be as much fun as my last CITCON.

More than anything else, this graph tells a story. Here, a relatively large team of developers, spread across 2 time zones are coming to grips with this new fangled “continuous integration” malarkey, over a period of 7 weeks after the new build system was switched on at the end of April.

• Initially, the builds are sporadic; the developers are unsure of the new tools (which includes a completely unfamiliar SCM tool and CruiseControl), as well as the CI commit rules (”don’t break the build!”).
• For the first week, there are only a handful of commits, about a third of them break. People feel a little overwhelmed by the changes; many of the developers are unhappy with the new system, there is a general muttering in the corridors.
• In the second week, developers gain more confidence, and by Wednesday, the number of commits starts to rise drastically.
• In the third week, the shit hits the fan as everyone jumps into the fray and starts to commit their changes as fast as they can. This week sees the first bouts of long term breakages, and developers check in broken code, and don’t know how to fix it. The muttering in the corridors gets louder, and people start complaining over the harsh strictures involved (”We didn’t have to worry about this before!”)
• The fourth week shows a slight improvement, though there are still a number of 3 hour breakages.
• By the fifth week, things appear to be running much smoother. Breakages are still occurring, but the developers have learned to watch out for them, and have become a bit more careful about verifying their code before they check it in. The week also marks the end of a major code release cycle.
• In weeks 6 and 7 the builds are going along much more smoothly, and the team has developed a good rhythm: code, test (sometimes), check in, and watch for problems. Bugs are fixed very quickly indeed. The grumbling in the corridors has mostly stopped, as everyone gets more familiar with what is expected. The release team had an easier job, as well this time. There was much less compilation problems as they tried to build a release.

From my perspective, it’s a fascinating process to watch, as a team comes to grips with a new process. I suspect the team is still pining for old “easier” ways, but there are now a number of developers and release engineers who have grown to appreciate the advantages.

http://www.build-doctor.com/2009/03/25/continuous-integration-is-not-a-silver-bullet

My problem today was that the Dashboard does not show raw build output by default (it will show plenty of reports and other bits and pieces from an Ant build). In my case, I’m calling an external build tool (not Ant) using the CC <exec> builder, but there is no way to actually see the output.

A quick look at the dashboard-config.xml file showed that there appeared to be a simple way to add another tab to the build display using classes from the Widget package. Looking in the CC source code, I spotted a class called net.sourceforge.cruisecontrol.dashboard.widgets.AntOutputWidget, which despite it’s name, looked like it would display the build output for anything. I tried adding a reference to this class to the configuration. Sure enough, a new tab appeared. Unfortunately, it only ever displayed the text “No information for Ant Output”. Doh!

Googling about, I was able to turn up a message from Pete Doyle who had exactly the same problem. He had even submitted a patch to the CC mailing list, but it appeared to have been missed. I tried out his patch, and everything worked fine (after a few fine tweaks). While I was there, I created a new task in the CruiseControl issue tracker: CC-899. Hopefully, this can get rolled into the next CC release. Score 1 for open source!

However, the story doesn’t end there. Before I moved on, I just happened to glance at the documentation for subtabs. There is a strong note warning that these widgets are deprecated:

Note that the dashboard widget API is deprecated and will be replaced in a future version of CruiseControl. We strongly recommend developers do not create widgets using this API, as it will not be supported in the future.

I found this curious: I didn’t think the code for Widgets was that bad, what New and Improved implementation was replacing it? The answer was harder to work out than I expected.

The problem (as I see it) is that there is a relatively convoluted set of Spring configuration files joining bits and pieces together in hidden ways, and somehow using a heap of Velocity templates to generate the UI. My issue is that although the code appears to be quite solid and well factored, the configuration that wires it all together is quite opaque. Because it is Spring configuration (programming by XML), I lost the ability to easily navigate the code using the IDE. It was back to searching for snippets of text, and then looking up the referenced classes.

XML is not a good language in which to write programs. I don’t mind Spring in small doses, but I’ve been burnt a number of times by very large and unwieldy Spring configurations that wire together lots of small pieces in such a way that it’s very hard to get a view of the important features. Scoping, encapsulation, IDE integration: trodden over in a very haphazard way.

I’m all for dependency injection and inversion of control, but I definitely think that most of that can be done more efficiently in code. There are a few posts floating about on the same subject. Here are some of my favourites:

• Crazy Bob (inventor of Guice): I Don’t Get Spring
• Floating point: A Critique of Spring.

• http://www.exubero.com/ccintro/ccintro-s5.html

However, that presentation is getting slightly dated. Searching around, I dug up some nicer examples from Slide Share. I especially liked the first one:

• http://www.slideshare.net/drluckyspin/continuous-integration
• http://www.slideshare.net/dnoble/testing-in-java