Wednesday, September 20, 2006

Subversion build labels in CruiseControl


I spent some time last week learning about setting up CruiseControl. My motivation was to figure out how to make the subversion revision number part of our build number. Example: say we are on build number 101, cruise kicks off and does a checkout. The code gets updated to revision 1001. That means the build number becomes 101.1001. This mapping from build to revision is useful. Say you had delivered build 85 to the customer, and there is a bug. In order to look at the code in production you need to update your working copy to the production build. If you don't have the revision in the build number you have to go rummaging through logs to find which revision build 85 is, or you can checkout the branch that cruise creates for each build, but for large codebases, that is time consuming. With the build-revision style label you can quickly update to that revision.

Making this happen should be easy, right? Just write a build labelIncrementer plugin that gets the SVN revision. The catch is that in cruise control build numbers are calculated ahead of time. I had made an unfortunate assumption that the build label was calculated at build time and so I wrote an entire plugin to calculate the label I wanted. Then I realized it would never work. Of course this behaviour is documented, but I wasn't bright enough to read that ahead of time. Here is the code to my SVN LabelIncrementer. You can use it but you might be disappointed when your revision numbers in the label are from a previous build. Or you can just skip ahead and read how I ended up satisfying my requirement without writing a plugin.

import net.sourceforge.cruisecontrol.labelincrementers.DefaultLabelIncrementer;
import org.jdom.Element;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

public class SVNLabelIncrementer extends DefaultLabelIncrementer {
private String workingCopyPath = ".";

public String incrementLabel(String oldLabel, Element buildLog) {

String SVNRev = "";
try {
SVNRev = getSvnRevision();
}
catch (IOException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}

String label = super.incrementLabel(getNonSubversionPartOfLabel(oldLabel), buildLog);
if (SVNRev == null || SVNRev.equals(""))
return label;
return label + "-" + SVNRev;

}

protected String getSvnRevision() throws IOException {
String rev;
Process p = null;
try {
p = Runtime.getRuntime().exec("svnversion " + workingCopyPath);
BufferedReader stdInput = new BufferedReader(new
InputStreamReader(p.getInputStream()));
rev = stdInput.readLine();
}
finally {
if (p != null)
p.destroy();
}

return rev;
}

public boolean isValidLabel(String label) {
return super.isValidLabel(getNonSubversionPartOfLabel(label));
}

private String getNonSubversionPartOfLabel(String label) {
int index = label.indexOf("-");
if (index == -1)
index = label.length();
return label.substring(0, index);
}

public void setWorkingCopyPath(String path) {
workingCopyPath = path;
}

}

Here are my tests for the plugin.

import junit.framework.TestCase;
import org.jdom.Element;

public class SVNLabelIncrementerTest extends TestCase {

public void testReturnsIncrementedLabelWhenRevNotPresent() {
SVNLabelIncrementerStub inc = new SVNLabelIncrementerStub();
assertEquals("FE6.0.2", inc.incrementLabel("FE6.0.1", new Element("foo")));
}

public void testReturnsIncrementedLabelAndRevisionWhenRevPresent() {
SVNLabelIncrementerStub inc = new SVNLabelIncrementerStub();
inc.setSvnRevision("1");
assertEquals("FE6.0.2-1", inc.incrementLabel("FE6.0.1", new Element("foo")));
}

public void testReturnsIncrementedLabelWhenRevNull() {
SVNLabelIncrementerStub inc = new SVNLabelIncrementerStub();
inc.setSvnRevision(null);
assertEquals("FE6.0.2", inc.incrementLabel("FE6.0.1", new Element("foo")));
}

public void testProducesValidLabelWhenRevNotPresent() {
SVNLabelIncrementerStub inc = new SVNLabelIncrementerStub();
String label = inc.incrementLabel("FE6.0.1", new Element("foo"));
assertTrue(inc.isValidLabel(label));
}

public void testIncrementsCorrectlyOverSeveralIterations() {
SVNLabelIncrementerStub inc = new SVNLabelIncrementerStub();
inc.setSvnRevision("1");
assertEquals("FE6.0.2-1", inc.incrementLabel("FE6.0.1", new Element("foo")));
assertEquals("FE6.0.3-1", inc.incrementLabel("FE6.0.2-1", new Element("foo")));
assertEquals("FE6.0.4-1", inc.incrementLabel("FE6.0.3-1", new Element("foo")));
}

public class SVNLabelIncrementerStub extends SVNLabelIncrementer {
private String revision = "";
protected String getSvnRevision(){
return revision;
}

public void setSvnRevision(String revision){
this.revision = revision;
}
}
}

There is more than one way to relabel a build. But without doing something gross, like having ant call a script to modify the build number after cruise generates it (you can change build labels in the xml logs that cruise uses to store info about previous builds) I couldn't think of a good way to get the SVN revision into the build label.

Then Andy Slocum came to my rescue with the bright idea of modifying the xsl that creates the html page from the xml logs. He simply added this: (Sorry this is an image, I couldn't figure out how to get wordpress not to supress the xsl.) to the header.xsl file, which cruise uses to generate the header of the build page and PRESTO, the revision appeared. Our build page now looks like this.
Glorious, eh? I've looked at the cruise code. It is actually quite clean and easy to understand. I might take a look at changing the build label to be generated at build time, thus making it possible to use the SVN revision in the label, but I would probably talk to some cruise gurus about it first.

12 comments:

Shlomo said...

I almost forgot that I was having a problem with my labelIncrementer plugin. I wanted to pass it a parameter from the config file. workingCopyPath. Notice I had a setter for it. My understanding was that by setting that in the config.xml of cruise the setter would get called. However it wasn't happening. Anyone have any idea why?

Shlomo said...

Why bother with the build number?

We wrote one at work, and we just use the revision ID of the checkout. This means we know _exactly_ the state of the source code in the build, and can reproduce it at any time by checking out that revisiojn.

Shlomo said...

I suppose there is not a really good reason for using a build number. The only thing I can think of is that if someone forced the build, meaning the revision number didn't change, you would have a non-unique build number. Whether or not that matters, I don't know.

Was your solution for cruise control? How did it work exactly, because from what I can see there is no clean way of having the subversion revision as the build label. The build label incrementer always gets called after a successful build, and the result stored for the next build. Thus the revision will be one step behind the actual build. How did you make it work?

Shlomo said...

Is there a reason you don't set

preBuildIncrementer=yes

??? In conjunction with your code above I think it will do just what you want.

http://cruisecontrol.sourceforge.net/main/configxml.html#labelincrementer

Shlomo said...

This config parameter has a misleading name. What this does it to increment the build label even when the build failed. So the "pre" means before the success of the build has been determined.

From the cruise control docs: "if true the build number will be incremented prior to the build attempt and thus each build attempt will have a unique build number."

I made the same assumption you did.

Shlomo said...

Josh,
Did you actually try it? I think the key phrase here is "build number will be incremented prior to the build attempt".

In the last couple days I set up a CC implementation using the SVNLabelIncrementer plugin by Pavol Zibrita (like yours, but using javahl and talking directly to the repository.)

Post: http://article.gmane.org/gmane.comp.java.cruise-control.devel/8598/match=svnlabelincrementer

Download: http://kopernik.cc.fmph.uniba.sk/~zibrita/svnli-cc2.3.1.zip

Right now I have working builds labeled by the current svn revision.

The only issues are:
1. I had to add some authenitcation code to the plugin because on win32 javahl can't read the svn auth file passwords
2. There's a small window for additional commits between the time the labelincrementer runs and the build does 'svn up'. This could easily be worked around by making the build 'svn up' to the revision in the build label.

Shlomo said...

Honest, I tried it. If you go in and look at the code to cruise control you can see the difference. Here it is from Project.java in the cruise code

if (labelIncrementer.isPreBuildIncrementer()) {
label = labelIncrementer.incrementLabel(label, log.getContent());
}

// collect project information
log.addContent(getProjectPropertiesElement(now));

setState(ProjectState.BUILDING);
log.addContent(schedule.build(buildCounter, lastBuild, now, getProjectPropertiesMap(now)).detach());

boolean buildSuccessful = log.wasBuildSuccessful();
fireResultEvent(new BuildResultEvent(this, buildSuccessful));

if (!labelIncrementer.isPreBuildIncrementer() && buildSuccessful) {
label = labelIncrementer.incrementLabel(label, log.getContent());
}

This is all happening in the same build loop. You can see that the only difference is that the PreBuildIncrementer just ensures that every build (regardless of pass/fail) will have a unique label.

Shlomo said...

Absolutely it works fine for me, so I wonder if the problem you were having isn't in another corner.

Your label incrementer is running svnversion on the working copy. Are you using a boostrapper to ensure that working copy is up-to-date? Otherwise that would perfectly explain why your "next build" always has the revision number that the last build updated the working copy to. The plugin I'm using uses the working copy to find the repository, but it asks the repository directly what the HEAD revsision number is, thus getting around this issue.

Shlomo said...

For clarification if this becomes a discussion of "the code clearly will do this" vs. "my installation does that": I'm running CC 2.5 on win32.

Patrick said...

Hi Josh

I just found this posting, and it looks like it may be the solution I'm looking for. However, the images aren't showing up. It looks like they didn't make the transition form WordPress to Blogger.

If you get a chance, could you take a look at that?

Thanks,
Patrick

Shlomo said...

patrick,

yes. various things from the migration got screwed up including all the images since the server containing the images is gone. However I still have them all and slowly as they are needed I am restoring them. Anyway, the images for this post are back. Hope it helps.

you will also notice that it looks like i was having a conversation with myself. that is another unfortunate side effect of the migration :(

Shlomo said...

Ok. Since my original solution didn't really need the cruise plugin i never took the time to figure everything out. Here is some code for a plugin that really works.

The only problem with the code here is that it doesn't let you configure the location of the working copy. That is why I thought the label incrementor wasn't working right.
http://www.cuberick.com/2007/10/cruisecontrol-subversion.html