Thursday, July 26, 2007

Wanna Draw? Try Inkscape!


This is another diagram I created for our presentation at OSCON to illustrate a particular project's cruise setup. I like it much better than my first attempt. Both attempts at diagrams were composed in a marvelous little open source program called inkscape. Check it out!



Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 License.

Sunday, July 22, 2007

Continuous Integration Diagram


This is a high level diagram that I made describing how Continuous Integration works. Click on it for the full size image.








Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 License.

Saturday, July 21, 2007

The Case for Continuous Integration

Most people don't need an introduction to Continuous Integration (CI), but there are still many programmers and teams out there who haven't adopted this practice. I know some of them personally. This writeup is for them. It is a series of short and hopefully persuasive arguments for adopting CI. The idea here is to give the adopter a variety of easy to digest reasons for using CI.

First, here is a brief definition for CI. Continuous Integration is automated compilation and testing that leads to the creation of ready to deploy software any time the code base changes. Now for the arguments.

The Code you Can't Deploy Argument
Code you can't deploy is useless code. You just put the finishing touches on a servlet six months in the making. Good thing too because the customer plans to go live this weekend. It works fine when deployed on your windows workstation from Eclipse, but the customer is deploying into a clustered environment with a Linux operating system. Guess what, it is going to be a looong weekend. Had you been using proper Continuous Integration you could tell your client with confidence that there wouldn't be any big surprises.
The Cost and Quality Argument
One of the basic operations of writing software is integration. As a developer you constantly integrate with external systems, libraries, teammates' code, even the code you wrote yesterday. Programmers are natural continuous integrators! But programmers are only human, and we make mistakes (better known as bugs). CI, done properly, is a foolproof way for rapidly discovering bugs. And the faster we discover a bug, the cheaper and less risky the fix. CI makes our software higher quality and lower cost.
The Nobody's Perfect Argument
Good programmer writes a test and the code to make the test pass. Next, good programmer runs all the tests, not just the tests around the change, and only after the tests pass does good programmer check in. Why bother with a continuous integration server like CruiseControl when everyone on your team is such a good programmer? Hmm, wait a minute, did good programmer remember to check in that 3rd party lib? Did good programmer forget to do an svn add on that new unit test for that critical 3rd party interface? Even good programmers make mistakes that can be troublesome for the rest of the team and costly down the road. Really good programmers know they make mistakes and love Continuous Integration for catching those mistakes before they impact the team and the project.
The Dependent Projects Argument
Team one has a build. Team two's build depends on what team one is building. Team one creates a new drop of their system once a week for delivering to team two. This means it might take up to a week for team two to realize that team one has broken their software. Perhaps it takes another week for team one to understand and fix what they broke. Meanwhile team two is building against the last working drop they recieved from team one, which is now three weeks old! This whole mess would have been avoided had teams one and two been using Continuous Integration (CI). With CI team one is automatically creating a new drop of their system every time their code changes, and team two is automatically testing their code against team one's latest drop. Now both teams know within minutes of checking in when and why their software no longer works.
The Better Communication Argument
A Continuous Integration (CI) server like CruiseControl gives developers a centralized place to see the status of the code. They can see which classes their team members are working on, grab the latest deployable, and get notifications when the status of the build changes. This information can be a catalyst for important project conversations. One developer might notice that another has been checking in code without tests. That is a good time for a quick talk about the importance of unit testing. Two developers might realize they are working on the same part of the system, another opportune moment to talk about how they might avoid commit conflicts . CI increases the points of contact between individual developers or pairs during the day and encourages, even forces, better team communication which can head off costly problems before they have a chance to arise.
It is difficult to leave it at that. I still want to give more examples, discuss all the different levels to which you can take CI, and talk about how even small projects and small or single developer teams can benefit, but so much has already been written, so I'll conserve space on the web by just linking to a couple good sources for further reading.

Thursday, July 5, 2007

Replacing Ant With Rake


The other day, as I was tracking down a bug in a large J2EE application, I had an idea for a new ant task. Hooray automation! The excitement of the new idea quickly vanished as the reality of dealing with ant set in.

The Problem:
Programatically changing text in a file. Oh noes! I will have to create some templates, copy files around, replace words and stuff. An hour or two to figure out what I could do with rake in 5 minutes. But adding a Rakefile to the project for just one task is lame! Dutifully, I popped open our 2k line build.xml and nearly gagged. Rake it is then. But how do I keep our build consistent? One way would be Re-implement our entire ant build in rake. Crazy talk! I needed a compromise. Thinking about it for a second I wondered, "What if rake could seamlessly use all my ant tasks?"

The Solution:
Making rake ant aware! Now i'll be able to call all my ant tasks as if they were rake tasks.
rake antTask
I can even mix and match ant tasks with rake tasks.
rake antTask rake_task antTask2
Hoo boy, this is gonna be great! But how do we do it? I need something like method missing for rake tasks. Determined to figure this out I checked out the rake source code. Nothing like method missing there, but it turns out just a few addidtional lines of code gives rake ant awareness. Here is what I ended up with in the top of my Rakefile.

#Redefine the way rake works so we can have it check our build.xml
module Rake

# open up the TaskManager to make it try ant before pooping out
module TaskManager
# Find a matching task for +task_name+.
def [](task_name, scopes=nil)
task_name = task_name.to_s
self.lookup(task_name, scopes) or
enhance_with_matching_rule(task_name) or
synthesize_file_task(task_name) or
call_ant_task(task_name) or
fail "Cannot find task called '#{task_name}'"
end
def call_ant_task(task)
return nil unless system("ant #{task}")
#return a do nothing task because ant is doing the work here.
Rake::Task.new(self, self)
end
end
end



Commentary on the solution:
This solution is a bit of a hack and a better way might be to handle the ant tasks with their own class like the FileTask class does it. But my special behavior is so simple that at this point it might be overkill. Anyway, this works for now.

The Limitations:

It is important to note that a major limitation of this is the inability to pass parameters to your ant tasks. We don't have many ant tasks that you have to pass parameters to (thank you build.properties), so I'm not going to worry about this right now. But going forward it would be nice to see that work. The problem is that rake parses the command line options before it initializes your Rakefile, so I can't overwrite the handling of the command line options from a Rakefile. Too bad. I am going to post a question on the Rake-devel mailing list to see if anyone there has some bright ideas on this, and I might whip up a patch for rake that would make this possible, though I haven't really thought through everything yet. Another limitation is that each ant task is a seperate call to ant, so if you run multiple tasks that depend on the same task that prerequisite task will get run multiple times. That could also be overcome by just storing all the "unknown" tasks as a list and making one ant call at the end. I leave that as an exercise for the reader. :)

Now you will go forth with the power and simplicity of rake without losing your investment in ant!