OSCON 2007

Look! Pimp My Build! My first conference presentation ever! I'm really looking forward to having some fun with this. The idea behind the presentation is to share some "pimp" (in the parlance of our times), uber-useful CruiseControl customizations. Andy Slocum is my co-presenter. I don't want to give too much away (read: we haven't written the presentation yet), but some of the customizations we will describe include: adding Lucene search to your build, writing plugins, eye-popping build status indicators and custom reporting.

Our original concept was to talk about all sorts of software development tools and how to extend them such as Cruise, Eclipse, Rails, etc... But that seemed way too ambitious for a 45 minute presentation. We had a funny title for that one: "Andy and Josh, (a couple of) open source tools." Good thing we refined the topic, eh?

How to add search domains in Ubuntu

I have lived with the following annoyance for the past year. At work I can't resolve various server names without editing the /etc/resolv.conf file and adding the following

search www.example.com

No big deal, except every time you connect to a new network this file is automatically overwritten with information received from DHCP. This is really annoying and after adding that line by hand for a year I finally got a solution!

It is really simple. Edit /etc/dhcp3/dhclient.conf and add the following line in there:

prepend domain-name "thoughtworks.com ";

duh! Many thanks to my friendly neighborhood sysadmin and this webpage

Fixtures Documentation out of date.

I just wanted to mention that this documentation from the rails api is confusing and out of date. I should probably figure out how to update it myself. That is on my list of things to do, meanwhile maybe somebody else will get there first. As of rails 1.0 instantiated fixtures are disabled by default, so you can't just say @users.joe.name to get Joe's name. You need to do users(:joe).name. If you want the first type of syntax you need to set self.use_instantiated_fixtures = true. This sure confused me. There is a really nice writeup on this stuff and the performance implications here.

Are Rails Test Fixtures Good or Evil?

First let's talk about just what the heck a fixture is. In real estate they are defined as follows.

Accessories fixed to structures or land in such a way that they can't be independently moved without damage to themselves, or the property housing them.

Doesn't sound like something you want in your software does it? But I've used fixtures plenty! They aren't that bad. They are just an easy way of setting up test data. The reason for this write up is that recently I have heard lots of negative things about fixtures and I wanted to figure out for myself if they really are evil.

Now I always liked fixtures because I like my tests to be short, clean and DRY. Fixtures are a very cool, built in way to do that, and they can really speed up your test writing, which is great because it lowers the barrier to writing more tests. So fixtures keep things DRY, but at what price? Jay Fields has an interesting article that suggests tests need not be DRY. Jay argues that isolation of tests is more important than clean, dry tests. I call this the "Every test for itself" philosophy of testing. Here is a quote from that article.

It means that when writing tests you should think about them in isolation. They run in isolation and the chances are when they break (if you write robust tests) they will be fixed in isolation; therefore, abstracting anything that is 'common' actually hurts maintainability. With every test, pretend it's the first and only test you are writing in that class. If that were true, it would be hard to justify those setup or private helper methods.

This is a totally awesome observation and agrees with my own experiences. As I type this I am thinking of all the convenience methods I have put into test classes. Whoops. Well, my tests were ultra clean, short and DRY tests for at least one day. The next day somebody who didn't notice my convenience method went ahead and re implemented it. Sigh.

Still, although I am going to try going without them for my next few projects, I think fixtures are convenient and when used properly there is nothing evil about them. So how do you use fixtures properly? Well perhaps it would be simpler to describe how to use them improperly. Here is a list from Tom Preston-Werner's RailsConf 07 talk on what you don't want from your fixtures.

1. swampy (shows photo of swamp with trees with bad smells with stink-lines)
2. fixtures become an unmanageable mess
3. hard to keep track of links between data
4. things aren't easy to refactor
5. no namespacing – i.e. you need to have lots of users in different states for good testing
6. brittleness – one day you add a new column to a table, and half of your tests fail
7. no validation – there is no automatic way for ActiveRecord validation for fixture data
8. contamination – tests pass independently, but fail when running all at once
9. performance – fixtures can be slow

In practice all these, except maybe number 3 can be avoided. Gripes 4 and 6 are easily fixed by the db:fixtures:migrate task. 2 and 8 are just a matter of being disciplined about writing, updating and using your fixtures. I'm not sure what swampy means, but it sure sounds bad. 9 can definately be a problem, especially if you are not using transactional fixtures (and if you aren't using a transactional database you aren't), but any test suite that uses the database can have that problem.

In the end it comes down to what you are comfortable with. Decide for yourself. If you can't live with Jay's "every test for itself" philosophy, then maybe fixtures ARE for you. Just keep clear of the 9 pitfalls Tom mentions and you will be fine. Remember it matters how you test, but the most important thing is that you test. Quoth Testivus

An imperfect test today is better than a perfect test someday

The perfect is the enemy of the good.

Don’t wait for best to do better.

Don’t wait for better to do good.

Write the test you can today.

The problems of maintaining a large suite of unit tests is considerable. However, the benefits of a test suite, even a troubled one, far outweighs the costs.

References:

• http://manuals.rubyonrails.com/read/chapter/26
• http://clarkware.com/cgi/blosxom/2005/10/24
• http://api.rubyonrails.org/classes/Fixtures.html

How to Migrate Fixtures in Rails

4/1/2008 - Update: check out this writeup on the fixture migrations for a more in depth analysis of the plusses and minuses of our code.

I did some work on a Rails app awhile back and we were using Cerberus for Continuous Integration. The build was in good shape. All tests passing. We turned it over to another team, but they never setup CI. Guess what happened? If you said, "The tests got stale and broken." then you are a winner. Quoth Testivus:

Tests not run waste away

Run your tests often.

Don’t let them get stale.

Rejoice when they pass.

Rejoice when they fail.

Looking at the failures it became apparent that many of the problems had been caused because of the fixtures being used. The model had been changed with migrations, but the fixtures were still old. That's easy, just change the fixtures. And I did. Yuck! Changing all the fixtures turned out to be a mind-numbing task! Consider this. Your fixtures are nothing but a representation of the data in your test database. That representation is dependent on the structure of your database. By manually changing my fixtures to use the new DB structure I had done a migration. By hand. Yuck!

I knew there was a better way, and so Jake Scruggs and I joined forces to tackle this problem. Our solution is a new rake task called db:fixtures:migrate, and you can get the code here To make it work, we have introduced the concept of a version to our test fixtures, and this makes sense because our fixtures are essentially a serialized, human readable version of the database, and in Rails the DB has a version, right? Right.

I haven't tested this task on every imaginable case, but it has worked for me on a couple of my own little projects. That is cool. Let me now bring up a side point. In learning about some of rails rake tasks I came across some thing I thought was ugly (please enlighten me if there is a reason for this). When you run tests, rails completely wipes the test db (which is good), but then instead of rebuilding the schema with migrations it just copies the schema from the development DB (which is bad). To me this is ugly, and it can cause headaches if you aren't watching out, because now your test DB depends on the state of your development DB. Two lines of code fix this, but I wanted to hear some old, wizened Rails gurus opinions on this before I went crazy with this idea. If you have a look at our rake task you can see that when we setup a test DB we use migrations to generate the schema. This is also good because if you don't run your migrations often, how do you know that they will work? You might as well run them every time you run tests. So go try it. Use it. And say what stinks.

Next up: Maybe I shouldn't use fixtures at all.

Bullet Points and Bullets, or How Television Killed Democracy

The first rule of political discourse is to be as vague as possible. George Bush, the master of this technique (or just a natural), began a war in Iraq March 20th, 2003. For two and a half years he kept the war machine running strong by sticking doggedly to this rule, saying nothing but vague yet decisive sound bytes.

I'll summarize his strategy in those years: Go, fight, win.

Good plan. Let us hope that political speech is completely bankrupt, because for political speeches to get any more meaningless or pedestrian than they already are, politicians will have to use gibberish (which I think Bush is working on).

November 30th, 2005 Bush finally put out his official strategy guide for achieving his goals in Iraq, following all the bullets with bullet points. I'm going to leave the discussion of his strategy guide for another day, but I'll say that if you are familiar with any video game strategy guides, you will no doubt be disappointed with this one. You don't stand a chance of saving the princess with it.

I'm getting sidetracked though, this isn't supposed to be about Bush. What I was really thinking about was political speech. Political speech is empty, duh, but why is that? I think if anyone is to blame it is the media, especially television. Television's tendency to sensationalize and take things out of context is poison to a democracy. A fundamental idea in democracy is that no single person has all the answers. We have to have a forum for discussion, get out different ideas and vote on the best one. This doesn't work if nobody can say anything without getting whacked over the head with a newspaper, and in Pavlovian fashion people avoid saying anything. Ideally a leader when asked a question will answer the question openly, and the media rather than trying to exploit the answer for a story can sensibly point out weaknesses and allow the leader to adjust their position accordingly during the course of the discussion. Instead, our leaders stand up at press conferences, which are a sort of TV-era running of the gauntlet.

I think I could talk about that for a long time, but I won't. I just wanted to mention a potential light at the end of this tunnel. Before Television media our democracy was still a place for people to have real debates, and ideas would go forth and do battle. Then television ruined it (go read Walter Cronkite's autobiography for more on this). Now that the internet has begun to revolutionize the media I hope that we can reclaim our democracy as a staging ground for good ideas. The internet, with its instant access to contextual information (think Wikipedia) might be just the forum for politicians to begin speaking again. The question is, who will be the first to ditch the double talk?

Down with television!