Dynamic Tweet Button Text and Counting Tweets

I deployed a new version of Hydration = which adds a little tweet button to the footer. One thing I wanted to do was make the text of the tweet different depending on what the user has done on the site. Since the site is just javascript and one static page I wanted to find a simple way to update the value that twitter sends for the tweet text. Unfortunately Twitter uses an iframe for this. The trick is to realize that the src attribute of the iframe contains the tweet message. So you can just update that with jQuery. That works pretty well. There is a slight annoyance because changing the src of the iframe causes the iframe to reload.

On a totally separate note. Another thing I bumped up against was getting the tweet count to increment. Beware URL's with a trailing slash. Twitter must strip that out of the URL before they use it as a key and so if your data-url attribute has a trailing slash the tweet count doesn't work.

The type casting edge case in Rails validation

Validation is one of the things Rails does very well. Most of the time you can manage your validation with one of the standard declarative validations. Occasionally you need something custom, but even then life is good and your custom validation is as simple as writing a two or three line method. However, there are edge cases and I'm going to talk about one. Let's start with the request. Request parameters come in to your application as strings and rails automatically maps them to the type prescribed by your DB.

If you aren't thinking about this (and it is easy not to think about it) you can be in for some surprises. If your field is an integer, but a user supplies a non numeric string, it will get type cast as zero. If your field is a date, but a user supplies an invalid date, like February 31, you will get March 3rd. If they were really off and supplied February 43rd, you will just get nil.

This leads to some confusing behavior. Imagine you have an optional date of birth field on a form and a manual validation to ensure that date is in the past. If somebody enters February 43rd 1969 as their date because of type casting the field is set to nil and nothing will happen. Which is weird.

If we want to give our users meaningful feedback about their dates we are going to have to do something about this. Looking at stackoverflow. Most people trying to solve this problem just override the accessor. Bad! Changing the behavior of an accessor in the best case is introducing unexpected behavior to your models and in the worst case will cause bugs because that isn't how a standard activerecord object works.

The Rails folks have already considered this problem and so there is a feature that works around it. It seems not too many people know about it, but it is documented! ActiveRecord has alternate versions of every attribute. They are aptly named *_before_type_cast. So in our case date_of_birth_before_type_cast would contain the string 2/43/1969. And you can use that value to give a more accurate error message. I lost a couple hours the other day researching this problem, but lesson learned and I thought I would spread the good word about _before_type_cast.

Lowering the barrier to metaprogramming

I've been trying to finish the Io chapter of Seven Languages in Seven Weeks, and I just added indentation to the LispML implementation. At the end of the Io chapter Bruce says

Io’s syntax is remarkably compact. You can learn Io very quickly. Once you understand the core syntax, everything else is learning the library structures. I found that I could work my way into metaprogramming quite quickly, within my first month of using the language. In Ruby, getting to the same point took a little longer. In Java, it took many months to get to the point where I could make any sense of metapro- gramming at all.

He is right on the money! But edit "within my first month" to "within my first week". Also, since there isn't special syntax for metaprogramming you aren't likely to forget how to do it. All you need to understand is how a method call in Io works, and you are ready to metaprogram.

Io Language: the init method

I'm slowly working through Bruce Tate's book "7 Languages in 7 Weeks". It is ambitious and lots of fun. In the Io chapter I struggled and I think it was because Bruce forgot to mention Io's init method. The init method is automatically called anytime a prototype is cloned and it is important becuase if you can't setup each clone, you can't have full fledged instance level behavior. Here is some code I wrote after reading Bruce's intro to Io.

Closet := Object clone do(
  closet := list()
  stash := method(item,
    self closet append(item)
  )
  peek := method(
    self closet foreach(item, item println)
  )
)

myCloset := Closet clone
yourCloset := Closet clone
myCloset stash("rollerskates")
yourCloset stash("bowling ball")
"My closet looks like:" println
myCloset peek

"Your closet looks like:" println
yourCloset peek

If you run that code you will see it print out the following:

My closet looks like:
rollerskates
bowling ball
Your closet looks like:
rollerskates
bowling ball

Whoops. My closets aren't behaving like separate instances! What is going on? Well, in prototype if an object receives a message and no slot exists to receive that message, it forwards that message up to its prototype. In this case, the closet slot is on the Closet prototype, so closet is like a class variable. It sounds simple, but it took me awhile to understand what was happening. I had been assuming that a "clone" was an object identical to its prototype, but it isn't identical at all. In fact, initially the closet clones have no state or behavior of their own. It felt like something was missing...

Of course! These clones need to be able to initialize themselves. With an init method each of the clones can have their own closet slot. No more shared state.

Closet := Object clone do(
  init := method(
    self closet := list()
  )
  
  stash := method(item,
    self closet append(item)
  )
  
  peek := method(
    self closet foreach(item, item println)
  )
)

Run that the same way we did previously and you get:

My closet looks like:
rollerskates
Your closet looks like:
bowling ball

Which is what I meant originally.

They say learning 1 new language a year will make you a better programmer. Bruce's book can help keep you well ahead of that curve.

Testing image generation with Cucumber and Sikuli

Image recognition can be a useful weapon in your testing arsenal. Our current project is a rails app that generates advertisements. We use sikuli's image recognition to detect that the text and images that we expected to be in the advertisement were actually in the ad. To integrate sikuli with our build we simply created a cucumber step that calls sikuli. Let's cut to the chase and see a sample test.

Scenario: Dealer places a Campaign Ad
 Given "Bill Murray"'s dealership "Ghostbusters" exists in the system
 When I log in as "Bill Murray"
 And I book a "Year End" campaign ad
 Then the ad proof will look like "ghostbusters_tag.png"

Cool. The magic sikuli step is the one that looks at the ad proof. Here is the step implementation.

Then /^the ad proof will look like "([^\"]*)"$/ do |filename|
 feature_dir = File.expand_path(File.dirname(__FILE__) + '/../../features')
 sikuli_script_dir = "#{feature_dir}/sikuli"
 verification_image = "#{feature_dir}/images/#{filename}"
 result = `/Applications/Sikuli-IDE.app/Contents/MacOS/JavaApplicationStub #{sikuli_script_dir}/verify_existance.skl '#{verification_image}'`
 result.should match(/hallelujah/)
end

Pretty straightforward. Call sikuli and pass it a sikuli script and the image. The sikuli script returns the word 'hallelujah' if it looks at your app and determines that the verification image exists anywhere on the screen. Ok. Lets drill down into what the sikuli script looks like. Get your Python goggles on...

import sys
image = sys.argv[1]
switchApp("Firefox.app")
type(Key.DOWN, KEY_CMD)
if exists(Pattern(image).similar(0.91)):
 print "hallelujah"
else:
 print "bugger"

This one is brief, but I'll explain a bit. Sikuli is a cross platform tool for scripting your computer as well as interacting via image recognition. In this script we make sure that firefox is the app in focus. The image we care about is at the bottom of the page so command down tells firefox to go to the bottom of the page. Finally it looks for the image match. The 0.91 number is basically telling sikuli to be damn sure. We print hallelujah if there is a match.

Ok. That is what I am doing, but in practice how does it work? Well, it has actually been pretty trouble free, and it even finds bugs and problems. Amazing! However it isn't perfect. Here is my long list of things that could be improved:

• Sikuli's documentation on how to run sikuli from the command line is out of date.


• You have to compile the python down to an skl file if you want to run from the command line. Annoying.


• Use sikuli IDE's screenshot mechanism. Other screenshot tools cause trouble.


• Set the similarity number high to prevent false positives. For example at 0.85 similarity, sikuli couldn't tell the difference between the word Ghostbusters and Ohostbusters.


• Creating the screenshots for tests is tedious


• My sikuli script is naive, which means if you have multiple firefox windows open it could fail erroneously


• A popup warning or some always on top window could cause the test to fail erroneously


• If the browser scales the image your test will fail erroneously

This has been indespensible for our application, because it lets us know if we have broken the integration with InDesign. It also gives us a way to test that our InDesign server is behaving. Image recognition is a bit of an edge case when it comes to testing web applications, but it is an interesting tool for testing and one that is highly adaptable. For images generation applications including charts or custom javascript controls I think image recognition would be the only way to test. It would be super cool to have an image recognition gem because then we would be able to do these kinds of tests without calling out to sikuli.

Just for fun I included this last screenshot. It is sikuli IDE's nifty little image recognition troubleshooting tool. Here I have turned down the similarity number to 0.5 and sikuli is showing 3 matches for my test image of the cucumber on the page. The bright red match is the strongest and the pink and purple are weaker.

Lines of code vs. Lines of test

My last project was a rescue mission. A 20k line rails app with only about 34% line coverage from the tests. No wonder the previous team had failed. 5 months later we made a successful rescue, and were up to about 71% line coverage. Watching those stats go from totally-screwed to mostly-un-screwed felt good. Metric_fu provided all the cool and useful graphs for that project: rcov, flog, reek, etc. One graph that was missing was lines of code vs. lines of test. I was sure that graph would have made a big beautiful X marks the spot, because LOC had steadily gone down and LOT had steadily risen. The moment those lines crossed would have been a great opportunity for toasting the health of the codebase.

Well, good news for those of you looking for excuses to toast your codebase. The brand spanking new Metric_fu 1.4 features a froody new LOC vs LOT graph.

I've been using it on my new project. This codebase is healthy, so no X marks the spot action here, but it is still interesting. From our graph it looks like things started out with 2 lines of test for every line of code, and have been working their way towards 3 lines of test for every line of code. (Our rcov % has stayed about 98% the whole time.) Does that mean that as the codebase increases you have to have an increasing ratio of tests to code? Given a sampling of 2 projects I can't really make any useful generalizations. But what I can do is recommend metric_fu to everyone. Graphs are a great way to start talking about how to improve your codebase and your process, and metric_fu makes it easy to get started. And remember, always use your Vanna White hands when you are exhibiting your graphs to your team mates. If it can make a game-show as lame as Wheel of Fortune successful, then it may work for you.

Build your views before working on the controller

For complex forms you save time by building your view before you write tests and code for the action in the controller. For example take the following view code:

- form_for :ad_request, @ad_request, :url => ad_request_feature_path do |f|
  = f.error_messages :class => 'error'
  .form
    #publication.form-item.first
    - f.fields_for :car_selections do |cs|
      = cs.label :headline, "Headline:"
      = cs.text_field :headline

Unless you bleed rails you probably can't guess what the params hash posted to the controller will look like. Fortunately for lazybones like me you don't have to guess. Simply create the view, run up your app in dev mode, and post this form. Of course you get an error, but if you look in your app logs you are rewarded with a real life example of your params hash, like this:

{"ad_request"=>{"car_selections_attributes"=>{"0"=>{"headline" => "Mini for $15,000 Driveaway", "id"=>"1"}}}

Great. Guess what you should do with that nugget? That's right! Use it as the starting point of your controller test:

it "should save headline" do
  car_selection = Factory.create(:car_selection, :id => 1)
  ad = Factory.create(:ad_request, :ad_type => 'retail', :car_selections => [car_selection])
  post :feature, {"ad_request"=>{"car_selections_attributes"=>{"0"=>{"headline" => "Mini for $15,000 Driveaway", "id"=>"1"}}}
  ad.car_selections[0].headline.should == "Mini for $65,000 Driveeaway"
end

This is one of those things that is almost too pedestrian to even talk about, but it can save you time and pain in the long run, especially for the occasional nasty forms with deeply nested objects.