Distributed Tests

At work, I created a Cucumber driven automation suite, that is pretty awesome but slow. It takes one browser 45min to complete all 110+ tests.  Compound that with other browsers, it gets expensive to run the tests.

I decided on a grid solution to distribute tests in parallel.  There are a few other things that can be done (such as using gem's to run multiple tests per core) but I started here.

Setting Up The Grid

I looked at WatirGrid, but it wasn't working for me.  I couldn't get it to talk to the VM's i was given on a blade. 

So instead, I set up Selenium Grid... it works great with Watir-Webdriver and Jenkins.  Here's my flow:

First I opened the appropriate ports on the VM's and physical boxes used in the distributed tests.

One VM was designated as the selenium grid hub.  I downloaded Selenium Grid 2. I basically ran it as:
java -jar selenium-server-standalone-2.31.0.jar -role hub

On the node machines (Nodes are selenium grid boxes that will do the work), I ran:
Java -jar selenium-server-standalone-2.31.0.jar -role node -hub http://[the hub server]:4444/grid/register

You should get confirmation they are talking to each other... you can also go to your Grid Hub like yourmachine:4444/grid/console and see that you got connections.

Configuring the tests

This was way easier then I thought.  I already abstracted the browser definition to the env.rb file in a before block.  Wherever you define your browser, you just modify it a bit to do this:
@browser = Watir::Browser.new(:remote, :url=>"http://[your selenium hub]:4444/wd/hub", :desired_capabilities=> :firefox)

Now any calls with @browser, go to the hub and it distrbutes the tests.

Configuring Jenkins

Simultaneous tests won't naturally happen when you run cucumber.  You have to drive multiple jobs to the hub.  To do this I use Jenkins.

So I have 7 test suites in Cucumber, each does it's own specific tests.  I basically created 7 separate Jenkins jobs and all they are, are running simple scripts.  So, I have a job that runs cucumber features/announcement.feature  and another set to run cucumber features/call_forwarding.feature.

Thanks to Deepu, who showed me that I can update my Jenkins to use more executors via Manage Jenkins / Configure System, I set them to 7 executors.

For now I kick off each job manually (I'll change this in the future), and each job goes to the appropriate VM.  The round trip average for a test suite to finish is roughly 5min. 

That took my 45min test in one browser, down to 5min!

You can have each machine set up with different browsers as well, so IE10 is tested against a spec box with that install, IE9 with another box with that install, Chrome on another box, and so forth!

Pretty awesome.