How I built My SIP Automation Testing Framework from the Ground Up

This is a pretty long and detailed (long) blog post.  I'll try and break it out later to smaller posts. I've tried to capture all the details I can, so that anyone can set up something like this relatively easily.

My current SIP Automation Framework, uses all open source tools to:

• Generate performance tests with our SIP framework
• Generate reports
• Generate Graphs
• Capture PCAP's during the 

A Tour Around the Framework

Dashboard

My Dashboard looks like this.  I reskinned Jenkins and gave it a bit of a visual polish.  I also added some new links on the left "Get Call Control Logs," "Latest PCAP," "Archived PCAP" and so forth.  In the main section the different SIPP preset tests are listed, with their failure/pass criteria.  At the bottom of the main section are some visual Graphs of the SIPP results.

Left Nav

The left nav has the new items mentioned above.  Call Control is a button to grab the logs off one of our servers.  Automation Home, goes to the QA Automation page.  Latest PCAP loads the last Packet Capture done with tshark.  Archived PCAPs points to a repository of PCAPs.

Main Body

The Main Body lists the jobs for the project.  In this case, "jobs" equates to a preset SIPP scenario with command line parameters.  It also shows the current pass/fail value in the first column, and the history of pass/fails on the far right column.

Bottom Graphs

The bottom graphs are utlizing a plugin to display JPG images. I'm creating the graphs with CLICharts, during the test run.  Jenkins points to these jpg's via the plugin.

Test Detail Page

Inside a SIPP Test (or job), I've modified it a bit to  have a larger graph, and some useful links.

The links are, Historic Data (which is Historic Graph data) and PCAP Captures (historic PCAP captures.) These links basically point to Workspace locations, so Jenkins is already set to show you a file structure like this: 

Tools Used

The tools I used were:

• Jenkins
• SIPP
• JMETER
• CLICharts
• Tshark/Wireshark
• a bit of Shell Scripting

OS Used

I started on Windows, migrated to OSX and finally ended up putting the whole thing on a Centos VM.  OSX couldn't keep up with the pace of SIPP and performance testing for our needs. I would get spikes.  This could be the machine I was using, but either way I didn't want the framework to be running on a machine in my office. 

I opted for a blade server hosting a Centos VM for me.  It's very fast and the SIPP performance spikes went down drastically... indicating it wasn't code related, but test machine related.

Jenkins

I'm using Jenkins CI to maintain all the SIP tests. My reasoning for this was two fold.  

1. SIPP was going to be used as our main SIP test/performance tool. It uses XML and Command Line params.  I wanted to capture test presets that had both of these. Jenkins will let me build a job that runs a SIPP command line with params i need, and specify the scenario file. So everything is maintained together
2. Using Jenkins CI was useful for us, since our developers use Jenkins, they could send a command to run the SIP tests once a build is ready.

To install Jenkins, I've installed it on Windows as well as OSX... When I went with the blade install I had our network admin install Jenkins CI there... I'm assuming he did a yum install jenkins  but i'm not sure.  

Either way you can find more info on installing Jenkins from: http://jenkins-ci.org/

SIPP

To install SIPP on CentOS, I just did a sudo yum install sipp

SIPP is a robust Open Source command line tool to generate SIP calls and traffic. It has two main aspects:

1. Scenario Files
2. Command Line Parameters

SIPP is one of the most complex command line tools I've used.  Hence the need to capture the command line params tha tare useful... as there are so many.  

Scenario files are just XML files that build packets.  You are litterally building out a packet in XML, and then saying what you expect as a response.  

SIPP comes with some default scenario files.  Unfortunatley those scenario files needed to be modified by me to work with our team's development.  There are certain responses we send, or do not send, that don't match the default scenario files.  

I won't get into writing your own scenario file, as I think it's outside the overall scope here of building the framework.  I'll post more on the scenario file creation in a later post.

For more information on SIPP, check out: http://sipp.sourceforge.net/

JMeter

To install Jmeter on Centos, I just did a sudo yum install for apache jmeter

I  have a specific need to update my test framework before SIP Load testing... to do this I use JMeter to quickly send some POST or PUTs to end points to set up the data I need. I think this is uncommon for most people doing SIP testing - so I won't cover this aspect in this post.

CLICharts

To install CLICharts on Centos, I just did the yum install.

SIPP will output it's performance data to a CSV file, if you choose.  I found a very useful unix tool called CLICharts.  CLICharts will take a csv file and do all sorts of analysis to it.  In my case, I use it to generate graphs at the end of a test. These jpg graphs are put in a folder that Jenkins points to and references with hard links.  This way each test run, updates the Jenkins page with the latest graph of reuslts.

For more information on CLICharts check out: http://clichart.sourceforge.net/docs/quickstart.html

Tshark/Wireshark

This was a bit different... you need to do a yum install for wireshark-gnome.  That will install a version of wireshark that is usable from the command line (just doing a yum install wireshark will install it in /sbin and not /usr/bin, so you can't just use it as a commandline)... and of course you get the tshark command line tool.

I use this tool to run during the SIPP test.  Since I know the total duration of a SIPP test, I set the default duration of the Packet Capture to the same duration.  Basically I wrote a shell script that calls tshark, sets a duration, and outputs a file.  The file is also copied to a archive folder and has the date/time stamp added to the file name.  These can be referenced by Jenkins as you'll see further in this post.

Putting It All Together

So if you have all the tools above installed, lets put it all together and get a SIP Automation Framework up and running!

Jenkins Plugins needed:

In Jenkins, install these plugins (SSH Slaves, Sidebar Link, Green Balls, Dashboard View, and if you want to restyle the CSS of your Jenkins, get the Simple Theme plugin.)

Once you've installed those Jenkins plugins, restart Jenkins.

Set Up SSH Slaves

Go into Jenkins / Configuration

Scroll down to SSH  remote hosts

Fill this out with the host that is running Jenkins, and you should have sudo access to this box.

You'll be using SSH to sudo your commands, even though you aren't "ssh'ing" anywyere, this is how Jenkins will log in as you to sudo commands.  

Setting Up A SIPP Job in Jenkins

First, open Jenkins... by default it's your localhost:8080.  

When you have Jenkins up, click "New Job"

Choose "Build a free-style software project"

Give the project a name and click OK.

Click "Advanced" button on the Advanced Project Options.

Then check "use customer workspace" and put the path/directory to your sipp scenario files.

Check off "Execute shell script on remote host using ssh"

in the SSH site dropdown make sure it's the same one you input on "Set Up SSH Slaves" above.

in the Pre build script, type:

cd [your directory to your sipp scenario files]

sudo sipp -s [service/number you want to dial] -sf [path to scenario xml file] -m [amount of calls you want to run] -mi [your ip of the jenkins server (media IP)] -d [pause duration override, if you need it] -trace_rtt -trace_err -stat_delimiter ,

The reason you need to sudo sipp here, is because if you are sending pcap audio, you need to use -mi which creates a live socket. You can't create a live socket unless you are sudo'd.

If you run this, it should work.

Adding tshark PCAP captures

I use a parent project, it kicks off the above as a downstream. Meaning, I have a parent project called "2000 calls @10 calls per min to Integration," that job just kicks off a downstream job... two actually.  The first is what we just created above (the SIPP job) and the second job it kicks off simultaneously is the tshark pcap capture.  They run at the same time.

How you do this is simple:

On the Server:

create a shell script in the folder with  your scenario files.  Input something like this:

sudo tshark -a duration:201 -w sip_test.pcap

sudo mv sip_test.pcap /var/lib/jenkins/userContent/

sudo chmod 777 /var/lib/jenkins/userContent/sip_test.pcap

sudo cp /var/lib/jenkins/userContent/sip_test.pcap /home/jenkins/sipp-3.3/pcap/archived/sip_test$(date +%F-%T).pcap

Ok, so this script will run tshark for 201 seconds.  My SIPP tests runs for 200 seconds.  it outputs a pcap file to this directory.  Then I move it to my Jenkins UserContent folder.  I change the permissions of this file so it can be read by other users. Finally I copy this file, and change the name of it to a date/time stamp and drop it in an archive.

Update this script to fit your specific needs and directory structure, then run it. Make sure it works.

Jenkins Packet Capture Job

Back in Jenkins, create a new Jenkins job, just like before.  Give it a name like "Capture PCAP."  This we'll run a SSH script just like before. But instead of running SIPP, we're going to call that shell script (i.e. sudo /somepath/yourscript.sh)

Ok, back at that parent job I just spoke about... remember, the job that just kicks off jobs?  Scroll down to the bottom and click Add Post Build Porcess and choose to build a project(s).  In the field input your SIPP project followed by a comma, and the job you just made to run the tshark script.

Run the Parent job, and verify that you get the pcap in your workspace.

Linking the PCAP to Jenkins

One plugin we installed for Jenkins was called Side Bar.  This lets us make our own Left Nav links.  Go ahead click Jenkins / Manage Jenkins / Configuration and in the section "Additional Side Bar Links" click add.  If you want to use your own icon, use the uploader to upload it. it will by default upload it to Jenkin's userContent folder.  It will provide you the path after it uploads.

For the Link Text, input whatever you want (I.e. "Latest PCAP File")

For the Link URL, point to the PCAP file location on your server.

Linking to Archived PCAPs

If you want to link to older PCAPs, well you would have had to do the script bit I did above, where I copy the pcap to a archive folder.  Hopefully this folder you copy to is also the same folder used in your Project workspace.  

That way you can go to the project workspace and point to your archive folder.  You do that by clicking on the project, then "workspace" and click the file structure till you get to your archive folder.  Copy that URL. Use that URL as your link.

Adding a Downstream Job to do Graphing and Clean Up

Ok, now that you got sipp to work, it should be dumping a csv file to your workspace. Great. lets graph that.

Creating a script to do Charting

Back on your Jenkins server, go to your workspace you're using for the other jobs (i.e. the folder with your sipp scenarios.

Create a new file

In that file enter something like this:

clichart -cfvl 0,1 *_rtt.csv -o /var/lib/jenkins/userContent/charts/chart1.jpg

cp /var/lib/jenkins/userContent/charts/chart1.jpg /home/jenkins/sipp-3.3/charts/sipcalls$(date +%F-%T).jpg

convert /var/lib/jenkins/userContent/charts/chart1.jpg -resize 510 /var/lib/jenkins/userContent/charts/chart1-s.jpg

mv *.csv archived_csv/

This runs clichart and picks the correct column of the default output of SIPP.  so keep the parameters.  I tell it to use whatever csv file is in the folder, and make a jpg of the results chart.  Remember our previous script auto archives csv files after each run... so only one csv should be there at a time.

Then ti copies the jpb to an archive and makes  a thumbnail using convert.  Convert is a special call from another command line tool that I forget the name of... 

Jenkins running your script

Using the process earlier, create another Jenkins job.  This time give it a name like "Graph Data."  In the job details, check off "Execute Shell script on remote host using ssh.  Make sure the SSH site is correct and in the Pre build script do:

cd [to your directory with the chart shell script]

sudo ./[your chart script.sh]

Save...

Getting Graphs on Jenkins Dashboard

There's a plugin we installed for Jenkins called Dashboard. if you Edit the View of your Dashboard or Main Tab, and scroll down you'll see some options for Dashboard Portlets... You can have "top, left, right, bottom" portlets... and one portlet option is IMAGE.  Choose Image and link to the hard link of your jpt graph from clicharts.

Run the Parent....

you should see:

1. Parent calls two jobs: SIPP and tshark
2. When those jobs finish up, a downstream job charts the data and clearns up csv files in the main folder
3. All images you've hard linked to, should work to show graphs.
4. All PCAP's you hard linked too should show PCAPS
5. your PCAP archive links should now show the archives

My SIP Testing Framework

After reskinning Jenkins, adding in some more functionality, and graphing capabilities, the SIP testing tool looks like this:

Jenkins: Error opening terminal: unknown.

I do a lot of shell scripts called from Jenkins.  Most of the time, I am logged in as a specific user.  In one case, I was logged in as me, to do a sudo command.  I got this error in the response:

Error opening terminal: unknown.

To fix it, I found a site online:
https://bbs.archlinux.org/viewtopic.php?pid=739747

The fix for me was to edit my user's .bashrc file and add:
export TERM=xterm

After that Jenkins ran the script fine.

Chrome Webdriver and Coordinate Errors

If you get coordinate errors with Chrome Webdriver, the most likely culprit (that I've found) is that this is due to a UI element obstructing the view of something you are clicking.

Yes, it's weird.  Normally webdriver would say "click this ID/Name/Class" and it clicks it no matter if a window is up or not.

But with Chrome Webdriver it's a bit different.

Take the screenshot below for example:

In this test, I first had Chrome Webdriver click on that menu.  that menu stays up when webdriver runs (not in a regular Chrome session), so then when it comes time to click "Save" it will throw an exception:

Element is not clickable at point (109, 604). Other element would receive the click:

The trick is to get all  your UI elements back away so you can send that click.  In my Case I either don't pull that particular menu, or I click it again to close it.

Buidling Automatic Graphs of SIPp SIP data

I finally got some automated SIPp results to look like this:

The screenshot above is my Jenkins install.  Each time I run this sip test of 2,000 calls it updates the graph there dynamically.

Here's how I did it.

I create a "Free Style" jenkins job.
In that job, I basically create a call to run a shell script.  The script is the SIPP call.  It has all the parameters for hitting my target box, phone number, and the number of calls.

One parameter on the sipp call is to output the RTT (-trace_rtt) - so after the test a csv file will be in the folder.  I also change the delimter from semi-colon to comma (why do they default to semi-colon???)

I set Jenkins to then fire a downstream job.  This job is another free-style job. it runs a shell script that basically runs CLICharts.

What is CLICharts?  CLICharts is amazing.

To install it, check out:
http://clichart.sourceforge.net/docs/quickstart.html

Basically I created a shell script that runs CLICharts headlessly.  Basically it's like this:
sudo clichart -cfvl 0,1 *rtt.csv -o /var/lib/jenkins/userContent/charts/chart1.jpg

That tells clichart to not care about dates (since I dont have dates it would understand) and tells it to only care about columns 0 and 1.  Finally to output this chart to a folder in userContent (oh yeah, I'm using Simple Theme plugin for Jenkins so this userContent folder is used by that to link to my own graphics.)

Downstream Jobs

So the Downstream Jenkins job just runs shell script.

On a tip I saw online, I used the main project, and edited it to add my own img src tag. This img tag points to the /chart1.jpg above.  Each time the test is run, it updates the jpg and therefore the page has a new graph soon after the test is run.

I also  set up a post build action on this Downstream job... this one emails me and others the csv file and the jpg.

Finally, after the test a second downstream job is kicked off to simply move the csv files to an archive folder (so only one rtt.csv file exists at a time in the main folder.) 

Other Set Up I had to Do

I had to do a few things to make this work... like add the ssh plugin to Jenkins. Even though I'm not
ssh'ing, the ssh plugin sets the user I'm sudoing as on the Jenkins box.  That user was updated in the /etc sudoer's file (via visudo) so that NOPASSWD:ALL was set for that user (so no pass prompt.)

Jenkins running shell scripts as sudo

I was getting this error:

sudo: sorry, you must have a tty to run sudo

To get Jenkins to run scripts that require sudo access, I had to do the following in the sudoer's file:
1. Fist in /etc running visudo I had to update the user Jenkins sudo's as to [user] (ALL)= NOPASSWD:ALL
2. Second I had to turn off the requiretty.  You can do t his per user, or global.

In Jenkins I added the SSH plugin and even though I'm not actually ssh'ing it seems to use this ssh user/pass/key to run sudo.  I set that ot the same host i'm on and the user i want Jenkins to use, that's in the sudoer file.

Jenkins and Cucumber Reports

I've provided details on this set up, but here's a visual walk through:

My Web Automation Jenkins looks like this the image below.  Each job listed (Announcements, Call Forwarding, etc.) is a specific feature file that Cucumber will run - so each entry runs a multitude of tests.  Each Test is sent to a Selenium Grid hub and is farmed to run on one of several VM's.  This simultaneous testing brings down the overall end to end duration significantly (from 45min to 6min) :

When you mouse over a job that has run, it will offer the reports view in the menu, like so:

Clicking on the Cucumber Reports link will first present you with a summary of results:

If you click the test link in the Feature Statistics table, it will break down more details on what failed and passed:

Distributed Tests

At work, I created a Cucumber driven automation suite, that is pretty awesome but slow. It takes one browser 45min to complete all 110+ tests.  Compound that with other browsers, it gets expensive to run the tests.

I decided on a grid solution to distribute tests in parallel.  There are a few other things that can be done (such as using gem's to run multiple tests per core) but I started here.

Setting Up The Grid

I looked at WatirGrid, but it wasn't working for me.  I couldn't get it to talk to the VM's i was given on a blade. 

So instead, I set up Selenium Grid... it works great with Watir-Webdriver and Jenkins.  Here's my flow:

First I opened the appropriate ports on the VM's and physical boxes used in the distributed tests.

One VM was designated as the selenium grid hub.  I downloaded Selenium Grid 2. I basically ran it as:
java -jar selenium-server-standalone-2.31.0.jar -role hub

On the node machines (Nodes are selenium grid boxes that will do the work), I ran:
Java -jar selenium-server-standalone-2.31.0.jar -role node -hub http://[the hub server]:4444/grid/register

You should get confirmation they are talking to each other... you can also go to your Grid Hub like yourmachine:4444/grid/console and see that you got connections.

Configuring the tests

This was way easier then I thought.  I already abstracted the browser definition to the env.rb file in a before block.  Wherever you define your browser, you just modify it a bit to do this:
@browser = Watir::Browser.new(:remote, :url=>"http://[your selenium hub]:4444/wd/hub", :desired_capabilities=> :firefox)

Now any calls with @browser, go to the hub and it distrbutes the tests.

Configuring Jenkins

Simultaneous tests won't naturally happen when you run cucumber.  You have to drive multiple jobs to the hub.  To do this I use Jenkins.

So I have 7 test suites in Cucumber, each does it's own specific tests.  I basically created 7 separate Jenkins jobs and all they are, are running simple scripts.  So, I have a job that runs cucumber features/announcement.feature  and another set to run cucumber features/call_forwarding.feature.

Thanks to Deepu, who showed me that I can update my Jenkins to use more executors via Manage Jenkins / Configure System, I set them to 7 executors.

For now I kick off each job manually (I'll change this in the future), and each job goes to the appropriate VM.  The round trip average for a test suite to finish is roughly 5min. 

That took my 45min test in one browser, down to 5min!

You can have each machine set up with different browsers as well, so IE10 is tested against a spec box with that install, IE9 with another box with that install, Chrome on another box, and so forth!

Pretty awesome.

Tutorial: Up and Running with Cucumber and Watir-webdriver in a few min.

Making a New Project with Watir-Webdriver and Cucumber

UPDATE: By request, I've added some more detail of the full step_def file, feature file and results to the end of this post.

UPDATE FOR REVIEWERS: Idiots and morons at Google have seen fit to unpublish this post. Reason? They say it is malware/virus related. Can you believe that? I thought Google hired smart people. Whoever flagged this doesn't know a virus from their ass. This is web automation, used in testing - you know what happens at places like google. It's also some 10+ years old. It's reasons like this that I am thankful I walked away from Blogger. Now I need to find out how to delete my blogger. I just want the moron who wrote this up to look at how stupid he/she is. STUUUUUPPPPPPPIIIIIIDDDDDDD. This tutorial uses the awesome RubyMine IDE by Jetbrains.

Step 1: Have Rails installed.  Go to your console and do a Rails new [project name]

Step 2: If you get a SSL issue...

Then edit the gemfile and change the https to http for the rubygems.org call

Step 3: Set up depdancies, in the above screen shot I've added a block for watir-webdriver and cucumber, like so:
group :test, :development do
  gem 'cucumber-rails'
  gem 'database_cleaner'
  gem 'rspec'
  gem 'spork'
  gem 'watir-webdriver'
  gem 'gherkin'
  gem 'rest-client'
end

Step 4:  Bundle... you can do this in RubyMine or command line... in ruby mine you do it like this:

Step 5: Configure the project with Cucumber... You do this with the Rails Generator.  You could use the command line, but I'm going to show the steps in screenshots from RubyMine:

It will now configure the project - an update will be displayed at the bottom of the IDE.
You're all set to write/run tests now!

Creating a Sample Test

Step 1: Right-click the Features folder and select "new"

Give it a name ending with.feature:

Step 2: BDD
Once you have a .feature file, lets give it some BDD's... in this example I used a scenario of searching for cucubmers on amazon.  So I typed:
Feature: lets search amazon for cucumbers
  Scenario: A user goes to amazon.com and searches for cucumbers
    Given a user at amazon.com
    When they search for "cucumbers"
    Then the results return for "cucumbers"

Notes the words in quotes?  Cucumber is smart enough to use that as a dynamic replacement.  In other words, if you say later, well i want to test that i can find onions, you replace "cucumbers" with "onions" and the variable is passed through dynamically without rewriting code.

Step 3: Create a step definition file.  This is where we put our code to drive the BDD elements.  You can follow these screenshots:

Step 4: writing the code
I need to now define the steps.  I define a class variable of @browser to be equal to Watir::Browser.new :firefox
Now whenver @browser is used it calls firefox (we can change the browser dynamically later)

So @browser.goto "http://www.amazon.com" goes to that URL.

For the When statement of supplying a value in the search box, I used firebug on amazon.com and saw this for the search input:
<input type="text" autocomplete="off" name="field-keywords" value="" title="Search For" id="twotabsearchtextbox" style="padding-right: 1px;">

We can use any unique identifying element within the input tag.  Note that watir webdriver uses the term "text_field" for inputs that take text... like: @browser.text_field(:name=>"field-keywords").set "cucumber"

We could have also used title, or id.  it's up to you.

My when clause looks like this:
When /^they search for "([^"]*)"$/ do |arg|
  @browser.text_field(:name=>"field-keywords").set "#{arg}"
  @browser.send_keys :return
end

So remember, if a person changes the BDD value in quotes from "cucumber" to "onion" it will pass into this, setting the value in that input field.

For the Then, i did this:
Then /^the results return for "([^"]*)"$/ do |arg|
  @browser.h1(:id=>"breadCrumb") == arg
end

there's a variety of ways you could assert, I choose this. 


There's so much you can do... you can send key commands like Control A, Right click, you can close and open pop ups... use the env.rb file to set default properties like logging in... all sorts of great stuff.

Update: Here's the code for this tutorial:

Step Definition File:

Given(/^a user at amazon.com$/) do
  @browser = Watir::Browser.new :firefox
  @browser.goto "http://www.amazon.com"
end
When(/^they search for "([^"]*)"$/) do |arg|
  @browser.text_field(:name=>"field-keywords").set "cucumber"
  @browser.send_keys :return
end
Then(/^the results return for "([^"]*)"$/) do |arg|
  @browser.h1(:id=>"breadCrumb") == arg
end

Feature File:

Feature: lets search amazon for cucumbers
  Scenario: A user goes to amazon.com and searches for cucumbers
    Given a user at amazon.com
    When they search for "cucumbers"
    Then the results return for "cucumbers"

This is just starter code, as you get going, you'll want to move the browser definition into the env.rb file... and make the browser a variable, so you can easily switch the tests from IE to FF, to Chrome, etc.  I have posts on doing just that. 

PASS and Fail.

If you right click and run the test in RubyMine, it will give you the results in the console. It will look like this:
 1 scenario (1 passed)
3 steps (3 passed)
0m10.526s

Similarly, if you run the tests from the command line using the command "cucumber" in your folder... it will output the same results.

Lastly, if you do want the results as a report, you can use a cucumber reports plugin with Jenkins, and when you build the jobs in Jenkins the output is added to a HTML report.  This last reporting method takes more work to set up and I have a post on setting it up.

Using Rsync to Pull and Monitor remote logs

I have been building a whole Load Test and Monitoring framework within Jenkins.  With that in mind, one of the servers I hit during load, is a Call Control server.  I wanted to collect the logs that were built during the load tests and I came up with this process. I'm sure there are other ideas out there, but this worked nicely for me:

I wrote a script that reads like this:

#!/usr/bin/expect

sapwn rsync -v -e ssh user@myremotebox:/tmp/ccv5/log/callcontrol.log /Users/Jenkins/.jenkins/userContent

expect "*?assword:*"
send -- "mypassword"
send -- "\r"
expect eof

What this script does is it uses rsync to synchronize a file between machines.  In this example above, it would sync the callcontrol.log on a remote box called myremotebox as user user, and update the same log in my local directory of /Users/Jenkins/.jenkins/userContent. 

Because I need to sudo to get access, I want to automatically pass the password prompting.  To get around that I'm using expect. Expect will expect a prompt and respond with some sort of defined response.  Like a password.  It could also be a password key.... but in this simple example i'm just passing a password.

Now, with Jenkins, I do the following:
I set up a build project to run the Load Test, with a downstream project that just executes the shell script above. 

I also used a sidebar add on for Jenkins and hard coded a link to /Users/Jenkins/.jenkins/userContent/callcontrol.log

Therefor, after each run of load, the link to callcontrol.log is updated with fresh content.