Category: Testing

Wait for it…

On my current Rails project, I have been working on constraining “check-ins” to once daily or once weekly. From a Cucumber scenario perspective, this means using Timecop to travel to a specific time/date, popping a jQuery dialog, entering some data, and pressing the Submit button. This procedure is then immediately repeated, traveling to a different point in time. Upon completion of the successful path, I was asserting that both “check-ins” took place by looking at the model/database. Unfortunately, this would fail quite randomly. It was as if the second time travel was taking place before the first check-in was being written to the database. If so, this would cause the second check-in to fail the daily or weekly constraint during validation.

What I needed to do was WAIT — wait for the check-in dialog to close before the second time travel.

It took some poking around, but I finally settled on the following step definition:

When /^I wait for (.*)to finish$/ do |sugar|
  page.wait_until do
    done = page.evaluate_script("window.jQuery.active == 0")
    putc done ? '*' : '+'
    done
  end
end

This allows me to write a step such as:

And I wait for the check-in to finish

This step prints a + every time through the loop while the page is still active, and then a * when it’s done. (Obviously, this is window dressing and not integral to the success of the step.) Not only did this fix the scenario failures I had due to the race condition, it also got rid of the random “connection reset” errors I was seeing.

Rails (3?): Skip Test

In updating an existing Rails app to run under Rails 3 (beta 4), I realized you could indicate that a certain test case should be skipped.  Literally, just add “skip” inside of the test (I added it to the top of one of mine as it was actually a work in progress) followed by an optional message.  For example:

test "should take the warp core offline" do
  skip
  warp_core.dismount
end

Very useful.

Running installer from cruisecontrol.rb

In the current Mac OS X / Cocoa project I’m working on, we need to be able to run an installer from code.  Naturally, I’ve written tests for this, which work fine on my own machine.  However, when I checked in the code, the test would literally hang at the point where it should be executing the “installer” command.

The continuous integration system we’re using is cruisecontrol.rb which is started at boot time by launchd under a non-root user running on a Mac Mini.  Consequently, when the test is run that fires up the installer command, it’s doing so as a non-root user.

This no workie.

I’m sure there’s some sort of interesting and obscure way to circumvent this, but the general size of things is that you can’t run the installer command unless you’re using sudo or you’re root.  Since running sudo in the above context is kinda obtuse, and since we’re not going to run cc.rb as root, it looks like these particular tests are now out of the automated test suite and into the manual test suite.  At least there’s an explanation now!

CruiseControl.rb+git (+xcodebuild!) on Mac OS X

With some encouragement from my colleagues, I expanded the original post here to be a more soup-to-nuts recipe for setting up CC.rb with git and xcodebuild on Mac OS X. Thanks, guys!

I am currently working on a cross-platform project which has multiple targets that need to build on Mac OS X. On my previous project, we were using CruiseControl.rb on linux to watch a Subversion repository for a Rails project and run all the associated tests upon checkin. Since we wanted to be like the Cool Kids on this new project, we of course decided to use git for source code management.  We use gitosis to manage multiple remote git repositories under a single user.  However, you can use any approach you want for the remote repository management as long as you can compose a git URL that points to it.

Assumptions:

  • You have a Mac that is probably not your development machine that will serve as a Continuous Integration (CI) server.
  • You have a remote git repository that the CI machine can monitor and pull changes from.

 
Procedure:
It took a little doing to make CC.rb play nice on Mac OS X.  In addition to the typical CC.rb installation, getting CC.rb to launch at system startup using launchd took the proper file in the proper location. Nevertheless, the whole setup is pretty darned straightforward.  Here’s how I did it:

  1. I created a user named “continuousintegration.”  You can name yours anything you like, or use an existing user.  Log in as this user.
     
  2. Download and install CruiseControl.rb.  I unpacked it into the the home directory of the continuousintegration user.  So, the absolute path to the CC.rb installation directory in my case is:  /Users/continuousintegration/cruisecontrol.rb.
     
  3. Create the work directory for CC.rb.  In my case, I use /var/cruise as the work directory.  This required admin privileges to create; just make sure that your CI user has access to this directory. 
    sudo mkdir /var/cruise
sudo chown continuousintegration /var/cruise

     

  4. Add your git repository to CC.rb. 
    cd ~/cruisecontrol.rb
./cruise add <project name> --url <git repository url> -s git

     

  5. At the root level of your git repository, add a file named cruise_config.rb. This provides some useful configuration information so that CC.rb knows what to do in order to build your project as well as whom to notify when the build fails. You provide the build command for CC.rb in cruise_config.rb by assigning to project.build_command. I actually made my build command point to ./ci_build.sh so that I can do anything I want for the build in a separate shell script named ci_build.sh in the git repository root. In my case, I am actually running xcodebuild to build a target I created in my Xcode project; you can do literally anything you want here, e. g. make check, rake… whatever! Witness my cruise_config.rb and ci_build.sh files. (If you create a ci_build.sh or equivalent file, be sure you make it executable.) Make sure you git push your changes to add the cruise_config.rb (and ci_build.sh if you choose to do so).
     
  6. You should now be able to start up CC.rb and look at the dashboard to see your project. 
    ./cruise start

    Browse to http://localhost:3333

  7. Wait! You’re not finished yet! There’s one more step: setting up CC.rb to start every time the system starts up. This is accomplished by creating a plist file in /Library/LaunchDaemons which starts up CC.rb. (For more information on what’s going on here, see the Apple documentation or man launchd.)
     
    Download this plist file and copy it into /Library/LaunchDaemons

    sudo cp localcruisecontrol.plist.txt /Library/LaunchDaemons/local.cruisecontrol.plist

     
    Now when you restart the CI machine, CC.rb should start up automatically.

 
Setting up CC.rb all by itself tends to be pretty easy; it’s getting the daemon to run at system startup as well as getting your build to work properly that tend to be the problems. If you’re having trouble getting things running after following these directions, or if you find errors in these directions, please let me know.

Setting ruby executable path in TextMate

I just recently received an update to TextMate.  However, I discovered that I could no longer Run Focused Unit Test as it would complain:

in `check_gem_dependencies': undefined method `ruby_version' for Gem:Module (NoMethodError)

I got a clue when I noticed in the RubyMate window that the patch level of the ruby interpreter was different from the patch level of the ruby I use from a Terminal window.  Clearly, TextMate was using a different Ruby executable than I was the rest of the time.

I poked around in the Ruby bundle a little bit and noticed references to a variable TM_RUBY.  Looking around some more, I discovered in the Preferences under Advanced there is a Shell Variables tab.  I added a TM_RUBY variable there and pointed it to the ruby I wanted it to use (/usr/local/bin/ruby in my case).  Restarted TextMate, et voilà — Run Focused Unit Test works perfectly again.

CppUnit Head Start

I was building a new project to do some experiments with wchar_t this morning. (Look here for the reason.) It seemed to me it took way longer than it should simply to get a CppUnit project setup in Xcode with a single test case.  (It was primarily due to C++ and pilot error; I gotta say I’m gonna get a cold constantly moving between Ruby and C++.)

Consequently, I have built a skeleton CppUnit Xcode project. This provides you with a single class with a test method which asserts FALSE, demonstrating that the test process is working properly. It’s up to you to do something interesting with this head start.

Prerequisite for all platforms: install CppUnit.

If you’re on Mac OS X, with any luck you can simply build-and-go and you’ll see the results of the test (failure) in the debugger console.

If you’re on Linux, just disregard everything Xcode-related; you can still build and run the code from the command line (see the README file).

If you’re on Windows, gosh I’m sorry. 🙂 Nevertheless, you should be able to load these files into Visual Studio, set your header search directory to wherever CppUnit is installed, add the CppUnit library to your project, build and run.

Please let me know if you have any problems with this so I can keep it updated and hopefully save others a little time during CppUnit ramp up.  Now GO WRITE A TEST!

Running ActiveRecord Tests

I have the occasion to need to extend ActiveRecord to support inserting and updating to nvarchar columns in MS SQL Server.  In order to accomplish this, I wanted to be able not only to run but also to extend the existing AR tests against SQL Server. So, I edited my ODBC data sources in /etc/odbc.ini and added a new connection underneath activerecord/test/connections for my database. Then, I needed to run the test/fixtures/db_definitions/sqlserver.sql script to have the tables in place. This wasn’t working out of the box with sqsh because of the format of the SQL script with semicolons as statement terminators. To solve this problem, I added the -L semicolon_hack=1 option to my sqsh command line, and the script ran no problem.

$ sqsh -U user -P password -S server -D database -L semicolon_hack=1 -i sqlserver.sql