Effectively Debugging Capybara Feature Specs

Capybara/Selenium feature specs can be one of the most frustrating types of tests. There is a lot of unexpected behavior and many pieces to keep in mind. Code runs in separate threads, and both AJAX behavior and Capybara find methods create timing issues. The aim of this post is to describe a workflow that makes debugging more efficient.

1. Understand Capybara waiting behavior

Default waiting behavior is the key aspect to understand in writing Capybara specs. The software consultancy thoughtbot has an excellent blog post on writing asynchronous integration tests with Capybara. As the post makes clear, you can use the default wait behavior of many Capybara methods to build a more reliable test that fails less often due to timing issues. For example:

expect(find(".user")["data-name"]).to eq("Joe")

The above method lacks waiting behavior since the attribute [“data-name”] is immediately requested. But the method below gains the benefits of default waiting behavior by using a css selector to find the element on the page.

expect(page).to have_css(".user[data-name='Joe']")

If Capybara cannot find an element immediately, it will keep retrying for a default of two seconds before the spec fails. You can configure the default wait time, which was recently renamed for semantic purposes to default_max_wait_time. Most find methods also have an option that you can pass to configure the wait time. find('.selector', wait: 5)

2. Use break points

I use pry for pretty much every aspect of development as well as for writing tests. My guess is that most developers write Selenium specs by repeatedly writing a few lines of code and running the test. Breakpoints give you a more efficient workflow — just insert a breakpoint, write some code and check that it works before adding to your test. You get a much faster feedback loop, and avoid the guessing and checking that commonly happens with selecting elements on a page. Check out the gif below; it demonstrates the coding process from within a pry breakpoint. Again, just run a command in pry; if it works, copy and paste it into your text editor.

3. Tail the logs

There are many benefits to tailing the test logs. You can see errors that don’t bubble up to the UI layer, providing more accurate information when your test fails. You can also see the actions performed on database records and associations. Watching the log is easy — run tail -f log/test.log.

4. Reload your objects before asserting object values

I often pair with teammates to help debug a spec. Too many times, we’ve gotten stuck on Capybara code that was written perfectly — except for one simple method called reload. Let’s check out an example:

fill_in 'name', with: 'David'
click_button 'Save'
expect(user.name).to eq 'David'

Although the above spec seems fine, it will actually fail. Why? The ‘user’ object loaded in memory hasn’t retrieved or reloaded the new data that we modified in the database. Adding a user.reload solves this problem.

fill_in 'name', with: 'David'
click_button 'Save'
expect(user.reload.name).to eq 'David'

Side Note: Some may argue that you shouldn’t test your database in a feature spec. After all, a feature spec’s role is to mimic the user’s perspective visiting a page. We do tread quite lightly on database actions in our feature specs. Most often, we’ll do a light check like this (assuming the click_button('Save') is saving a new user to the database)

expect { click_button 'Save' }.to change { User.count }.by(1)

5. Test Failure Screenshots

We run our test suite on TravisCI. Sometimes, specs that pass on a developer’s computer will fail when being run by TravisCI. This can happen for a variety of reasons: timing issues related to hardware differences, poorly written specs that can have side effects, etc.

It can be very difficult to debug these specs. One problem is the lack of tools. You could use puts statements around failing code and watch your TravisCI server logs. This can be time consuming and you’ll still have no idea what’s going on in the view.

A useful alternative is to take a screenshot of the page on failure and to use the travis-artifacts library to upload it to a storage service like Amazon S3. The documentation on setting up this tool? Let’s say not so hot. Perhaps we can talk about the process in an upcoming blog post.

UPDATE (03/13/18) - We no longer use TravisCI and have been using Solano CI for about two years now. Solano automatically takes a screenshot of the page on failure, and you can see the screenshot on their UI.

We use the Selenium driver when developing our tests. This lets us view how the test runner interacts with the page. For performance reasons, we switch to the capybara-webkit driver once we’re happy with the test.

Tell us what you think in the comments and feel free to share some tips that you use during your everyday writing of Capybara tests.

We’re hiring!

Originally published at tech.greenhouse.io on October 7, 2015.