The Golden Middle Path - a blog by Amit K Mathur

Rails testing 101

However beautiful the strategy, you should occasionally look at the results. — Sir Winston Churchill

As promised, firstly, the whys

As you probably know by now, Ruby is an interpreted language and along with
the upside of faster development cycle, there is a downside of not having a
compiler: there is no syntax or type checking after you write the
code. Executing the code is the only way to know if its valid and tests are a
great way to execute code.

Tests also provide you a guarantee against any regression in your code. If you
have ever added a “feature” which broke something which was always working in
the past, you know what regression is. Tests allow you to guard against such
breaking changes.

A gentle introduction

If you have been testing your app through the browser (of course you do, silly
me) or by trying things out in the console, writing test takes only a little
bit more effort than that. Writing tests basically automates your steps and
thus saves your from repeating the same steps in the browser or console,
later. If you have never tried testing your Rails app, this is how you should
start:

First, prepare your test database:


$ RAILS_ENV=test rake db:create 
$ rake db:test:prepare

Next, write some fixtures: Fixtures are the initial contents of the test
database, before a test runs. Just like you keep some data in your development
database while you are testing your app through the browser, fixtures are a
way to populate some seed data in the test database. Rails automatically
creates a fixture file for each model in your project.

Let’s say you are building a bug tracking system where you have a model called
Ticket. The fixture file for Ticket should contain an entry for each row that
we want to insert into the database. Following fixture file has two
entries. Each entry is given a name. the name is significant while setting
associations.


# fixture for Ticket
# file: test/fixtures/tickets.yml
bug_filed_by_john_customer:
  title: "Signup form does not retain previously filled data on user error"
  description: "some more explanation here"
  type: "bug"
  state: "open"
mary_managers_question:
  title: "How can I change the color of the shirt once it has been added to the cart "
  description: "I want to ..."
  type: "question"
  state: "assigned"

Fixtures are as simple as they seem – each of these records gets loaded as
rows in the table. There is just one thing to learn here: do not initialize
the id columns and if you have associations, the names you have given to the
rows can be used to fill in the foreign key columns. e.g. suppose, you also
have a Update model which captures updates on tickets by different users.


# fixture for Update
# file: test/fixtures/updates.yml
question_by_peter_developer:
  ticket: bug_filed_by_john_customer
  description: "please provide steps to reproduce"

response_by_john_customer:
  ticket: bug_filed_by_john_customer
  description: "here's how you can see the problem for yourself ..."

So, specifying the row’s name (bug_filed_by_john_customer) would populate
the foreign key (ticket_id) in updates table correctly.

Various types of tests

Rails recommends that you divide you tests into three categories. Unit tests,
which test your models; Functional tests, which test controllers and
integration tests, which tests almost the whole application end to end. We
will learn about all of these.

In the directory test/unit you would find a file corresponding to each model and similarly in test/functional a file for each controller.

Unit tests

Writing unit tests in very similar to trying out stuff in Rails
console. Continuing with our example, let’s say the Ticket model is defined
like this:



# file: app/models/ticket.rb
class Ticket < ActiveRecord::Base
  has_many :updates
  validates_presence_of :title
  validates_presence_of :description
  validates_inclusion_of :type, :in => %(bug enhancement question)
  validates_inclusion_of :state, :in => %(open in_progress not_reproducible closed)
end

You should be testing all the code you have added, like the validations above.


# file: test/unit/ticket.rb
class TicketTest < ActiveSupport::TestCase
  test "should have title and description" do
    ticket = Ticket.new
    assert !ticket.valid?

    ticket = Ticket.new(:title => "some title", :description => "some desc")
    assert ticket.valid?
  end
end

Then run the test:


$ rake test

You can of course use the data that you have put into the database through
fixtures. Suppose, your model code put restriction that you couldn’t have two
tickets with the same title:


  test "each title should be unique" do 
    ticket = Ticket.new(:title => tickets(:bug_filed_by_john).title)
    assert !ticket.valid?
  end

As in this test, you can access the models loaded by the fixtures as: table_name(:fixture_name).

While writing code and trying it out, you should add a test whenever you discover a bug. Suppose you got a customer complaint and figured description should be made into a text column rather than a string so that it can hold more than 255 characters. It would be a good idea to create a test for that. It guards against description ever being made into a string again.


test "description can be longer than 255 chars" do
  ticket = Ticket.new(:title => "some title", :description => "a" x 500)
  assert_not_raised ticket.save!
  assert_equal 500, ticket.description.size
end

There are a number of such assert functions. Here’s a list of the useful ones:

  • assert(boolean, “optional message”)
  • assert_equal, assert_not_equal
  • assert_match, assert_no_match
  • assert_nil, assert_not_nil
  • assert_raised, assert_nothing_raised
  • assert_valid(ar_object) # calls valid? on the ar_object
  • flunk(“message”) # always fails
  • assert_difference ‘expr’ do … end, assert_no_difference

Since tests are normal Ruby code, you can use usual techniques to organize your test code eg. Loops, separating out common pieces in a function, etc.


test "ticket when created should have a correct type" do
  valid_types = %(bug question enhancement)
  invalid_types = %(not_reproducible incomplete)

  valid_types.each do |valid_type|
    t = Ticket.new(:title => "some title", :type => "valid_type")
    assert_valid t
  end
end

Functional tests

Functional tests are meant to test controllers. Since, you have already tested the models in unit tests, you should focus on test only the logic that’s present in the controllers. For example, here’s a sample functional test:


# file: test/functional/tickets
test "access should be restricted to only logged in users" do
  get :show, { :id => tickets(:bug_filed_by_john).id }
  assert_redirect_to login_path
  assert_equal "Please login", flash[:notice]
end

As you can see, functional tests give you a method called get (also post, put, head and delete) using which you can simulate a web request to an action. Then, use the assert methods to check up the response. On top of what you saw with unit tests, functional tests also have these additional asserts:
assert_response
assert_redirected_to
assert_template

For the most part, views are not tested here. You can test them (see assert_select) but it can get cumbersome.

Integration tests

Integration tests simulate a continuous session between one or more virtual users and our application. It gets the closest to how the application will be used in production.

In integration tests, you can send in requests, monitor responses, follow redirects, and so on. Ideally, a integration is something your customer should be able to understand – there are no models and controller and other internal details.

Instead of using the Rails build in integration tests, you can use webrat to make writing the tests lot more easier as it lets you describe a test in pretty much the same steps as you would do things with a browser.


$ sudo gem install webrat

In test/test_helper.rb, right at the end of the file, add this:


require "webrat"
Webrat.configure do |config|
  config.mode = :rails
end

Rails does not generate any integration test skeleton automatically. So, use the generator:


$ ruby script/generate integration_test ticket_life_cycle


# file: test/integration/ticket_life_cycle_test.rb
  test "ticket life cycle" do
    # a users signs in
    visit new_session_path
    fill_in "Email", :with => "john@customer.com"
    fill_in "Password", :with => "john's password"
    click_button 'Sign in'
    assert_redirected_to tickets_path

    # creates a new ticket
    visit new_ticket_path
    fill_in "title", "new problem..."
    fill_in "description", "some description about it..."
    click_button 'Save'

    # someone adds an update to that and marks it fixed

    # customer adds another update that it works

    # engineer closes it
  end

As you can see, with webrat you can write the tests in a clear declarative
style. It works like a web crawler similar to Mechanize or Watir. Webrat also
does some verifications on the page. For example, here’s what click_button does
according to the documentation: click_button verifies that a submit button
exists for the form, then submits the form, follows any redirects, and
verifies the final page was successful.

Here are some useful webrat commands:

  • visit
  • fill_in “name_or_label”, :with => value
  • click_button “name”
  • click_link “name”
  • select “option”, :from => “name or label”
  • choose “id of option”
  • attach_file
  • check “name”
  • reload

All the asserts you have seen earlier, like assert_response, assert_template
etc. are also available. You can also access the database, like in unit tests,
to verify stuff.

More testing

There are some more kinds of testing you may want to. Most importantly,
performance testing, where you determine how fast your application is
performing and where are the bottlenecks in getting a higher performance. Put
it simply, you first have to do benchmarking to determine your application’s
current performance and then run a profiler to see the relative performances
of methods for that run. Rails also allows you to test routes and mailers.

Additionally, you can do browser testing to determine whether it runs
reasonably in all major browsers.

Ready to move ahead

There are few more tests related concepts to be familiar with:

Stubs: Often you have a module or over the network call in your application
that you don’t want to invoke everytime the tests are run. e.g. Credit card
gateway or an external storage. Stubs allow you to mask those calls. Create a
file like test/mocks/development/file_name.rb where file_name.rb should be the
same name as model, controller or lib file that contains the class you are
trying to stub out. In this file, just reopen that class and refine the
relevant methods as stubs.

Mocks: As you can see the functional tests also load the fixtures and
execute the model classes, although, models have already been tested in unit
tests. So, you can substitute model classes by mocks and avoid loading
fixtures and make the tests run faster. On top of this, mocks check if they
are used as specified or else they raise an error, giving you an additional
level of testing. Have a look at Mocha if you want to use mocks.

Once you have mastered writing tests, you can try to write your tests before
you write the code. This is similar to the idea of developing HTML/CSS mocks
before writing backend code for your application. If you write tests first,
you would be writing the code to a predetermined specification. This practice
is called “Test driven development” or TDD.

Now, some people have gone ahead with the idea of tests serving as
specification and called them, well, specs. This style of development is also
called “Behavior driven development” and most popular framework that supports
this is Rspec.

There are several other testing frameworks you can try e.g. Shoulda and
Zentest. There are some useful plugins like Factory Girl, Machinist (both are replacements for fixtures) which makes writing tests easier.

There are also tools which help with testing like autotest which runs your
tests automatically whenever the code changes. Of course, continuous
integration packages also run tests.

If you are interesting in seeing how much testing you have been doing:


$ rake stats

will show you how much test code you have written compared to application code. For more serious mesurement you can try calculating code coverage using rcov.

Tips

  • Aim of developer testing is not so much to find bugs – if its does, consider
    that a bonus. For a complex application, you should definitely prepare a
    test plan and do more comprehensive testing, preferably by involving a test
    engineer.
  • Organize your test code. Use setup method, create helper methods in test classes
  • Keep each test method small and test just one thing in one test
  • If you are following RESTful architecture and putting all your domain logic
    in models, write unit tests and do integration testing
    thoroughly. Functional testing can be de-emphasized.
  • In the beginning, don’t bother testing pathological cases, start simple.
  • For integration testing there are a variety of choices. e.g. use selenium if you are already familiar with it.

Conclusion

Testing a Rails application using the built-in testing framework is easy and
fast. Although there are lot of things to learn in testing, it is easy to
start small and build gradually as you gain more knowledge and
experience. Happy testing.

Share:

Post a comment


(Formatting help)