RushCheck
a lightweight random testing tool for Ruby

2006/10 Archive

RushCheck 0.7 is shipping out

(2006-10-28)

Now I’m writing testcases for RushCheck by RSpec. Because some tests or specifications requires many instances, I think they should be generated randomly as random testing. I’m very happy because now RushCheck can be used to test RushCheck itself! This means the status of RushCheck becomes more stable.

Though I’ve not finished writing test cases, I decide to ship the next version 0.7 because “reck wreck”, who tried RushCheck, reported a bug at the discussion forum in rubyforge. This bug was fixed now and hopefully another bugs are also fixed.

Until the next release of ver 0.7, hopefully until the first weekend of November, I will finish to write test for every functions in RushCheck. Please send me a bug report if you find it. Your comment encourages me greatly!


Combine another testing frameworks and RushCheck

(2006-10-08)

In this article, I will introduce how to combine Ruby’s unit testing ‘test/unit’ and RushCheck. I have also tried RSpec which supports BDD (Behaviour Driven Development) style checking specifications.

To watch the testing code, we need a target of tests. I quote the following simple example from the tutorial of RSpec.

# stack.rb
class Stack
  def initialize
    @stack = []
  end

  def empty?
    @stack.empty?
  end

  def push(item)
    @stack.push item
  end
end

This class Stack is not implemented fully, and it’s only a subset of Array, but don’t mind.

To test the Stack with ‘test/unit’, we have to write testing codes. Here is an usual unit test cases without RushCheck:

# test_stack.rb
require 'test/unit'
require 'stack'

class TC_Stack < Test::Unit::TestCase

  def test_empty_stack
    stack = Stack.new
    assert stack.empty?
  end

  def test_not_empty_after_push
    stack = Stack.new
    stack.push 1
    assert(! stack.empty?)
  end

end
The result of the above test codes:
% ruby test_stack.rb
Loaded suite test_stack
Started
..
Finished in 0.004112 seconds.

2 tests, 2 assertions, 0 failures, 0 errors

Because testcases are too small, it takes only 0.004112 seconds. However, if we are not satisfied that the code tested only one case ‘stack.push 1’, then we can apply RushCheck to it.

begin
  require 'rubygems'
  require_gem 'rushcheck'
rescue LoadError
  require 'rushcheck'
end

def forall(*cs, &f)
  assert(RushCheck::Claim.new(*cs, &f).check)
end

require 'test/unit'
require 'stack'

class TC_Stack < Test::Unit::TestCase

  def test_empty_stack
    stack = Stack.new
    assert stack.empty?
  end

  def test_not_empty_after_push
    stack = Stack.new
    forall(Integer) do |item|
      stack.push item
      assert(! stack.empty?)
    end
  end

end

The difference between two test codes are: (1) require ‘rushcheck’ (2) define ‘forall’ (3) test many cases with the forall method.

Results:
Loaded suite test_stack
Started
.OK, passed 100 tests.
2 tests, 101 assertions, 0 failures, 0 errors

The above result shows that just 100 assertions are tested in the second case and the total time takes 0.185286 seconds. Of course it is longer than the first example, but still less than 1 seconds.

The ‘forall’ method is a sugar, i mean it is an alias of RushCheck::Claim.new, and make the code easy to read.

I’m satisfied that after combining unit tests and random tests, we get usuful automatic testing in lightweight way.

However, I heard more nicer testing framework from IRC #ruby-lang@freenode channel (thanks to all), called RSpec. It’s installable with RubyGems in a moment, like
% sudo gem install rspec

The BDD-style testing code is more readable than unit testing code. Here is another testing code with RSpec and without RushCheck:

# stack_spec.rb
require 'stack'

context "A new stack" do
  setup do
    @stack = Stack.new
  end
  specify "should be empty" do
    @stack.should_be_empty
  end
end

context "An empty stack" do
  setup do
    @stack = Stack.new
  end
  specify "should not be empty after 'push'" do
    @stack.push 1
    @stack.should_not_be_empty
  end
end

The code looks like an English phrases. Excellent. To check, say in BDD-style we use ‘check’ instead ‘test’, we use the ‘spec’ command which is distributed by RSpec module.

% spec stack_spec.rb -f s

A new stack
- should be empty

An empty stack
- should not be empty after 'push'

Finished in 0.007394 seconds

2 specifications, 0 failures

The above results of checking specifications are also well-formated! In my opinion, the advantages of RSpec are (1) readable testing code (not testing but checking specifications) (2) also readable outputs (3) lightweight implementation based on test/unit.

As we have seen, we can combine RSpec and Rushcheck in the same way for unit testing. Here is the last code snippet in this article.
# stack_spec.rb
begin
  require 'rubygems'
  require_gem 'rushcheck'
rescue LoadError
  require 'rushcheck'
end

require 'stack'

def forall(*cs, &f)
  RushCheck::Claim.new(*cs, &f).check.should_equal true
end

require 'stack'

context "A new stack" do
  setup do
    @stack = Stack.new
  end
  specify "should be empty" do
    @stack.should_be_empty
  end
end

context "An empty stack" do
  setup do
    @stack = Stack.new
  end
  specify "should not be empty after 'push'" do
    forall(Integer) do |item|
      @stack.push item
      @stack.should_not_be_empty
    end
  end
end
% spec stack_spec.rb -f s
A new stack
- should be empty

An empty stack
OK, passed 100 tests.
- should not be empty after 'push'

Finished in 0.211421 seconds

2 specifications, 0 failures

See also ‘data/examples/rspec’ in the distribution for details.

Note: The ‘forall’ trick is not enabled until ver 0.5. Please install the current release of RushCheck if you have old one. At the current release (ver 0.6), the restriction about the number of arguments of Assertion.new is deleted for this trick at unit testing.


Shipping out RushCheck ver 0.4

(2006-10-06)

I am happy to announce the next version of RushCheck. I have done many bug fixes and change some main interfaces. Especially, the notation of pre-condition with guards is changed. From this version, the number of a block of assertion must be equal to the number of arguments of Assertion.new.

before:
Assertion.new(Integer, String) do |x, y, g|
  g.guard { # a pre-condition here }
  ...
end
after:
RushCheck::Assertion.new(Integer, String) do |x, y|
  RushCheck::guard { # a pre-condition here now! }
  ...
end

Sorry for this incompatibility, but I think it becomes more simpler and makes a sense. Since the tutorial is now translated into a hypertext, it is easier to refer notations.

At the end of last month, I visited Chalmers university for a week to join a internal meeting for developing a proof assistant Agda. The end of that week, on Friday afternoon, Prof. John Hughes was kindly to give a chance to discuss about QuickCheck. He talked about recent development and success of Erlang’s QuickCheck and showed me several examples. Because they are so impressive, I will steal and implement them into RushCheck from now on (in not this version, but the next, hopefully). I showed him the previous implementation (ver 0.3) of RushCheck and slides(PDF) for RushCheck ver 0.3. I thank not only him but also the members of AIM5 that we had a discussion about QuickCheck. In the discussion I got many good idea.

I want to introduce this random testing technique to Ruby’s developer. Even it is better to write testcase in a functional language, such as Erlang or Haskell, however, for Ruby’s programmer, it is easier to write testcase in Ruby itself. I wish RushCheck helps Ruby testing programmer. The next chance may be Kansai OpenSource workshop, and I will submit my talk at Kansai-area in Japan. Please give me a comment when you try to use this library, thanks!


Written by Daisuke IKEGAMI
HTML and CSS Design by Nicolas Fafchamps
Generated with Rote and Rog