RushCheck
a lightweight random testing tool for Ruby

Blog

RushCheck has been moved to GitHub

(2011-02-24)

The author should say RushCheck maybe not work with Ruby 1.9 because of changes about Proc object and lambda from Ruby 1.6 features. To hack sources as you like, we decide to create a repository of a repository RushCheck at GitHub. Feel free for making a branch of this repository.

As our knowledge, there has been already another implementation of QuickCheck for Ruby; not RushCheck, but original one. Check out "rantly".

Development for RushCheck 0.8 is delayed

(2006-12-04)

Why does the development of Rushcheck seem to be delayed? There are several reasons that I may have to say. The main reason is that I’m now concentrate upon writing a paper whose deadline is Jan 8, 2007, which is not related to RushCheck, but my favorite mathematics. On the other hand, another reasons are related to RushCheck. I’ve two talks about the library at local seminars in Japan. One has already given and you can see the slide of talk. Another talk will be given recently, and I’m now write the stage directions. Recently, Bjorn Bringert announced that new QuickCheck is developed and added some attractive features. I saw an feature such as shrinking fail test cases at John’s room when I visited Chalmers last time, and I like it very much. Dons Stewart announced another new concurrency features for QuickCheck. Then, the next RushCheck might be added similar new features hopefully (in RushCheck version 0.9?).

At the end of this blog, I want to say what is left to ship the next version. Fixing bugs are most important things. I start to write testing code for RushCheck by using itself and RSpec. Therefore I will postpone shipping the next release until the next year. However sending a patch or bug report is always welcomed!


RushCheck 0.7 is shipping out

(2006-10-28)

Now I’m writing testcases for RushCheck by RSpec. Because some tests or specifications requires many instances, I think they should be generated randomly as random testing. I’m very happy because now RushCheck can be used to test RushCheck itself! This means the status of RushCheck becomes more stable.

Though I’ve not finished writing test cases, I decide to ship the next version 0.7 because “reck wreck”, who tried RushCheck, reported a bug at the discussion forum in rubyforge. This bug was fixed now and hopefully another bugs are also fixed.

Until the next release of ver 0.7, hopefully until the first weekend of November, I will finish to write test for every functions in RushCheck. Please send me a bug report if you find it. Your comment encourages me greatly!


Combine another testing frameworks and RushCheck

(2006-10-08)

In this article, I will introduce how to combine Ruby’s unit testing ‘test/unit’ and RushCheck. I have also tried RSpec which supports BDD (Behaviour Driven Development) style checking specifications.

To watch the testing code, we need a target of tests. I quote the following simple example from the tutorial of RSpec.

# stack.rb
class Stack
  def initialize
    @stack = []
  end

  def empty?
    @stack.empty?
  end

  def push(item)
    @stack.push item
  end
end

This class Stack is not implemented fully, and it’s only a subset of Array, but don’t mind.

To test the Stack with ‘test/unit’, we have to write testing codes. Here is an usual unit test cases without RushCheck:

# test_stack.rb
require 'test/unit'
require 'stack'

class TC_Stack < Test::Unit::TestCase

  def test_empty_stack
    stack = Stack.new
    assert stack.empty?
  end

  def test_not_empty_after_push
    stack = Stack.new
    stack.push 1
    assert(! stack.empty?)
  end

end
The result of the above test codes:
% ruby test_stack.rb
Loaded suite test_stack
Started
..
Finished in 0.004112 seconds.

2 tests, 2 assertions, 0 failures, 0 errors

Because testcases are too small, it takes only 0.004112 seconds. However, if we are not satisfied that the code tested only one case ‘stack.push 1’, then we can apply RushCheck to it.

begin
  require 'rubygems'
  require_gem 'rushcheck'
rescue LoadError
  require 'rushcheck'
end

def forall(*cs, &f)
  assert(RushCheck::Claim.new(*cs, &f).check)
end

require 'test/unit'
require 'stack'

class TC_Stack < Test::Unit::TestCase

  def test_empty_stack
    stack = Stack.new
    assert stack.empty?
  end

  def test_not_empty_after_push
    stack = Stack.new
    forall(Integer) do |item|
      stack.push item
      assert(! stack.empty?)
    end
  end

end

The difference between two test codes are: (1) require ‘rushcheck’ (2) define ‘forall’ (3) test many cases with the forall method.

Results:
Loaded suite test_stack
Started
.OK, passed 100 tests.
2 tests, 101 assertions, 0 failures, 0 errors

The above result shows that just 100 assertions are tested in the second case and the total time takes 0.185286 seconds. Of course it is longer than the first example, but still less than 1 seconds.

The ‘forall’ method is a sugar, i mean it is an alias of RushCheck::Claim.new, and make the code easy to read.

I’m satisfied that after combining unit tests and random tests, we get usuful automatic testing in lightweight way.

However, I heard more nicer testing framework from IRC #ruby-lang@freenode channel (thanks to all), called RSpec. It’s installable with RubyGems in a moment, like
% sudo gem install rspec

The BDD-style testing code is more readable than unit testing code. Here is another testing code with RSpec and without RushCheck:

# stack_spec.rb
require 'stack'

context "A new stack" do
  setup do
    @stack = Stack.new
  end
  specify "should be empty" do
    @stack.should_be_empty
  end
end

context "An empty stack" do
  setup do
    @stack = Stack.new
  end
  specify "should not be empty after 'push'" do
    @stack.push 1
    @stack.should_not_be_empty
  end
end

The code looks like an English phrases. Excellent. To check, say in BDD-style we use ‘check’ instead ‘test’, we use the ‘spec’ command which is distributed by RSpec module.

% spec stack_spec.rb -f s

A new stack
- should be empty

An empty stack
- should not be empty after 'push'

Finished in 0.007394 seconds

2 specifications, 0 failures

The above results of checking specifications are also well-formated! In my opinion, the advantages of RSpec are (1) readable testing code (not testing but checking specifications) (2) also readable outputs (3) lightweight implementation based on test/unit.

As we have seen, we can combine RSpec and Rushcheck in the same way for unit testing. Here is the last code snippet in this article.
# stack_spec.rb
begin
  require 'rubygems'
  require_gem 'rushcheck'
rescue LoadError
  require 'rushcheck'
end

require 'stack'

def forall(*cs, &f)
  RushCheck::Claim.new(*cs, &f).check.should_equal true
end

require 'stack'

context "A new stack" do
  setup do
    @stack = Stack.new
  end
  specify "should be empty" do
    @stack.should_be_empty
  end
end

context "An empty stack" do
  setup do
    @stack = Stack.new
  end
  specify "should not be empty after 'push'" do
    forall(Integer) do |item|
      @stack.push item
      @stack.should_not_be_empty
    end
  end
end
% spec stack_spec.rb -f s
A new stack
- should be empty

An empty stack
OK, passed 100 tests.
- should not be empty after 'push'

Finished in 0.211421 seconds

2 specifications, 0 failures

See also ‘data/examples/rspec’ in the distribution for details.

Note: The ‘forall’ trick is not enabled until ver 0.5. Please install the current release of RushCheck if you have old one. At the current release (ver 0.6), the restriction about the number of arguments of Assertion.new is deleted for this trick at unit testing.


Shipping out RushCheck ver 0.4

(2006-10-06)

I am happy to announce the next version of RushCheck. I have done many bug fixes and change some main interfaces. Especially, the notation of pre-condition with guards is changed. From this version, the number of a block of assertion must be equal to the number of arguments of Assertion.new.

before:
Assertion.new(Integer, String) do |x, y, g|
  g.guard { # a pre-condition here }
  ...
end
after:
RushCheck::Assertion.new(Integer, String) do |x, y|
  RushCheck::guard { # a pre-condition here now! }
  ...
end

Sorry for this incompatibility, but I think it becomes more simpler and makes a sense. Since the tutorial is now translated into a hypertext, it is easier to refer notations.

At the end of last month, I visited Chalmers university for a week to join a internal meeting for developing a proof assistant Agda. The end of that week, on Friday afternoon, Prof. John Hughes was kindly to give a chance to discuss about QuickCheck. He talked about recent development and success of Erlang’s QuickCheck and showed me several examples. Because they are so impressive, I will steal and implement them into RushCheck from now on (in not this version, but the next, hopefully). I showed him the previous implementation (ver 0.3) of RushCheck and slides(PDF) for RushCheck ver 0.3. I thank not only him but also the members of AIM5 that we had a discussion about QuickCheck. In the discussion I got many good idea.

I want to introduce this random testing technique to Ruby’s developer. Even it is better to write testcase in a functional language, such as Erlang or Haskell, however, for Ruby’s programmer, it is easier to write testcase in Ruby itself. I wish RushCheck helps Ruby testing programmer. The next chance may be Kansai OpenSource workshop, and I will submit my talk at Kansai-area in Japan. Please give me a comment when you try to use this library, thanks!


moved to darcs

(2006-08-18)

Because I prefer darcs than CVS, RushCheck has moved to a darcs repository. No changes should be committed to the CVS repository at RubyForge from now on. You can get the latest development version of RushCheck by

% darcs get --partial http://rushcheck.rubyforge.org/repos/rushcheck

Then you can get all new patches from the main repository whenever after cd the repository by

% darcs pull -a

Unfortunately, now RubyForge does not support any darcs feature, (I should try to request the support team, anyhow), there is no web interface to watch the changes of the repository. Another disadvantage is the percentage of the activity of the project at the project page may be in the decrease. However, I don’t mind it and the project is active while this blog is updated. :)


RushCheck new

(2006-08-09)

2004-05-11, Prof. John Hughes visited our laboratory and gave his talk “Random Testing with QuickCheck” at the 1st workshop on “Types for Verification”. Because his talk was interesting for us, Sakai-san and I discussed to him in detail at the next day.

At that time, I started to implement QuickCheck in Ruby, however, hacking was interrupted so often. Then suddenly I lost any files by disk crash and I have no backup unfortunately.

2 years later, I restart the project again. Even RushCheck is in alpha-stage, I’m truly happy to ship it on RubyForge.


Written by Daisuke IKEGAMI
HTML and CSS Design by Nicolas Fafchamps
Generated with Rote and Rog