Unit Testing in Haskell
QuickCheck is the gold standard for testing pure functional (side-effect free) code in Haskell. It allows you to assert properties of functions, such as "this function returns a non-negative result for any input." QuickCheck generates input cases and checks the proposition automatically.
HUnit is the usual xUnit clone for Haskell. It deals with actions (Haskell’s term for functions having side-effects). You perform some action then assert the resulting state has been correctly mutated.
It is mighty convenient to use both for TDD on any non-trivial program.
Test Collection
Relying on the coder to remember to add tests to suites will eventually produce orphaned suites. Then comes the false confidence: I ran all the tests and they passed, and I know it doesn’t break assertion X, because there’s a test for that. I’ve seen it.
As OneSock has gotten a little bit bigger (it’s still small, don’t worry), I start to worry about this. I did a bit of research and found some test-collection tools:
quickcheck-script
This script will scan the source files supplied on the command-line for QuickCheck assertions and run them. It is very useful and easy to use. The drawback here is that it will not run my HUnit tests.
hstest
This program will scan for QuickCheck1 and HUnit tests in all sources in the current directory and run them. The drawback is its current lack of support for QuickCheck2. The philosophy is right for what I want to do - have a simple "brain idle" command for my TDD cycle that runs all the tests.
test-framework
This package provides a class with instances for HUnit, QuickCheck1, and QuickCheck2. For the most part, manual collection is required; however, there is a test-framework-th package which uses the TemplateHaskell extension to provide a meta-function that scans the current source file for tests and collects them.
The Decision
I’ve decided to go with hstest.
quickcheck-script doesn’t run HUnit tests.
While test-framework seems to have more mindshare and also seems to be well-thought-out, there is no script to automatically collect tests. The TemplateHaskell trick is nice; however, introducing another layer of complexity just for test collection is a bit overwhelming at this early stage-especially because it doesn’t solve the original problem: now a programmer must remember to register all source files rather than all tests in a test-runner program.
To work around the current lack of support for QuickCheck2, you can supply parameters to hide and expose packages. I’ve written this shell function:
function t() {
hstest --expose-package=QuickCheck-1.2.0.0 --hide-package=QuickCheck-2.1.0.3
rm -f hugsin
return $?
}I’ve found that my current few properties run unaltered under QuickCheck1.
I’ve also specified QuickCheck1 in my cabal file for the moment so that I don’t use features that hstest doesn’t like. This provokes a warning:
jfelice@flarp64:~/src/onesock$ cabal configure
Resolving dependencies...
Configuring OneSock-0.1...
Warning: This package indirectly depends on multiple versions of the same
package. This is highly likely to cause a compile failure.
package OneSock-0.1 requires QuickCheck-1.2.0.0
package Crypto-4.2.1 requires QuickCheck-2.1.0.3
jfelice@flarp64:~/src/onesock$No build issues occur. Maybe the linker is eliminating all QuickCheck references (which are unreachable from main).
