[Published in Open Source For You (OSFY) magazine, January 2015 edition.]
In this article we shall cover testing of Haskell programs.
HUnit is a unit testing framework available for Haskell. It is similar to JUnit, which is used for the Java programming language. You can install HUnit on Fedora using the following command:
$ sudo yum install ghc-HUnit-develConsider a simple example that follows:
import Test.HUnit
test1 = TestCase $ assertEqual "Test equality" 3 (2 + 1)The TestCase is a constructor defined in the Test data type. The definition is as follows:
-- Test Definition
-- ===============
-- | The basic structure used to create an annotated tree of test cases.
data Test
-- | A single, independent test case composed.
= TestCase Assertion
-- | A set of @Test@s sharing the same level in the hierarchy.
| TestList [Test]
-- | A name or description for a subtree of the @Test@s.
| TestLabel String TestOn executing the above code with GHCi, you get the following output:
$ ghci test.hs
GHCi, version 7.6.3: http://www.haskell.org/ghc/ :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main ( one.hs, interpreted )
Ok, modules loaded: Main.
ghci> runTestTT test1
Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package HUnit-1.2.5.2 ... linking ... done.
Cases: 1 Tried: 1 Errors: 0 Failures: 0
Counts {cases = 1, tried = 1, errors = 0, failures = 0}You can build a test suite of tests with TestList as shown below:
import Test.HUnit
test1 = TestList [ "Test addition" ~: 3 ~=? (2 + 1)
, "Test subtraction" ~: 3 ~=? (4 - 1)
]The ‘~=?’ operation is shorthand to assert equality. Its definition is as follows:
-- | Shorthand for a test case that asserts equality (with the expected
-- value on the left-hand side, and the actual value on the right-hand
-- side).
(~=?) :: (Eq a, Show a) => a -- ^ The expected value
-> a -- ^ The actual value
-> TestThe ‘~:’ operation is shorthand to attach a label to a test. Its definition is shown below:
(~:) :: (Testable t) => String -> t -> Test
label ~: t = TestLabel label (test t)By compiling and executing the above code with GHCi, you get the following result:
ghci> runTestTT test1
Cases: 2 Tried: 2 Errors: 0 Failures: 0
Counts {cases = 2, tried = 2, errors = 0, failures = 0}A failure is reported when there is a mismatch between the expected result and the observed value. For example:
import Test.HUnit
test1 = TestList [ "Test addition" ~: 3 ~=? (2 + 1)
, "Test subtraction" ~: 3 ~=? (4 - 2)
]Running the above code reports the failure in the output:
ghci> runTestTT test1
### Failure in: 1:Test subtraction
expected: 3
but got: 2
Cases: 2 Tried: 2 Errors: 0 Failures: 1
Counts {cases = 2, tried = 2, errors = 0, failures = 1}If our test case definition is incorrect, the compiler will throw an error during compilation time itself! For example:
import Data.Char
import Test.HUnit
test1 = TestList [ "Test addition" ~: 3 ~=? (2 + 1)
, "Test case" ~: "EARTH" ~=? (map "earth")
]On compiling the above code, you get the following output:
ghci> :l error.hs
[1 of 1] Compiling Main ( error.hs, interpreted )
error.hs:5:48:
Couldn't match expected type `[Char]'
with actual type `[a1] -> [b0]'
In the return type of a call of `map'
Probable cause: `map' is applied to too few arguments
In the second argument of `(~=?)', namely `(map "earth")'
In the second argument of `(~:)', namely
`"EARTH" ~=? (map "earth")'
error.hs:5:52:
Couldn't match expected type `a1 -> b0' with actual type `[Char]'
In the first argument of `map', namely `"earth"'
In the second argument of `(~=?)', namely `(map "earth")'
In the second argument of `(~:)', namely
`"EARTH" ~=? (map "earth")'
Failed, modules loaded: none.The correct version of the code and its output are shown below:
import Data.Char
import Test.HUnit
test1 = TestList [ "Test addition" ~: 3 ~=? (2 + 1)
, "Test case" ~: "EARTH" ~=? (map toUpper "earth")
]The expected test output is as follows:
ghci> runTestTT test1
Cases: 2 Tried: 2 Errors: 0 Failures: 0
Counts {cases = 2, tried = 2, errors = 0, failures = 0}In Haskell, one needs to check for an empty list when using the head function, or else it will throw an error. An example test is shown below:
import Test.HUnit
test1 = TestCase $ assertEqual "Head of emptylist" 1 (head [])Executing the above code gives:
ghci> runTestTT test1
### Error:
Prelude.head: empty list
Cases: 1 Tried: 1 Errors: 1 Failures: 0
Counts {cases = 1, tried = 1, errors = 1, failures = 0}Other than the assertEqual function, there are conditional assertion functions like assertBool, assertString and assertFailure that you can use. The assertBool function takes a string that is displayed if the assertion fails and a condition to assert. A couple of examples are shown below:
import Test.HUnit
test1 = TestCase $ assertBool "Does not happen" TrueExecuting the above code in GHCi, gives you the following output:
ghci> runTestTT test1
Cases: 1 Tried: 1 Errors: 0 Failures: 0
Counts {cases = 1, tried = 1, errors = 0, failures = 0}Consider the case when there is a failure:
import Test.HUnit
test1 = TestCase $ assertBool "Failure!" (False && False)The corresponding output is shown below:
ghci> runTestTT test1
### Failure:
Failure!
Cases: 1 Tried: 1 Errors: 0 Failures: 1
Counts {cases = 1, tried = 1, errors = 0, failures = 1}assertFailure and assertString functions take a string as input, and return an Assertion. These are used as part of other test functions. The assertFailure function is used in the definition of the assertBool function as shown below:
-- | Asserts that the specified condition holds.
assertBool :: String -- ^ The message that is displayed if the assertion fails
-> Bool -- ^ The condition
-> Assertion
assertBool msg b = unless b (assertFailure msg)You can also use assertString for handling a specific case. For example:
import Test.HUnit
test1 = TestCase $ assertString "Failure!"Executing the above code in GHCi results in the following failure message:
ghci> runTestTT test1
### Failure:
Failure!
Cases: 1 Tried: 1 Errors: 0 Failures: 1
Counts {cases = 1, tried = 1, errors = 0, failures = 1}Hspec is another testing framework for Haskell, similar to Ruby’s RSpec. You can install it on any GNU/Linux distribution using the Cabal tool:
$ cabal update && cabal install hspec hspec-contribA simple example is shown below:
import Test.Hspec
main :: IO ()
main = hspec $ do
describe "Testing equality" $ do
it "returns 3 for the sum of 2 and 1" $ do
3 `shouldBe` (2 + 1)Executing the above code with GHCi produces the following verbose output:
$ ghci spec.hs
GHCi, version 7.6.3: http://www.haskell.org/ghc/ :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main ( spec.hs, interpreted )
Ok, modules loaded: Main.
ghci> main
Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package HUnit-1.2.5.2 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package random-1.0.1.1 ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package stm-2.4.2 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package unix-2.6.0.1 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package pretty-1.1.1.0 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package QuickCheck-2.6 ... linking ... done.
Loading package ansi-terminal-0.6.2.1 ... linking ... done.
Loading package async-2.0.1.6 ... linking ... done.
Loading package hspec-expectations-0.6.1 ... linking ... done.
Loading package quickcheck-io-0.1.1 ... linking ... done.
Loading package setenv-0.1.1.1 ... linking ... done.
Loading package primitive-0.5.0.1 ... linking ... done.
Loading package tf-random-0.5 ... linking ... done.
Loading package hspec-core-2.0.2 ... linking ... done.
Loading package hspec-discover-2.0.2 ... linking ... done.
Loading package hspec-2.0.2 ... linking ... done.
Testing equality
returns 3 for the sum of 2 and 1
Finished in 0.0002 seconds
1 example, 0 failuresThe keywords describe and it are used to specify the tests. The context keyword can also be used as an alias for describe. A particular test can be marked ‘pending’ using the pending and pendingWith keywords. The pendingWith function takes a string message as an argument, as illustrated below:
import Test.Hspec
main = hspec $ do
describe "Testing equality" $ do
it "returns 3 for the sum of 2 and 1" $ do
3 `shouldBe` (2 + 1)
describe "Testing subtraction" $ do
it "returns 3 when subtracting 1 from 4" $ do
pendingWith "need to add test"The corresponding output for the above code snippet is provided below:
ghci> main
Testing equality
returns 3 for the sum of 2 and 1
Testing subtraction
returns 3 when subtracting 1 from 4
# PENDING: need to add test
Finished in 0.0006 seconds
2 examples, 0 failures, 1 pendingYou can use the after and before keywords to specify setup and tear down functions before running a test. For example:
import Test.Hspec
getInt :: IO Int
getInt = do
putStrLn "Enter number:"
number <- readLn
return number
afterPrint :: ActionWith Int
afterPrint 3 = print 3
main = hspec $ before getInt $ after afterPrint $ do
describe "should be 3" $ do
it "should successfully return 3" $ \n -> do
n `shouldBe` 3Executing the above code yields the following output:
ghci> main
should be 3
Enter number:
3
3
should successfully return 3
Finished in 0.6330 seconds
1 example, 0 failuresThe different options used with Hspec can be listed with the –help option, as follows:
$ runhaskell file.hs --help
Usage: spec.hs [OPTION]...
OPTIONS
--help display this help and exit
-m PATTERN --match=PATTERN only run examples that match given PATTERN
--color colorize the output
--no-color do not colorize the output
-f FORMATTER --format=FORMATTER use a custom formatter; this can be one of:
specdoc
progress
failed-examples
silent
-o FILE --out=FILE write output to a file instead of STDOUT
--depth=N maximum depth of generated test values for
SmallCheck properties
-a N --qc-max-success=N maximum number of successful tests before a
QuickCheck property succeeds
--qc-max-size=N size to use for the biggest test cases
--qc-max-discard=N maximum number of discarded tests per
successful test before giving up
--seed=N used seed for QuickCheck properties
--print-cpu-time include used CPU time in summary
--dry-run pretend that everything passed; don't verify
anything
--fail-fast abort on first failure
-r --rerun rerun all examples that failed in the
previously test run (only works in GHCi)Other than the shouldBe expectation, you can also use assertions like shouldReturn, shouldSatisfy, and shouldThrow. Examples of each are given below:
import Test.Hspec
import Control.Exception (evaluate)
main = hspec $ do
describe "shouldReturn" $ do
it "should successfully return 3" $ do
return 3 `shouldReturn` 3
describe "shouldSatisfy" $ do
it "should satisfy the condition that 10 is greater than 5" $ do
10 `shouldSatisfy` (> 5)
describe "shouldThrow" $ do
it "should throw an exception when taking head of an empty list" $ do
evaluate (1 `div` 0) `shouldThrow` anyException Executing the above code provides the following output:
ghci> main
shouldReturn
should successfully return an Integer
shouldSatisfy
should satisfy the condition that 10 is greater than 5
shouldThrow
should throw an exception when taking head of an empty list
Finished in 0.0015 seconds
3 examples, 0 failuresThe evaluate function can be used to check for exceptions. Its type signature is as follows:
ghci> :t evaluate
evaluate :: a -> IO aYou can also re-use HUnit tests and integrate them with Hspec. An example is shown below:
import Test.HUnit
import Test.Hspec
import Test.Hspec.Contrib.HUnit (fromHUnitTest)
test1 = TestList [ TestLabel "Test subtraction" foo ]
foo :: Test
foo = TestCase $ do
3 @?= (4 - 1)
main = hspec $ do
describe "Testing equality" $ do
it "returns 3 for the sum of 2 and 1" $ do
3 `shouldBe` (2 + 1)
describe "Testing subtraction" $ do
fromHUnitTest test1Testing the code in GHCi produces the following output:
ghci> main
Loading package hspec-contrib-0.2.0 ... linking ... done.
Testing equality
returns 3 for the sum of 2 and 1
Testing subtraction
Test subtraction
Finished in 0.0004 seconds
2 examples, 0 failuresThe Hspec tests can also be executed in parallel. For example, computing the Fibonacci for a set of numbers, and asserting their expected values can be run in parallel as shown below:
import Test.Hspec
fib :: Int -> Int
fib 0 = 0
fib 1 = 1
fib n = fib (n-1) + fib (n-2)
main = hspec $ parallel $ do
describe "Testing Fibonacci" $ do
it "must return 6765 for fib 20" $ do
fib 20 `shouldBe` 6765
it "must return 75025 for fib 25" $ do
fib 25 `shouldBe` 75025
it "must return 832040 for fib 30" $ do
fib 30 `shouldBe` 832040
it "must return 9227465 for fib 35" $ do
fib 35 `shouldBe` 9227465You can compile and execute the above code as shown below:
$ ghc -threaded parallel.hs
Linking parallel ...
$ ./parallel +RTS -N -RTS
Testing Fibonacci
must return 6765 for fib 20
must return 75025 for fib 25
must return 832040 for fib 30
must return 9227465 for fib 35
Finished in 0.9338 seconds
4 examples, 0 failures