Tests by Example in Clojure and Common Lisp
Later: Adding Tail Call Optimization to A Lisp Written in Go
Earlier: (Yet Another) Lisp In Go
While playing around with some dice-rolling simulations recently in Common Lisp (a topic for some future post, perhaps), I looked around for a way of making histograms. Finding none close to hand (it being Common Lisp), it wasn't too hard to "roll my own" library (it being… Common Lisp) to create text-based histograms in the tradition of libraries from CERN that I used as a fledgling physicist in the late 1980s.
Since the project was just for fun, I drove most of the code forward just by REPLing. But I am trying to follow the same habits in my "for fun" GitHub repositories as I would for paid work. This means, among other things, unit tests (in fact, for critical work I still prefer to use TDD when I can).
So, in fleshing the tests out after the fact, I found myself writing somewhat repetitive code that looked like this:
(test hist-values-test
(let ((h (histogram '(1 2) 2)))
(is (= 2 (hist-count h)))
(is (= 1 (hist-min h)))
(is (= 2 (hist-max h)))
(is (equalp #(1 1) (hist-bin-heights h)))
(is (equalp #(1 2) (hist-bin-xs h))))
(let ((h (histogram '(1 2 1) 2)))
(is (= 2 (hist-count h)))
(is (= 1 (hist-min h)))
(is (= 2 (hist-max h)))
(is (equalp #(2 1) (hist-bin-heights h)))
(is (equalp #(1 2) (hist-bin-xs h))))
(let ((h (histogram '(1 2 2 3) 3)))
(is (= 3 (hist-count h)))
(is (= 1 (hist-min h)))
(is (= 3 (hist-max h)))
(is (equalp #(1 2 1) (hist-bin-heights h)))
(is (equalp #(1 2 3) (hist-bin-xs h)))))
This is using the minimalist 1AM testing library, which I like for its
simplicity. It is also similar to the standard clojure.test library
I'm used to in Clojure. It lacks, however, one important feature of
clojure.test, namely the are macro. The macro allows one to
represent tests as a table of examples:
;; Clojure code:
(deftest example
(are
[nums prod] ;; [1]
(= prod (reduce * nums)) ;; [2]
[] 1 ;; [3]
[1] 1
[0] 0
[0 1] 0
[1 1] 1
[2 2] 4
[2 3] 6
[-1 -1] 1
[2 2 2] 8
[1000 1000] 1000000))
An invocation of are has three parts: a vector of symbols to be
bound to values from the examples to follow [1]; the expression to be
evaluated for each example [2]; and the actual examples [3], with one
example per line being the usual, but optional, convention. (In the
code above I have added extra spaces to make clear that nums goes
with the first column of data, and prod goes with the second
column.)
If at first you didn't understand that code being tested ([2]) was
related to multiplication, you'd probably be able to figure it out
just by looking at the examples. This is the primary advantage of
are, namely that examples often provide as good or better
documentation than a written description of functionality does.
Missing such a compact testing idiom for my unit tests, I found myself wondering how hard it would be to port it to Common Lisp. The answer, as you might expect, was: Not that hard. As is often the case, there is not much actual Clojure code behind the function in question, and the corresponding Common Lisp code is similarly compact, as we shall see.
If you look at the actual function definition in clojure.test, most
of the macro proper is error checking; the actual work is delegated to
a completely different namespace, clojure.template (in what follows, I
will ignore the error checking):
;; Clojure code:
(defmacro are [argv expr & args]
;; ... error checking
`(clojure.template/do-template ~argv (is ~expr) ~@args)
;; ... error checking
)A few macro expansions shows what's happening:
;; Clojure code:
(macroexpand-1
'(are [nums prod]
(= prod (reduce * nums))
[] 1
[1] 1
[2 2 2] 8))gives
(clojure.template/do-template
[nums prod]
(clojure.test/is (= prod (reduce * nums)))
[] 1
[1] 1
[2 2 2] 8)
do-template is also a macro. Simply running macroexpand-1 again shows
what it does:
;; Clojure code:
(macroexpand-1
'(clojure.template/do-template
[nums prod]
(clojure.test/is (= prod (reduce * nums)))
[] 1
[1] 1
[2 2 2] 8))gives
(do (clojure.test/is (= 1 (reduce * [])))
(clojure.test/is (= 1 (reduce * [1])))
(clojure.test/is (= 8 (reduce * [2 2 2]))))
The expanded code is exactly equivalent to what you would write if you
didn't have the are macro. This is accomplished by simply repeating
the code being invoked, but substituting the actual example values for
nums and prod.
In order to do accomplish the same in Common Lisp, we take a bottom-up approach.
First, we need to split our test cases into groups corresponding to
the number of columns in our argv vector (in this case, 2):
(let ((cases '(#() 1
#(1) 1
#(2 2 2) 8)))
(partition-n 2 2 cases))
;;=>
'((#() 1)
(#(1) 1)
(#(2 2 2) 8))
The output looks very similar to cases, but each pair has been
grouped into its own list. Here I use partition-n from cl-oju, a
Common Lisp library I wrote specifically to take advantage of sequence
idioms from the Clojure core library; partition-n is a translation
of Clojure's partition function. (The equivalent code in
clojure.template also uses partition for the same purpose.)
We now need to flesh out those cases by making use of each example in the expression being tested, whose Common Lisp translation is
(is (= prod (reduce #'* nums)))
For example, if nums is #(2 2 2), then prod should be 8.
In other words, we want our test cases to be:
(is (= 1 (reduce #'* #())))
(is (= 1 (reduce #'* #(1))))
(is (= 8 (reduce #'* #(2 2 2)))))
The replacement of symbols with their actual values in the Clojure
code is accomplished with a tree-walking function called
postwalk-replace. We can do it in Common Lisp with the subst
function, which works on linear or nested lists:
(subst 3 'b '(* (+ b b) (/ b (- b b))))
;;=>
'(* (+ 3 3) (/ 3 (- 3 3)))But because we have multiple bindings or example arguments we have to convert, we need to do it repeatedly:
(defun apply-bindings (bindings case expr)
(loop with ret = expr
for b in bindings
for c in case
do (setf ret (subst c b ret))
finally (return ret)))
(apply-bindings '(x y) '(2 3) '(+ x y))
;;=>
'(+ 2 3)Now we have everything we need to expand our test cases:
(let ((argv '(nums prod))
(expr '(is (= prod (reduce #'* nums))))
(c (length argv))
(cases '(#() 1
#(1) 1
#(2 2 2) 8)))
(loop for case in (partition-n c c cases)
collect (apply-bindings argv case expr)))
;;=>
'((IS (= 1 (REDUCE #'* #())))
(IS (= 1 (REDUCE #'* #(1))))
(IS (= 8 (REDUCE #'* #(2 2 2)))))
All the macro has to do is wrap this in a progn:
(defmacro are (argv expr &rest cases)
"
Analog of clojure.test/are. Apply multiple assertions
based on some set up code and variations of one or more
variable bindings.
"
(let ((c (length argv)))
`(progn ,@(loop for case in (partition-n c c cases)
collect (apply-bindings argv case expr)))))giving, for our example:
(macroexpand-1
'(are (nums prod)
(is (= prod (reduce #'* nums)))
#() 1
#(1) 1
#(2 2 2) 8))
;;=>
'(PROGN
(IS (= 1 (REDUCE #'* #())))
(IS (= 1 (REDUCE #'* #(1))))
(IS (= 8 (REDUCE #'* #(2 2 2)))))
This is equivalent to the macroexpansion of Clojure's are that we
saw above. I leave as an exercise for the reader a further
exploration of clojure.template/apply-template and its use in the
do-template macro; and the addition of error checking such as making
sure the number example values is a multiple of the number of binding
arguments.
One rough edge that I glossed over: in my first attempt, my macro couldn't find
apply-bindings at compile time until I wrapped that function's definition as follows:
(eval-when (:compile-toplevel :load-toplevel :execute)
(defun apply-bindings (...) ...))This is apparently needed for functions used by macros defined in the same file; it's not an issue for functions defined in other files. Since Clojure has a single-pass compiler, this sort of ceremony is not needed for Clojure macros.
I expect to use this macro again, so will probably spin it out into a
small stand-alone library or incorporate it into a thin wrapper around
1AM. Here it is in action in the hbook library where our discussion started:
(test hist-values-test
(are (h c mn mx heights xs)
(progn
(is (= c (hist-count h)))
(is (= mn (hist-min h)))
(is (= mx (hist-max h)))
(is (equalp heights (hist-bin-heights h)))
(is (equalp xs (hist-bin-xs h))))
(histogram '(1 2) 2) 2 1 2 #(1 1) #(1 2)
(histogram '(1 2 1) 2) 2 1 2 #(2 1) #(1 2)
(histogram '(1 2 2 3) 3) 3 1 3 #(1 2 1) #(1 2 3)))
The code which are makes possible is usually denser and can be
harder to parse initially than more repetitive test code would, as is
clear from this example. However, I find the brevity of are tests
extremely helpful, especially when comparing the test examples to each
another. Such examples are especially helpful when calling out edge
cases where behavior changes. I have given such tables to stakeholders
many times, in order to make sure we are aligned on what the behavior
should be. (Even if the stakeholder doesn't care about the details of
your implementation or your test code per se, they probably will
care about the specific examples you provide).
Parenthetically, here is the histogramming library in action, showing the distribution of the sum of 1000 six-sided dice:
(defun d () (1+ (random 6)))
(defun dn (n) (loop repeat n sum (d)))
(princ (hbook (loop repeat 100000 collect (dn 1000))
80
30))
;;=>
4962 X X
4791 X X
4620 X X X X
4449 X X X X
4278 X XXXXXX X
4106 X XXXXXX X
3935 X XXXXXXXXXX
3764 XXXXXXXXXXXX
3593 XXXXXXXXXXXX
3422 XXXXXXXXXXXX
3251 XXXXXXXXXXXXX X
3080 XXXXXXXXXXXXXXXX
2909 X XXXXXXXXXXXXXXXX
2738 XXXXXXXXXXXXXXXXXX
2566 XXXXXXXXXXXXXXXXXX
2395 XXXXXXXXXXXXXXXXXXXX
2224 XXXXXXXXXXXXXXXXXXXXX
2053 XXXXXXXXXXXXXXXXXXXXXX
1882 XXXXXXXXXXXXXXXXXXXXXX
1711 XXXXXXXXXXXXXXXXXXXXXXX
1540 XXXXXXXXXXXXXXXXXXXXXXXXX
1369 XXXXXXXXXXXXXXXXXXXXXXXXXX
1198 XXXXXXXXXXXXXXXXXXXXXXXXXXXX
1027 XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
856 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
684 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
513 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
342 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
171 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
0 X XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX X
111222233344544544443332211111
11223568933623981977003313702313558540076333111
11244625381526456400925471219232518023527888048930961854221 1
1 11235564827658185382948873231241519783970927498485883199710549330480234 1
33333333333333333333333333333333333333333333333333333333333333333333333333333333
22222222233333333333333334444444444444445555555555555555666666666666666677777777
45566788900122344556778990112334456678890011233455677899001223445667889901123345
40639628518417306395285184073062952851740739629528417406396295184173063962851841
................................................................................
04826059371504826159371604827159372604827159382604837159382604937159482604937150
02457912468913578024579134680135780246791346802357902467913568023579124689135680
Later: Adding Tail Call Optimization to A Lisp Written in Go
Earlier: (Yet Another) Lisp In Go