Continuous test coverage in Clojure
For those building Clojure applications backed by tests, you may be wondering just how broad, or deep, your test coverage is. Fortunately, automated coverage analysis can be integrated in a handful of minutes using cloverage and codecov.
What is test coverage?
The term test coverage used in the context of programming / software engineering, refers to measuring how much a software program has been exercised by tests. (Fault Coverage). With cloverage, you can analyze the coverage of your clojure.test or Midje tests.
Trying out cloverage
You can add cloverage to your
~/.lein plugins, but, in order to allow anyone
who clones your repo to run coverage analysis, it’s recommended to instead place
the following inside your
Note: cloverage may have updated since the time of writing, so double check the latest version on clojars.
Now that cloverage is specified, you can have cloverage run by simply issuing:
$ lein cloverage
For Midje users, specify
--runner :midje as well.
Examining the output
By default, cloverage will run all of your tests and generate an HTML output at
target/coverage/index.html. Open that up with your favorite browser and you’ll
see something like this:
Running it continuously
Manually running cloverage is great, but something like this should be running with each push to Github. This is where codecov comes in. Like Github, codecov is free for an unlimited number of FOSS projects. Sign up using your Github session and add one of your Clojure projects. The following assumes that you already have continuous testing using travis-ci.
Add the following to your
after_success: - lein cloverage --codecov - bash <(curl -s https://codecov.io/bash) -f target/coverage/codecov.json
Adding a README badge
If you’d like, you can add a codecov badge to your
README.md, like so:
No, seriously, that’s it. Whenever you push and the build succeeds, travis-ci will run your coverage analysis and output the results in a specific codecov format. Those results will then just be uploaded to codecov, which already knows and trusts travis-ci servers, and your project’s codecov page will be update shortly thereafter. Have a look around the codecov site, which provides a similar view into your source, with various graphs and metrics.
Quick note about test coverage
It’s worthwhile to note that 100% test coverage probably isn’t worth the effort. Furthermore, test coverage doesn’t mean that your tests are any good and that they are representative of the inputs and behaviors your application will see in the wild. Especially in Clojure, so much of our programs rely on the data, not the code.
Assuming that’s been noted, the coverage analysis is superb for pointing out dead code, missed corner cases, and other areas which don’t see much love in your test suite. Your argument parsing, for example, or codegen in the case of jank. By putting a coverage badge on your README, you’re entering a silent contract to improve your coverage with quality tests.