Add more test coverage to CI

[zanieb]

A regression like #405 shouldn't make it to the release stage — and we shouldn't need to do manual testing. We should automate more extensive test coverage, notably using the binaries for real-world tasks.