Author Archives: Bill Havanki

About Bill Havanki

Software Engineer for Cloudera.

Don’t build your tests when you run your tests

You’ve got a Jenkins job set up to run your tests automatically, like you should! Good job. 5 points to Ravenclaw.

git clone
cd awesome-test-framework
mvn -DskipTests install
java awesome-test-framework/awesome-test.jar -d where-my-tests-at run

Wait, what have you done. I’m very disappointed. 10 points from Ravenclaw.

The purpose of your test is to … run your test, not build the tools needed to run the test. You can do that anytime. When you build testing tools as part of the test:

  • you waste time – the build of the test tool will come out the same every time
  • you add an extra source of failure – if building the test tool fails, the test fails, but not because of anything wrong in what you want to test
  • you make the test more difficult to run – because the test run also needs the requirements for building the test tool
  • you make the test longer
  • you reduce test stability – if you build from a branch subject to change, like, say, master
  • kittens die

What should you do instead? You should treat the tool like the project it is, and that means creating builds for it, tagging those builds as releases, and publishing them for download. Then, tests can simply install the tool and use it. This runs quicker, has less prerequisities, and is more stable.

curl -O http://repo/awesome-test-1.2.3.jar # BOOM
java awesome-test-1.2.3.jar -d where-my-tests-at run

The obvious counter to this is that it’s an awful lot of work to devote to a test tool that isn’t going to be released to anybody; by always building from source, you always get the latest version of the tool! I’m sorry, but this doesn’t hold up under scrutiny.

  • If you build from source every time, how do you know what the tool is doing? If someone inadvertently breaks the tool, your test will fail on you and it won’t be your fault. If someone changes how the tool is used, your test will fail on you and it won’t be your fault.
  • By now, we all now refactoring is a vital part of software development. If every test is building a tool, that work must be refactored into one build pass, which means it should just be built ahead of any test. DRY.
  • You work in an organization with lots of other people, and when they use the tool, they are the customers of those who wrote the tool. Don’t those customers, your colleagues, deserve the same respect and devotion of effort that you’d give to others? Wouldn’t you love it if someone gave you a one-line installation of a cool tool?
  • Calculate the time that it takes to build a tool. Say it’s one minute. If you run a test hourly that uses the tool but builds it itself, you waste over 15 days of compute time per year that you could use for actually running tests.
  • Test tools need to be rock solid, or else you can’t be sure that your test results are valid. Did everything pass legitimately or is the tool busted and passes when it shouldn’t. You need to work from known good states of the tool, and that’s what releases are for.

Don’t give in to laziness. Set up binary releases for test tools and install them during tests like Maven, Git, or any other utility you need to run a test.


Protecting the master branch from you

An ounce of prevention is worth a pound of cure. – Benjamin Franklin

Eighteen new commits on master with questionable commit messages. Yay.

One of the most critical things you do when working with remote git repositories is pushing your changes to master. Once that is done, everyone else can see what you’ve done. And, if you mess that up, everyone sees your mistakes, and fixing it swiftly becomes a trial requiring team-wide coordination and the ugly push --force command. A couple of months ago finally messed that up one too many times, so I decided to use Git’s pre-push hook to protect the master branch from my own mistakes.

Essentially, I wanted Git to put up roadblocks to me pushing to master. The pre-push hook can perform checks and reject a push attempt purely on the client side, so it’s a perfect spot for implementing those roadblocks. I took a hook from this useful page and augmented it. While the original hook merely asks if you’re sure you want to push, I added two more checks, neither of which can be overridden interactively.

  1. A file named “ok-to-push” must exist. If it doesn’t, the hook rejects the push. This forces me to stop for a moment and not sweep right into pushing.
  2. No more than one commit can be pushed. Pushing multiple commits at once is a dead giveaway that I merged in a work-in-progress branch, instead of the polished and tested final branch that should be distributed. There is no way around this check except to do one commit at a time.

I posted the hook as a gist.

Everyone makes mistakes, but this is one mistake I’m happy to make less often. The hook is enforcing a higher level of discipline, and hopefully, in the end, will become obsolete as I adopt better habits.

The Java logging quagmire

I’ve started working on a new Java project, and it uses the popular SLF4J logging framework. I’ve never really had the opportunity to use it before; I’m mostly a Log4j guy with the occasional foray into Apache Commons Logging. SLF4J is highly similar to both of those other frameworks, but I got a bit confused because a dependency of this project, Apache HTTPClient, uses Commons Logging, and so there was this weird JAR to bridge them in there, and then Logback was around too. It was then that I fully realized what a mess the Java logging ecosystem is. This post attempts to detangle it.

Continue reading

Kicking off background jobs over SSH

I needed this information yesterday, as I was trying to use SSH from one machine to start a shell script running in the background on another.

You’re probably using the OpenSSH server, and started a background process on the server which you intended to continue after logging out of the SSH session. Fix: redirect the background process stdin/stdout/stderr streams (e.g. to files, or /dev/null if you don’t care about them).

Works perfectly. So, not this:

ssh remotehost " &"

but this:

ssh remotehost "nohup < /dev/null >script.out 2>script.err &"

This is also needed for pssh.

pssh -h remotehosts.txt "nohup < /dev/null >script.out 2>script.err &"

Programming Font: Inconsolata

Time for another of my favorite programming fonts: Inconsolata.

Inconsolata specimen

Inconsolata specimen from

As its creator states, it bears a resemblance to the Consolas font that ships with more recent Windows versions, and which isn’t all that bad either. This one has a bit of an edge in my opinion: it’s freely available so I’m good to go on my Mac with it.

It’s a bit sharper and bolder than Droid Sans Mono, and the letters are a bit more spaced out. So, based on your mood, I could see using either one. For some reason it also reminds me warmly of the monospaced font Sun would use in their old Java books in the 90’s – probably Vera Sans Mono.