Category Archives: tradecraft

Don’t build your tests when you run your tests

You’ve got a Jenkins job set up to run your tests automatically, like you should! Good job. 5 points to Ravenclaw.

git clone git@github.com:joebob/awesome-test-framework.git
cd awesome-test-framework
mvn -DskipTests install
cd ${WORKSPACE}
java awesome-test-framework/awesome-test.jar -d where-my-tests-at run

Wait, what have you done. I’m very disappointed. 10 points from Ravenclaw.

The purpose of your test is to … run your test, not build the tools needed to run the test. You can do that anytime. When you build testing tools as part of the test:

  • you waste time – the build of the test tool will come out the same every time
  • you add an extra source of failure – if building the test tool fails, the test fails, but not because of anything wrong in what you want to test
  • you make the test more difficult to run – because the test run also needs the requirements for building the test tool
  • you make the test longer
  • you reduce test stability – if you build from a branch subject to change, like, say, master
  • kittens die

What should you do instead? You should treat the tool like the project it is, and that means creating builds for it, tagging those builds as releases, and publishing them for download. Then, tests can simply install the tool and use it. This runs quicker, has less prerequisities, and is more stable.

curl -O http://repo/awesome-test-1.2.3.jar # BOOM
java awesome-test-1.2.3.jar -d where-my-tests-at run

The obvious counter to this is that it’s an awful lot of work to devote to a test tool that isn’t going to be released to anybody; by always building from source, you always get the latest version of the tool! I’m sorry, but this doesn’t hold up under scrutiny.

  • If you build from source every time, how do you know what the tool is doing? If someone inadvertently breaks the tool, your test will fail on you and it won’t be your fault. If someone changes how the tool is used, your test will fail on you and it won’t be your fault.
  • By now, we all now refactoring is a vital part of software development. If every test is building a tool, that work must be refactored into one build pass, which means it should just be built ahead of any test. DRY.
  • You work in an organization with lots of other people, and when they use the tool, they are the customers of those who wrote the tool. Don’t those customers, your colleagues, deserve the same respect and devotion of effort that you’d give to others? Wouldn’t you love it if someone gave you a one-line installation of a cool tool?
  • Calculate the time that it takes to build a tool. Say it’s one minute. If you run a test hourly that uses the tool but builds it itself, you waste over 15 days of compute time per year that you could use for actually running tests.
  • Test tools need to be rock solid, or else you can’t be sure that your test results are valid. Did everything pass legitimately or is the tool busted and passes when it shouldn’t. You need to work from known good states of the tool, and that’s what releases are for.

Don’t give in to laziness. Set up binary releases for test tools and install them during tests like Maven, Git, or any other utility you need to run a test.

Protecting the master branch from you

An ounce of prevention is worth a pound of cure. – Benjamin Franklin

Eighteen new commits on master with questionable commit messages. Yay.

One of the most critical things you do when working with remote git repositories is pushing your changes to master. Once that is done, everyone else can see what you’ve done. And, if you mess that up, everyone sees your mistakes, and fixing it swiftly becomes a trial requiring team-wide coordination and the ugly push --force command. A couple of months ago finally messed that up one too many times, so I decided to use Git’s pre-push hook to protect the master branch from my own mistakes.

Essentially, I wanted Git to put up roadblocks to me pushing to master. The pre-push hook can perform checks and reject a push attempt purely on the client side, so it’s a perfect spot for implementing those roadblocks. I took a hook from this useful page and augmented it. While the original hook merely asks if you’re sure you want to push, I added two more checks, neither of which can be overridden interactively.

  1. A file named “ok-to-push” must exist. If it doesn’t, the hook rejects the push. This forces me to stop for a moment and not sweep right into pushing.
  2. No more than one commit can be pushed. Pushing multiple commits at once is a dead giveaway that I merged in a work-in-progress branch, instead of the polished and tested final branch that should be distributed. There is no way around this check except to do one commit at a time.

I posted the hook as a gist.

Everyone makes mistakes, but this is one mistake I’m happy to make less often. The hook is enforcing a higher level of discipline, and hopefully, in the end, will become obsolete as I adopt better habits.

Finally trying a text expander

Being a code monkey, I can type pretty fast. More importantly, I can type naturally, just expressing my thoughts without thinking so much about where my fingers go. It’s the same as when you get good enough at driving that you don’t think about turning the wheel or pushing pedals, you just drive. Martial artists develop the same union with weaponry; the sword becomes an extension of themselves, connected to their center and not a separate entity.

Anyway … I’ve gotten to the point where I prefer to retype something than to try to save keystrokes by retrieving it from, say, my shell history, or by copying and pasting. It interrupts the flow, makes me think about the tool again. By the time I’d mentally work out the “more efficient” method, I could just have retyped it, wasting keystrokes but staying in the flow.

I use a set of software to help keep me flowing and efficient, stuff like Quicksilver and Fluid. But I haven’t used another time saver, a text expander, until the past couple of weeks. A text expander watches what you type and replaces it on the fly, so you can just type “tyvm” and presto, it turns into “thank you very much”. Sounds like a no-brainer to add to my toolset, but I was reluctant to bother since I could just type instead.

So what changed? I think it has been working on Accumulo. That is a tough word to type. Even after these few months with it, it can still tangle my fingers. So, after consulting Lifehacker I gave aText a shot. So, now when I type “accm” I get “accumulo” substituted. “Accm” becomes “Accumulo” and “ACCM” becomes “ACCUMULO”. Yes, I need all three cases. As a bonus, I configured it to make a satisfying pop whenever it expands. a-c-c-m *pop*

Just yesterday I added “SNPS” for SNAPSHOT, since I work with Maven all the time. I suppose that now that I have the tool on hand I’ll start to notice more and more places where it makes sense to use it. I intend to use it only in narrow circumstances, and rely on more traditional – and portable – shortcuts like shell aliases and code editor snippets (ah, Sublime Text) for the majority of cases.

There’s an art to defining expansions. Your abbreviated form has to be something unique and unambiguous, or else you’ll get the expansion when you don’t want it. If I just used “acc” for “accumulo”, I couldn’t write “accept” or “accelerate”.

There is still the extra thought involved right now, remembering that I have the option to type less. Still, I can just write the words out and not worry about it. With time and practice, I expect that I’ll remember to use the tool more often, and the mental speed bump will fade away.

One thing I worry about is that I’ll get too used to the tool, and when I’m on a machine without it, I’ll have to un-remember to use it, and work through another mental speed bump. We’ll see if that becomes an issue. For now, I’ve ponied up the five bucks and added it to the family.