Category Archives: productivity

Flaky test Kessler syndrome

I have been working within a monorepo (that is, a gigundo Git repository with lots of separate projects in it) and participating in maintaining its test health. This includes unit tests.

  • After every commit to the repository, all of the unit tests across the whole thing are run by our CI system. The idea is that the entire monorepo should always build successfully.
  • Also, for every pull request (proposed change), all of the unit tests across the whole thing are run by our CI system. Same idea as above.

It isn’t feasible for individual contributors to do this themselves, as the monorepo is just too big and varied. I don’t even try to compile the entire thing myself, let alone run all of its unit tests. All I do, and expect from others, is work within the projects that are relevant to the task at hand (and their dependencies), including making sure that existing and new tests pass.

We really need existing tests to be reliable. A flaky test that fails randomly once in a while crops up in random test runs in the CI system. For a PR test, it’s annoying because that test failure seldom has anything to do with the proposed change. For a post-commit test, it’s just noise. In both circumstances, someone tasked to investigate test failures has to spend time discovering that the flaky test is at fault – a false alarm. Additionally, someone proposing a PR has to re-run tests so that, hopefully, that flaky test doesn’t flake out again. This all produces friction in the development process.

Say you have multiple flaky tests. Now, a test run may fail if one or more of them fails. If there are only a few flaky tests, the effect isn’t much worse than having one. But, as the number of flaky tests grows, probability starts to be your enemy.

Continue reading

When in doubt, gather more data

My work day today was mostly uninterrupted, which gave me the uncommon opportunity to really get deep into a technical area that my brain has, so far, been resisting. I had an immediate goal, but I didn’t know exactly how to accomplish it, and so it was time to buckle down and learn the system I was working with.

Continue reading

Implicit no

I’ve felt busier at work this year than usual. (Hence why it’s been a year since my last post – yikes!) I’m probably not actually busier as measured by, I dunno, theoretical units of work effort. It’s that there is a wider variety and breadth of what I have to deal with each day than, I think, ever. Rather than concentrating on fewer items, my time and attention are spread across many items.

I don’t like working like this, and I’m also not used to it, compared with diving deeply into technical efforts. I don’t have the mental habits in place to work well this way (at least by my own assessment).

So, I’ve been learning and reading about this conundrum. While researching one topic, how to make effective requests, I ended up reflecting on how I respond to requests. I try to take care of everything on my plate, regardless of who placed it there, but this year I’ve seen just how finite the size of that plate is. I can’t respond to everything that either I want to or others want me to.

For this common work scenario, you’ll hear the recommendation to learn to say no. I’ve realized how very rarely, nay, even seldom, I out-and-out say no. I mean, I totally buy into the idea, but when it comes time … that little word barely ever passes between my lips. Just thinking about saying that to a coworker feels bad to me when I imagine it.

The problem is that, because of my reluctance to explicitly say no, I am instead implicitly saying no, now more than I used to. That’s even worse.

Continue reading

From Dropbox to BitTorrent Sync

I’ve been a fan of Dropbox for years, mainly due to how dead-simple it is to use and how well It Just Works. Lately, Amazon and Google have been trying to move in on the cloud storage game with their own offerings, and of course there’s Box, Ubuntu One, and other systems already competing. Obviously, the ability to sync files across your machines is super-handy.

My main problem with Dropbox has always been how little space you can get. Right now I have 4.42 GB of space in my personal account, over and above the standard free amount due to several referrals and uploading photos and trying Dropquest 2012. It’s a decent amount of space, but … Google Drive kicks you off with 15 GB right away, and Amazon even starts at 5 GB. Box starts you with 10 GB. Dropbox still starts you with just 2 GB.

I would have figured that Dropbox would start bumping things up with the growing competition, but instead their latest announcement was for some sort of photo gallery thing. I mean, I guess they have the whole file sync thing down pat so they need to branch out, but I don’t get how a gallery app is a logical next step. I really want more gigabytes. I could start using the storage in new broad ways.

A while back a friend of mine had me set up BitTorrent Sync in order to transfer some files. Unlike all of those other systems I mentioned, this one has no central server. Instead, your files are shared democratically across all computers you have linked using the same shared secret. As a bonus, all files are encrypted during transport, so that plus the lack of a central server means your files are better protected from anyone who might like to see them without your permission. That’s all fine and good, but what I really care about is that the space is unlimited, since you bring the storage.

So, I’m transitioning from Dropbox to BT Sync. Right now I have two Dropbox accounts, one for work and one for personal use. I’ll have the work one switched over relatively quickly, and the personal one before much longer. I’ll leave the personal one alive for a few isolated items, and as an alternative for cases where BT Sync isn’t as convenient (e.g., web access). Plus, I worked hard for those extra 2.42 GB, dangit.

I have an Ubuntu machine at home, and I installed BT Sync there to take the place of the central server in my scheme. If nothing else, it serves as a backup location for the files in my Sync folders. Although the system really doesn’t need an anchor like that, it helps me transition over. A better option would be to sync to a trusted friend’s systems, which would give me offsite backup. If there were a way to encrypt those copies, it’d be just about perfect.