I’ve been working on some build automation processes. They were built up over several years, by several bright people most of whom are no longer around, and it does still work. But, it’s weird in one way.
One process, among these processes, uses Jenkins to build an artifact, and then it uses a complex arrangement of Docker, Spinnaker, and a Debian repository built using aptly to copy it to an Amazon S3 bucket. Then, more Jenkins jobs pull that build down from S3 and do other things with it.
I say this is weird because Jenkins is perfectly capable of taking an artifact from one job and feeding it directly to another. The long trip to and from S3 isn’t needed for those later Jenkins jobs. It is true, at least, that there actually happens to be a legitimate, independent reason why the artifact needs to be placed in S3. However, it is much simpler and easier for those Jenkins jobs to just trigger from the initial one.
But wait! I can hear some say. The current scheme has “the nice property” that those later Jenkins jobs definitely use the exact same artifact that we provide to customers.
Whenever you discover that some process has “the nice property”, that means that the process is overly complicated, but you’ve discovered to your surprise that it still does the right thing.
When I rework the process I described to avoid the round-trip to S3 for the later Jenkins jobs, you’d look at it and no longer feel spurred to think pleasingly, “Ah, this has the nice property that the exact same build of the artifact is pushed to S3 and used for other parts of the process” … even though that will be true. But that’s because the simplified process will make that so obvious that such a comment is a platitude. And that’s exactly how it should be.
When you think about complexity, you can distinguish between:
- essential complexity, which is the unavoidable difficulty of the problem you are trying to solve
- accidental complexity, which is basically how you mess up and make things harder than they need to be
The dependency on S3 for those later Jenkins jobs is a great example of accidental complexity. It may have made some sense at the time to do things that way, but once the need for expediency has passed, it’s up to us to destroy accidental complexity to keep the problem space manageable – especially for those who didn’t set up the complexity in the first place.
The utterance “the nice property” is a tell that you are observing accidental complexity and somewhat relieved that it actually does do the right thing anyway. The next time you catch yourself saying that, follow it up with “why wasn’t that obvious?” and see where it leads you. You should realize that there’s a simpler way to go.
Software is hard enough. Eschew accidental complexity.
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.Brian Kernighan