Exercise 3: On Quality

I'm going to propose a scientific theory about cognition I cannot prove:

The act of remembering what you did makes you think the end product is correct.

This is from an observation in nearly every creative thing I've ever done that goes something like this:

• You create something requiring a long period of time. This could be software, a painting, writing, or anything that takes time.

• You "finish" and then step back to marvel at how good it is, when a friend walks up.

• Your friend then points out 1 glaringly obvious problem, and suddenly your entire vision of what you created changes.

• Now all you see is this mistake your friend pointed out, and you have no idea how you possibly missed it.

I believe that this phenomenon happens because you have memories of how you made it that are influencing your concept of what you perceive. The act of creating tends to be a positive flow of ideas and work, so your memories are more on the positive or neutral side. This then colors your perception of the work to make you think it's way more awesome than it really is but also hides many flaws and details. There is also an emotional attachment to the work since you created it and remember doing it, which clouds your judgment of the work. Your friend, however, has none of these memories and sees the work more objectively, which makes it easier to see the flaws. This is why copy editors find many more errors than writers. Or, why security professionals find many more defects in software than those who created them. These external reviewers simply have no emotional attachment and no memories of how you created it, so they see it more clearly.

In the world of painting this is so common that painters have numerous tricks to subvert the phenomenon. These tricks are even mentioned by Leonardo Da Vinci in his notebooks, and they're designed to give the painter the perspective of their critical friend:

• Turn the painting upside down and look at it from farther away. This points out obvious problems with color and contrast while also showing you repeating shapes you need to alter. In fine art repetitive shapes are undesirable.

• Look at the painting in a mirror, which flips it horizontally so your brain has no concept of how it was created. Flipping it horizontally turns it into a fresh new painting you've never seen before, and then suddenly you are the obnoxious critical friend.

• Look at the painting through a red glass or in a black mirror, which removes color and only shows it in black and white. This shows areas where the painting is too bright or dark, which makes it look strange in color.

• Look at both the painting and subject through a mirror placed on their forehead, looking up into the mirror, which flips both the painting and the subject upside down so you can compare the two. This shows obvious problems with drawing and makes both the scene and painting look like abstract shapes your brain has no memory of.

• Put the painting away for a few months so you forget how you did it then take it out to look again.

• Ask your obnoxious friend to look at it and have them tell you what they see.

Some painters go so far as to have a mirror behind them while they paint so they can simply turn around to check their progress. I frequently use a black mirror (or just your mobile phone's screen when it's turned off) placed on my forehead to check paintings.

In other creative disciplines there's not as many of these self-criticism techniques, and in software there's very few. In fact, I find that programmers are notorious for being "programmer done" with their code. "Programmer done" is where a programmer hacks and hacks on a piece of code, piling it on until it barely compiles, then declares their work done and moves on. The truth is there's a mountain of work to do after that, from cleaning the code up, performing quality assurance checks, adding invariants and assertions, writing tests, writing documentation, and confirming it works within the larger context of the whole system. But nope, programmers are frequently done when the compiler (or test suite) runs without errors.

In this book you're going to learn to conduct your own set of checks that are similar to what painters use. They are ways of looking at your code disconnected from the history of how you made it, and the secret is going to be checklists. The way you subvert your memories of your work is to force yourself to follow a set of checks that assume you've written something defective. The quality process I'm teaching you won't catch everything, but it will help you spot as many errors as you can find yourself and also help you track what kinds of errors you keep making so you can avoid them in the future. After that you'll be encouraged to have other people audit your code and also audit other people's code so you can get fresh eyes to find even more flaws.

The philosophy I have with defect reduction is one of probabilities. You cannot remove all defects, ever. Instead you'll work on reducing the probability that there is a defect and be able to give a rough estimate of that chance. This frees you from the panic of not knowing if your code is defective and helps you get an idea of how you keep making bad software. Instead of being "programmer done" all the time, you'll have a good idea of when you are finished and ready for review. Instead of being constantly worried about every impossible edge case, you'll be able to assess the probability of these edge cases and deal with the most likely ones.

Exercise Challenge

In this exercise you are going to find a piece of code you wrote months ago and then review it. You may not know how to audit a piece of code yet, but just go through and write comments in the code for anything you find that you do not like. The key is to go line-by-line and file-by-file and look at each line of code. You will then tag that code you find objectionable and write out why. It doesn't have to be a very large piece of software, just something you wrote a while ago.

Study Drills

• Compile a list of all the defects you found doing this and try categorize them. You can look up official defect categories, but a good basic set is: logic, data type, calling. A logic error is when you write an if-statement or loop that's wrong. A data type error is when you used a variable and assumed it was the wrong type. A calling error is when you called a function and did it wrong. These aren't official categories, but they're a good start for you at this time.