Part III: Data Structures

You're well on your way to building a personal process that gets you started quickly and with limited friction. Having a good starting process and developing an ability to simply let go and hack is the foundation of creativity. The creative mindset is one of fluidity and relaxation. If you have friction and frustration getting started, then it's difficult to get into the flow. Learning to "click" your brain into creative loose hacking mode helps you solve problems creatively and be more productive.

There is no point being creative if what you make is junk. At first, yes, obviously, the vast majority of what you make will be junk, but you don't want to make terrible software for the rest of your life. You need to balance the creative hacking mentality with a rigorous quality mentality. I advocate that people switch between modes of creative expression and critical thinking. You get your ideas out and realized by being loose and creative, but then you make them tight and high quality by being critical of your own work.

In Part II you actually did this when you tracked the number of features you could complete in 45 minutes and then tried to find places you could improve your starting process. However, you couldn't hack and analyze your process at the same time because the critical thinking mode is a killer to creativity. This advice spans nearly every creative discipline I know and helps you not get in your own way while you work.

NOTE: Criticism during creation kills your imagination. Creativity without criticism produces nothing but garbage. You need both but not at the same time.

In Part III you'll switch gears to focusing on quality and developing personal processes that increase your quality. To keep things simple I am only going to define quality as this:

A low defect rate and understandable code.

Most programmers are absolutely terrible at both of these. The vast majority of developers consider their work completed when the compiler finishes and that's it. They ran the test suite so it's done! I call this "programmer done," where there is no self-critical evaluation of their own work because they totally trust in their computer to find all the defects. They also seem to never care if anyone else can understand their code, focusing only on whether it worked well enough to satisfy the bare minimum. If you were ever to ask them what their daily defect rate is they'd just glare at you and say that's not important. Code coverage? Bah. Their test suite has 100k lines of code! It must test everything!

To become a better programmer you must begin the brutal work of looking at your own quality metrics and practices. I say this work is brutal because it demonstrates clearly and obviously exactly how bad you are, and that can be tragic for people who blissfully think they're awesome all the time. Those with impostor syndrome will find this quality analysis refreshing because it will give you a decent idea of how well you're doing and a plan to improve.

Learning Quality Through Data Structures

Data structures are a simple enough concept. Your computer has memory and data to put in that memory. You can either stuff it in random places or come up with a structure that makes it easier to process the data. Since the beginning of Computer Science people have been analyzing exactly how to structure data for different purposes and then how well those structures work. Since data structures are so well defined we can use them to study your quality practices. You will implement each data structure and a test for it and then go through two steps to determine the quality of your implementation.

Your process for doing each of the Data Structure exercises is as follows:

• Each exercise will describe the data structure and what you can do with it. This description will be in English, diagrams, and sample code. I'll give a complete description of the structure without code since you are to implement this and get it right.

• You may have a set of tests that you must pass as well, but these tests might also be written in English, so you'll also be writing an automated test.

• You'll want to continue your training of doing work in 45 minute bursts with breaks, but you can take more time on each implementation. I recommend you do a few quick hacking sessions, then "get serious" and refine your implementation over more sessions.

• When you believe you are "done" you'll then switch into critical mode and begin to find out how you actually did. You'll follow an auditing process that takes you through your code with a critical eye and looks for bugs, tracking how many you make.

• Finally, you'll fix your defects found during the audit phase and continue working on the exercise until you're done.

This process is involved, so the first two exercises in this part (Exercises 13 and 14) will be done by me, live, with all the defects I make and all the code I have. You'll be able to see how this process works in the videos and read my code in the exercises so you can understand what's expected. I will follow the strict process I outline above as closely as I can, so you'll want to watch the videos carefully.

How to Study Data Structures

There's a formal way to study algorithms and data structures mathematically, but I won't get too far into the theory behind them. If this light introduction interests you then you can read several books on the topic and study this branch of Computer Science for years and years. In this book I'm going to give you exercises so you can learn how to implement them from memory and understand how they work. You won't need formal proofs for that, just simply Python code and repeated attempts.

With these exercises I want you to follow a specific way to study them so you can implement them from memory. I use this same process when I study music and when I attempt to paint what I see. It works with anything where you need to remember a concept but also apply it creatively to varying situations, so you can't just rote memorize it. Instead you do what I call "Memorize, Attempt, Check":

• Set up all the information and material that describes what you have to retain. Do anything you can to memorize it and retain it, even if it's just a small part of the information.

• Put all of the information away so you can't see it. I like to put it in a different room so I have to get up from my work if I need to look it up again.

• Attempt to create, from memory, what you need. Try to put anything down, right or wrong.

• When you've exhausted what you could retain, take what you've done and walk back to your information and compare it. Mark off all the things you got wrong and then put your attempt back.

• Using your list of errors, focus on memorizing so that you correct the errors on the next attempt and do it all again.

I like to do 2-15 minutes of memorization, then 10-45 minutes of attempts, but you'll know when you've run out of knowledge and need to go get more. I'll give a concrete example by explaining how I paint or draw from memory:

• I'm going to paint a flower, so I put the flower in one room of my house and my painting set in another room.

• I sit in the flower room and stare at the flower. I do drawings of the flower. I trace it with my finger and try to imagine it in my minds eye. I visualize myself painting each petal, stem, everything. I remember proportions. I may even write down notes on color and try to mix the colors in the flower room.

• I leave everything in the flower room. I quickly walk back to the painting room and try to conjure up the memory of the flower to find out what to paint next. Maybe a leaf is finally clear. I paint that. Maybe the pot is clear now, I do some of that. I keep closing my eyes, attempting to remember the image, then try to paint it.

• When I'm stuck or I run out of time, I get up and take my little canvas into the flower room and compare it to what I see. I'll then make notes on what I got wrong. Is one petal too long? Is the pot the wrong angle? Is the dirt too dark? I sketch notes and figure out what I got wrong.

• Then I take the painting back into the painting room, walk back into the flower room, and use this list of mistakes to continue studying from memory for the next round.

The paintings I make from this process are usually fairly strange but close to the original, depending on how many rounds I take and how often I practice this. Eventually this helped me get better and capture what I see quicker since I could retain more visual information in my memory for longer periods of time.

When you're doing these algorithms exercises you can use the same process to work on your ability to regurgitate them on demand in an interview. You should first just sit down and implement them using all the information you can and learn how they work. It's difficult to memorize something you don't understand. After you have an okay implementation, you can then start to train your memory of it.

• Put all the books, notes, diagrams, and information about that algorithm in one room and your computer in another. Print out your code if you need to.

• Spend a good 15 minutes studying the information in your algorithm room, taking notes, drawing more diagrams, visualizing how data flows, and doing anything else you can think of to learn it.

• Leaving all that information in the algorithm room, walk into the laptop room and sit down to attempt your implementation from memory. Don't spend more than 45 minutes on this before checking your work.

• Walk into the algorithm room with your laptop and take notes (that stay in the algorithm room) on what you got wrong.

• Put your laptop back, and then go back into the algorithm room and do another round of memorizing and studying before doing it all again. Focus on all the things you got wrong, and it will make it easier.

The first few times you do this it will be frustrating, but pretty soon you'll find it becomes easier and, in many cases, meditative to work this way.