51: What is Data Munging?

At this point in the course you know Python. You may not feel confident in Python, and you don't know all of Python, but neither do most of the people using Python. There are supposed professionals who actually don't know you can use dis() to study the Python bytecode. These "professionals" also have no idea that Python even has bytecode. Given that you know how to analyze the bytes Python uses to process your code I'd say you could be more knowledgeable than many Python programmers working today.

Does that mean you're good at Python? No, not at all. Memorizing arbitrary facts about programming languages does not make you capable with that language. To become a capable programmer you have to combine your understanding of how Python works with actually using it to build software. Programming is a creative practice similar to Music, Writing, and Painting. You can memorize every note on the fretboard but if you can't actually play those notes you don't know how to play guitar. You can memorize every rule of English Grammar but if you can't actually write a compelling story or essay then you can't write. You can memorize every quality of every pigment but if you can't use those pigments to paint a portrait then you can't paint.

The goal of the final module is to take you from "I know about Python" to "I can create software with Python." I'm going to teach you how to convert the ideas in your head into working software, but I must warn you, this process is very frustrating. Many beginners find it difficult to even express their ideas well, let alone well enough to create software. The way you become better at expressing your ideas in code is through experience. You simply have to do it over and over again until it's easy to do. That's why it's so frustrating to learn because it feels like you're making no progress until finally you do.

To accomplish this goal I'm going to present three things to you in the next 6 exercises:

1. An abstract or poorly defined challenge to solve. Don't take these challenges as attempts to trick you like a bad job interview. I'll tell you the secret any secrets I think you need. Take the challenges as being "loose" so you have freedom to find your own solution. Since I don't give you an exact problem, I don't expect any specific solution.
2. A new advanced Python concept to incorporate into your solution. I suggest creating a first version of your solution any way you can, and then doing a new version that uses the new Python concept.
3. More technologies to explore that might make the problem easier, or are related to the topic. Being able to explore new technologies is important as a programmer, but it's also half the fun sometimes.

In this first exercise I'm going to also describe a process for taking your ideas and turning them into code. It's important you read this process carefully and use it until you feel confident in your own skills. After you're comfortable with the process you can modify it to suit how you work, or experiment with new ways to turn your ideas into code.