Convoluted Team Code and Software Urban Planning

This is a series of blog posts on thoughts about developing a whole new course hosting service entirely by myself, for myself. The service is currently in Alpha and will be released to past buyers shortly.

Imagine you want to code a modern JavaScript application and you run into someone who tells you, "You have to organize all of your files into a convoluted hierarchy of needs by feature and then by file and then by individual functions." Being a good diligent little hacker you do as you're told and find out that now you need special tools to find anything. Where before you would just put a component in components/ now you have to search for individual components by features in separate folders. Where once was a single 1 or 2 pane view of everything you wanted to edit is now a 10 pane code editor with a whirlwind of convoluted connections between each component.

This is becoming too much for an individual to work with so you go back to this ultra professional who said you have to rework your code into a million files and ask him why? He replies, "Look, I've worked at every FAANG and every time some noob like you gets acquired I have to spend one whole week reworking your code to support a 1000 person team. Just do it now so I don't have to spend one whole week making your code easier to use in our monorepo with 1000 people."

There's a series of problems with this demand:

1. If all it takes is a week of work by some programmer getting paid a few million dollars a year then I'm fine with that.
2. If all it takes is a week to rework the code then I can do that before you acquire me.
3. Facebook is never going to acquire me. Nobody will. I'm a single person working on a small business by myself.
4. The cost to me over the next 4 years being forced to increase the friction in my development process is far higher than the cost of paying one developer to work for one week.
5. The added mental overload of juggling a 10x increase in the number of code locations causes friction which makes it difficult for one person to create something from scratch.
6. If your structure adds 5 more hours of work a week then it's a significant cost to me, and I'm not Facebook.
7. If this structure adds more complexity then that causes defects and security problems. What I need is less complexity, because everyone needs less complexity.

When you point out these problems, Ultra Prophesh Facebook Guy will scoff at you and simple denounce your complaints because you're clearly not a "team player".

You laugh but this is a very common framing of developer productivity. To people working at large companies your technology isn't good unless they can point a team of 1000 terrible programmers at it and sustain that team's poor communication skills under the odd choice of a monorepo. This demand for team size support then incentives open source projects to focus entirely on teams and in many cases they ridicule anyone who wants to work alone.

There's also the idea that focusing on teams makes technology better for individual developers. I really don't buy it, and nobody can actually show me any studies that support this claim. It's simply accepted by everyone in software that if a project requires you to edit 10 files so 10 people can work in it, then that's also great for 1 person. If you press them on why you edit 10 files they'll claim:

1. 10 files for 1 thing makes it more "logical", because apparently a filename...adds logic?
2. 10 files makes it easier for 10 people to work on it...but they fail to mention this makes it harder for 1 or 2 people to work on it.
3. 10 files makes it better for git since now 10 people can hack on their individual little parts while never talking to each other and magically all 10 pieces will come together and work.
4. 10 files--and all 10 files all the time--are needed to make sure the software is "designed well" and follow some kind of Model-View-Controller structure...even though that same MVC structure could just go in a single file.

Obviously none of these claims are true for invididuals. An individual needs fewer files to edit. An individual doesn't have to coordinate commits with 10 people across 10 files. An individual doesn't need more convoluted structure to make something understandable, they need less convolution and structure to reduce cognitive load and increase understanding.

For a team? Probably true but I could also say maybe the company just made bad decisions that require this configuration. If I ran into this place I would most like ask questions like:

1. Why do you have 10 people working on a single feature? Is it just because you have 10 files?
2. Why don't people on your team coordinate more so you don't need to jail everyone's little file commits to prevent them from conflicting?
3. Have you actually asked your team if they prefer 10 files vs. editing a single small file?
4. How big are these 10 files? I've seen projects where there's just one line of boilerplate configuration in many of the files.
5. Why does your system only support a 10 file structure for every feature? Couldn't it support a range or organizing systems so small features can live in one file?
6. Why doesn't the system you're using simply automate the majority of the things you're putting in these files?
7. If you constantly have conflicts in your monorepo, then does your team need training in how to use git branches?
8. Where's your release manager? Is your release manager just sitting there doing nothing and then pushing one button to release? The point of a release manager is to manage this communication and coordinate the features going into a release, so if you constantly have conflicts and need 10 files you should fire your release manager.

I could go on forever, but that'd be pointless because once a team goes down this road it's impossible to undo the decisions. I also think, yes, if you have 1000s of people working on a project then you do need to start segmenting and carving their work areas to make communication easier.

You just don't need any of this team oriented structure when you're one person making something on your own in a single git repo. What an individual needs is less structure because structure add friction.

Tone Policing

The problem with hating on technology that's team structured is the people who love convoluted team structured systems will call you a loser developer. You clearly aren't a professional if you hate editing 20 files to change one word on a single web page. A real developer would love having all of their code segmented out into tiny little bits of disconnected nonsense to assist the future possibility that Facebook will acquire you and point 1000 programmers at your code.

I think the most obnoxious criticism you get from people who love Convoluted Team Code is that you're an unemotional anti-social robot who needs to learn empathy. The criticism is that if you don't love a system designed for teams then you must hate all humans. The irony is anyone who loves only convoluted team code is denying the existence of the vast number of lone hobbyist programmers, beginners, and small businesses that are crushed by their Humanist Empathic Code structure. Saying only convoluted team code is viable as a project structure is actually anti-human and lacking in empathy.

The Town to City Analogy

I should be clear and say that Team Code is actually necessary and doesn't have to be convoluted. You can think of a software project as a small town growing to become a big city. At first a town might be just a house, and that one farmer doesn't need more than one road. As the town grows you need more roads, shops, infrastructure, and eventually you need large highways to support all the people driving.

Except...none of that needs to be convoluted and detrimental to individuals. Continuing the city analogy, you don't need to completely wipe out the pedestrian luxuries to support cars. You don't need insane amounts of parking to support cars. You don't need high levels of lead based pollution to support cars. In fact, if you make a large city still livable for individuals just walking around you'll cut down on the number of cars and generally improve everyone's lives.

This also means that a single farmer back in 1880 doesn't need a clover leaf super highway leading to the house. That's the situation we're in with software currently. Someone wants to just build a 1 person house with a garden and a city planner is running to them yelling that they need a 20 lane super highway with off ramps in the off chance their little plot of land turns into a 1 million person city 100 years later.

"You love people don't you? You'd be a robotic monster if you didn't make your 1 house farm support 1 million people 100 years from now right this very second."

Urban Planning for Code

Maybe what software needs is the concept of Urban Planning for software projects. An Urban Planner wouldn't add a massive 10 lane clover leaf super highway to a 100 person town in the middle of North Dakota (I hope). They'd plan a single main street, a park, side roads, nice structure, walkable spaces, bike lanes, etc. If the city suddenly explodes a good Urban Planner would be able to manage the complexity of the city expanding without damaging the individual livability of the city.

Software projects could use this kind of management. Someone who manages the complexity of the increase in people without damaging the individual productivity someone has gained on the project so far. I even think a lot of the same metrics and analyses found in Urban Planning might apply to Software Urban Planning. I actually have no idea, but it's a guess that they are similar metrics and goals.

Convoluted Team Code

The worst thing someone can do is turn their one person project into Los Angeles. If you've never been to LA it's a nightmare of convoluted streets and bizarre Urban Sabotage. It's almost as if the city has had multiple Urban Planners that use flipped coins to figure out where things should go, and then assumed everyone has and wants a car. You can't get from any part of LA to any other part without a 30 minute drive either on a huge highway or convoluted back streets.

LA's streets seem almost sabotaged rather than planned. I'm actually not joking. There's a great conspiracy theory that GM actually destroyed the National City Lines but nothing anyone could prove even after a Congressional Inquiry, and proof GM did this with other transit companies across the US. Either way, someone planning LA let the street cars die, created tons of roads and highways, and then apparently did it randomly while drunk so that you'd be forced to drive your entire time in LA.

Convoluted Team Systems feel like walking in Los Angeles. Django doesn't feel like a nice little town with a single main street and a few cozy shops. Django feels like being forced to drive a Giant City highway system even though the city only has a population of 100. Ruby on Rails feels like the San Francisco transit system where, sure, someone else is driving, but the City is so poorly planned and convoluted that walking is faster.

Without Software Urban Planning many of these Convoluted Team systems are a detriment for everyone, teams and individuals. With Software Urban Planning you should be able to go from a simpler Individual Oriented system and grow it into a Convoluted Team system without sacrificing too much productivity. What we have now is the worst of both worlds. Everyone is forced to use Convoluted Team Systems, no matter how big your team is, accused of "not being a team player" if you hate on those systems as an individual, and the teams using them do zero planning or organization, adding wasted human effort for no reason.

How This Destroys Individuals

I believe that our current focus on only Convoluted Team Systems forces individuals to operate in a way that's detrimental to their creativity and productivity. The idea that only a team structure to code is logical makes it more difficulty for a single person to easily get started and build from simple beginnings. This idea that Convoluted Team Systems are automatically logical is also entirely unfounded. They aren't more logical. They're just more convoluted, and most of that convolution doesn't have to exist.

At the same time, a system that can't grow to support a team should explicitly say that to avoid being criticized for not supporting teams. I'd rather have people who create software focus entirely on the lone developer making something small for their own enjoyment or benefit, and then eventually add support for teams based on actual planning and analysis of real teams. In fact there could be a business opportunity in doing this. Make a free version that's easy for lone developers, and offer a teams version that solves the problems specific to teams and their organizational habits.

Either way, I'm finding that I simply avoid most software that starts off right away assuming I'm Facebook. If I have to open more than 2 files to get something done you're out. If I have to run 100 services in a cluster of 20 machines just to make Prometheus work, you're out. If I need a massive RabbitMQ server just to let 100 people chat, you're out. What I've found is 90% of the time I can find another tool that focuses on individuals, or just write my own that does what I need for myself in about a week.

But, I'd rather not route around Convoluted Team Systems. I'd rather have software that's aimed at me. Hell, I'd pay for software that made me a monster on my own and didn't care about teams.