Software Development

Oh software, how do I envision thee? Let me count the ways

Software is very abstract in nature. There is no real-world physical structure that is, ever, the equivalent of the code you’re writing. This is contrary to mechanical engineering or electrical engineering, where the model or schema maps practically one-on-one onto the final product. Software is not a representation of something that will be. Software is, in and of itself, something that will do.

Of course, we can think of a bike or a car as something that does something, because it has movable parts. And if you get metaphysical, a bike and software are alike in the sense that both require an external source of energy to do something. I’m sure that if we had the time, we could show how everything that a specific software system does, does exist in our physical world. We can, if we really want to, describe everything that happens in terms of electrical charges, potential energy and whatnot.

And yet, even though it’s surely physics all the way down, the abstraction level of software is bloody ridiculous. A mechanical engineer specifies how a product must be assembled, physically, and describes the end product in terms of its physical properties. An electrical engineer also specifies a physical device in terms of its physical properties. You assemble it like so and once you’ve done that, it’s there. It exists. (Okay, yes, and it can also probably do stuff, but that’s a consequence of the physical properties, agreed?) A software engineer, however, specifies behavior only. Everything is behavior. That webpage you’re being shown? It is not a thing, it is your computer showing you a pattern of lights. It’s behavior.

Inherent to software being a specification of behavior, is that you cannot describe it in terms of physical properties. Software isn’t three-dimensional. Software doesn’t have mass. Software doesn’t have dimensional tolerance: It has bugs or it doesn’t. Either it behaves in the way you told it to, or you told it wrong.

Code versus ‘software’

Many people think of software as “whatever it is you can see happening with your own eyes”. Something appears on a screen, a part of a machine moves, an e-mail is sent. To them, the thing that happens is the software. To them, software is behavior. We install software on our computers. We run a software program. But in fact, the term software is overloaded. We sometimes use it to describe both the code itself, and the behavior that results from this code.

Do we use the word software because we don’t see a difference between the code and the behavior? Or do we not see the difference because we use a single word for both? The answer is probably ‘yes’.

The code is not the behavior. The code is a recipe. Following a recipe (executing the code) can get you the end result, i.e. the desired behavior. But it doesn’t have to be a good recipe. It can be a recipe full of spelling and grammar mistakes, text that has been striked out and scribbles in the margin on how to convert ounces to grams. A post-it note on the side mentions “For better consistency, use milk.”

If the only thing you are concerned about is making a single cake, once, and you will toss the recipe in the bin immediately afterwards, then scribble in the margins all you want. But if you want to make that cake again and again, maybe one with strawberries and one with blueberries and if you’re using strawberries you need a bit less sugar than for the one with blueberries and somewhere down the line someone else will execute your recipe and their cake turns out completely wrong… then what?

While you’re coming up with the recipe for that cake for the very first time, if you’re only looking at the end result for that single cake, then your recipe will be full of scribbles and your kitchen will be a mess. And that’s what a lot of code is. A big ball of mud that bears witness to how you experimented your way to something that works.

Building the right thing and building the thing right

Building the right thing.. is an issue in any field, I think. It’s not beholden to software to not know exactly what to build yet. I’ve talked about this with Joep many times when he was still working as a mechanical engineer. Oftentimes, he’d be asked for a time or cost estimate of a rough first idea. It’s normal. People want to know whether it’s worth pursuing an idea or not. It’s also pretty common, in many engineering fields, to have specs come in as you go along. Eventually, we’ll build the right thing. That part is not easy, but it’s nothing compared to building the thing right. This is a challenge that software seems to face more than its sister engineering fields. I think it is a logical consequence of software having no physical properties.

If a machine is a composition of poorly designed components and is slapped together with rope and duct tape, it just doesn’t look right, and the entire company knows. Heck, even the receptionist will get a funny look that says “Is that supposed to… er…?”. Regardless of whether the machine does what it should, regardless of whether there is a user manual or a requirements specification, we all know this needs some rework. What the engineer creates, the schema or model, is very close to the product that it descibres. This is not true in a literal sense of course, a piece of paper is different from an actual machine. But it is true in terms of how we understand it: The picture of the thing, is to our brains, nearly the same as the actual thing. We can rotate the picture in our minds. We can look at it from different angles, even with our eyes closed. We can, before we’ve built the machine, determine whether this gap over here is big enough to reach with a screwdriver. “This bit here, what’s up with that? Yeahh… no. We’re going to need to spend some time on this, this is not okay.”

It seems to me that in software engineering, we are both empowered and handicapped. Our medium is incredibly powerful. It’s not a three-dimensional picture. Talking about dimensions isn’t even appropriate here. Code is a textual representation of something that can become anything.

The programmer, like the poet, works only slightly removed from pure thought-stuff.

Fred Brooks, The Mythical Man-Month

In order to work with our medium, you need to load the program in your head. This takes time and effort and it is an incremental process. The first time load is tough. It helps if someone explains some things to you. Some sensible documentation also helps. And the more often you’ve loaded that particular program, the more you will come to understand about it. In practice, the only ones who load the program in our minds fully, are the software engineers working on it. It’s a language we speak, daily. For people who are not fluent or who can’t be bothered to open the code themselves, we make simplified translations. This is why we draw UML diagrams and take a piece of paper to explain what’s going on in the code. Not just for ourselves, but for others as well. We talk about architecture and design. Design patterns. Separation of concerns. Single Responsibility Principle and Don’t Repeat Yourself. YAGNI and KISS. These are tools to help anyone to load (parts of) the code in our minds.

What if we can’t do it? What if we can’t to build the thing right? Most of the time the signal that we are not building the thing right, shows up as slowness. It takes fucking ages to get bugs fixed, to get any new functionality implemented. It takes forever for a new employee to get up to speed. You’re spending months not getting any traction. Whenever you discuss a plan, when you actually sit down to implement it, the idea just doesn’t work. And almost always, nobody can satisfactorily explain why. Almost always, nobody has a sufficiently complete working mental model of the code.

The knee-jerk reaction of many people is to then start drawing more pictures. To make more UML diagrams. To write more documentation on the side. Sometimes documentation represents the current state. Sometimes it represents the desired situation. Most of the time it’s a mix of both. So we have more meetings. I’m not saying that documentation or drawing pictures is a bad thing per se. We humans are storytelling apes. We need to do it this way.

“The anthropologists got it wrong when they named our species Homo sapiens (‘wise man’). In any case it’s an arrogant and bigheaded thing to say, wisdom being one of our least evident features. In reality, we are Pan narrans, the storytelling chimpanzee.”

― Terry Pratchett, The Globe

But here’s the thing: Someone has to look at the code, first and foremost. Someone in your team has to come to grips with it and load the actual code in their minds, as completely as possible. Because that is your real-world situation. Anything else is just a view, a perspective of your code. But unless someone has personally walked through the territory, your map will be incomplete at best, wrong at worst.

Your paper world isn’t real; code is leading

All the stories that you tell about the code, are nothing more than that: stories. Your stories often tell how you wish things were. But you will never know if you can even get there, unless you open the code and try to walk your way towards where you want to be. Rotate it, look at it from different angles. Look at the details as well as the architecture. Zoom in, zoom out. Until the current state of the code makes sense to you, you cannot ever hope to get to a better place.

Anything that is not done in code, is nothing more than a derived representation. And it will be completely out of sync unless you make an effort to keep them in sync. Unless someone walks back into that territory and checks the map against the real world, your documentation is a form of wishful thinking.

This is why I always say the code is leading. Even though many people will disagree with me and maintain it isn’t and that the documentation is leading, it really is not. Sure, the documentation should instruct software engineers on what to build. And it provides one of many derived views on the code and as such it helps you load the code in your mind. But in the end, the documentation is nothing but a subset of all the information that your code contains. In fact, I can throw away all documentation and we’d still have a product. Furthermore, I can recreate the documentation by inspecting the code and the behavior. Sure, it may not be easy, but it will be easier than recreating the code from the documentation. So I maintain: Once you have code, code is always leading.

Most people don’t truly think about the code. Even some software engineers, while writing code, don’t actually look at the code they are writing. They don’t load the program in their brains. Sure, they type in whichever incantation they think will work. They will search for the spot where they think the incantation can be placed, and add the 75th parameter to the function that encompasses it. They trace that up and up to a spot where they can figure out where to get the data they need and pass it down. The next thing they do is run the program. If the program does what they want it to do, they think they are done.

If you really want to know how to write better code, you need to start really, really looking at it. You need to start picking code apart and figure out, like a kindergartener, how you can fit the blocks back. You search for all the places where there’s pieces of string, rolls of duct tape and garden hoses that run water from the kitchen to the bathroom via the living room. There’s a trick that artists use to see if their drawing or painting is realistic, well-proportioned or has the right perspective: They turn it up side down. Any inconsistencies will reveal itself. The same holds true for software. You have to look at it from different angles. The whole picture has to make sense no matter how you look at it.

You then take it apart and reassemble it in a way that makes more sense. You repeat that process. Forever. And every time you revisit your code, you will find it woefully lacking. And every time you do this, every time you disassemble and reassemble the code in a better way, you become a better software engineer.

Leave a Reply

Your email address will not be published. Required fields are marked *