I believe that I was born to be a computer programmer. Somewhere, deep in my soul, there is a need to organize my thoughts in ways that are both new and interesting, but also foundational and reusable at the same time. I’ve always felt that there is something atypical about this kind of work, and about the people who have chosen to do it. Not that it is better or worse than any other profession mind you, but that it was very unique, and at the same time both interesting and powerful.
However over the years, I have learned that this is not exactly a commonly-held belief.
Years ago (it seems like another lifetime now), a manager of mine was adamant that we create a “software development factory”. This person worked in Information Technology, but was never a programmer. This person (hence forth referred to as “IT”) was in love with “process”. “IT” had worked up through the I.T. ranks as an analyst at first, but had been able to cultivate the right look and absorb the right words to be promoted through past similar personalities into a position of real power.
At this one historic moment, “IT” was in the driver-seat of a team of managers and developers. I was one of them. The problem was, “IT” had no had no idea what our jobs entailed. Since “IT” did not like to think there was a concept,”IT” did not understand, “IT” decided to “improve” the team. “IT” had the bright idea that we should take a team of developers and create a “factory” out of them. In this person’s mind, programming was a rote exercise that could be turned into a repeatable process. In the “I.T.” view, programming did not take any real thinking. You took inputs, processed them, and created outputs. “IT” believed this was the most menial work possible, only important enough to be treated in the most dismissive of ways.
Suffice to say, “IT” and I did not get along. However, instead of rolling over, I tried to fight back.
- I argued up and down, and down and up, left ways and right ways that my team of developers were not factory workers, and they did not create widgets.
- I explained over and over that each project was different and required a unique solution that could not be picked off a shelf and plugged-in automatically.
- I created 100 page documents that described in detail how software development worked and how it was more iterative and creative than simply straight-forward and rote.
- I created diagrams of every type imaginable showing the tools, the process, and the creative thinking that went into designing software.
- I pasted images of all our work on the walls and in the conference rooms.
- We adopted SCRUM and Extreme Programming methodologies to show that development was iterative and not simple a step-by-step process,
- I hung statements from notable software designers and developers in the hallways.
- I brought in great software developers to teach classes on design and creative software development
In the end, of course, it did not work.
I was demoted.
My team was cut in 1/2, then 1/2 again.
Work was outsourced.
When it stayed internal, it was done by generic contractors instead of the type of hand-picked software wizards I knew would make the best developers.
Quality slipped, and so did deadlines.
The super-effective and proud team that I once led was turned into a hallow carcass.
This proved to me that there is a common misconception about what software development really entails. I’m not talking about “software support”, but real, honest programming. Truthfully, I don’t think many people outside of core development circles understand how software is made, or what it means to the people that do it.
Worse, this misunderstanding is not just superficial. It colors decisions made by people in the highest places of power. It’s ill-informed, destructive, and in some cases, could even be dangerous.
A few years ago, I decided to try to find out what other people have written on this subject. I was on a quest to find a way to describe programming that would make someone in “IT”‘s position understand the reality of “programming”.
I started at he top. One of the masters of computer science, Donald E. Knuth, wrote about The Art Of Computer Programming in 1974 (he is also the author of the widely read multi-volume set of books by the same name) . Knuth chose to describe programming as art:
“My feeling is that when we prepare a program, it can be like composing poetry or music…Some programs are elegant, some are exquisite, some are sparkling. My claim is that it is possible to write grand programs, noble programs, truly magnificent ones!”
“…computer programming is an art, because it applies accumulated knowledge to the world, because it requires skill and ingenuity, and especially because it produces objects of beauty. A programmer who subconsciously views himself as an artist will enjoy what he does and will do it better.”
While his ideas on the subject are a close approximation of my own, I needed to try to find some other perspectives as well. With a little more searching I found this great article about Art And computer Programming, by John Littler. It contains many quotes from developers on the subject, as well a very relevant quote from none-other than Albert Einstein.
“After a certain level of technological skill is achieved, science and art tend to coalesce in aesthetic plasticity and form. The greater scientists are artists as well.”
While both of these sources were awesome, I was not sure that “art” was the only description I was looking for. Programming may well be an “art” but it is also a “science” too, so just calling it an “art” is not quite accurate. Furthermore, calling it an “art” certainly would not have changed the mind of “IT” . In fact, “IT”, being of simple mind (in my opinion anyway) , would have probably just found that idea “elitist”, and dismissed it immediately. Because of this, I wanted to find another word that did not seem quite as lofty. I found it a few lines down in Littler’s article.
“To me, it relates strongly to creativity, which is very important for my line of work”
This , a quote from Guido van Rossum (the creator of the Python programming language), was getting closer to what I wanted to read. Being “creative” was certainly necessary for art, but it was also necessary for many other pursuits. “Creativity” could describe a beautiful painting, as well as an effective strategic battle plan, or even the solution to complex problem with no clear-cut answer. To me, software development involved at the very least, all three of these things. I decided to follow this line to see where it would take me.
Over at AnswerBag.com, I found that someone named “guitar man” had asked this question: “Is computer programming creative? or is it a just an analytical type process? ”
There was one answer, and it came from a guy with the very creative name: “Jeztyr – whispering in the ears of kings” :
“Programming is an art form that fights back. It requires creativity to solve the seemingly unsolvable, and analysis to make it better, faster, more efficient. A lot of programming is mundane, ritualistic stuff, but other times it’s rewardingly convoluted.”
This answer really hit home with me. I liked the use of the words “creativity” and “analytical” at the same time. However, the best word for me was “convoluted“. Some how that word seemed to describe the software development process in a way that I had never considered before. I decided to look-up the word “convoluted” on Dictionary.com. Here is what it said:
con-vo-lut-ed: Adjective: complicated; intricately involved: a convoluted way of describing a simple device.
“Hmm.” I thought. I then looked at the synonyms: “elaborate,
intricate,
involute,
involved,
knotty,
labyrinthine,
tangled,
baffling”.
All of these words seemed to point to something that was underlying all of this, but not quite yet on the page. Yes, software development is “intricate” and “elaborate”, but the words “tangled” and “baffling” also stood out to me. Those words seemed to describe to me the state of “software development” when you know the problem you need to solve, but you don’t yet know how to solve it. It is also the same part of the process that can require a creative “spark” to surmount, and once a solution is in place, the process becomes more scientific. This limbo state of development always seemed to the most “unordered “to me the most…chaotic.
“Chaos.”
I then recalled something from our time dabbling in SCRUM (a software process that embraces change instead of pushing back on it). The phase was “controlled chaos“. While the SCRUM definition was not necessarily what I was looking for, the term seemed to be appropriate. Software Development was an ever-evolving process of taking chaos and creating order. Creating order from chaos is not an easy thing to do, but it is something that certain individuals (including many talented programmers) thrive upon.
I searched for some thoughts on this, and I found one that was so blunt and and final, even “IT” could have internalized it. Software Engineer Robert L. Glass described his role this way:
“Eat Chaos, Poop Order.”
In the most base way possible, Glass had crystallized my thoughts on software development. He continued to clarify his position.
“Chaos and order are the theme of my life. I consume one and produce the other.“
I could not agree more.
At this point, I seemed to have come to the end of my journey. All of these quotes I had found sort of swirled around in my head until I came to a realization of what it all meant to me, and it is the following:
“Programming is at once, both disciplined, and undisciplined . You must follow some rules, but also strive to break others if you want to make breakthroughs and discover new ways to make better software. It truly is art and science mixed, however the amount of each depends on the problem you are trying to solve. However, there is something else. Programming is like making sense of the senseless. It starts as chaos,and through sheer will of the mind, that chaos is organized into something amazing. It is also a stunningly enjoyable profession that feeds your mind and soul at the same time. In my nearly 30 years of programming experience, the initial surge of energy I feel when sitting down to start developing a new program has never dissipated nor has the sense of satisfaction when the last line of code is written and I hit the [Enter] key for the final time. If anything, the process has only grown greater and more important as the years slip by. In the final analysis, far from being a rote exercise, creating software just might be the the ultimate creative medium. With the proper knowledge, creativity, and computer power, you can build almost anything you can imagine. ”
It was long-winded, but I was satisfied with my answer. However, if I had been able to express these thoughts properly all those years ago, would I have been able to change the mind of someone like “IT”, and finally prove the reality of software development to someone in power?
Probably not, but at least I proved it to myself.
(originally posted June 9, 2009)