« The Most Common, and Most Costly Possible, IT Project Management Failure | Main | The Future of Advertising »

February 28, 2007

Thoughts on Programming's Future

I came across this interview with Richard Gabriel, from 2002, that takes a wonderful view of one possible direction for programming:

So, because you can program well or poorly, and because most of it is creative (in that we don't really know what we're doing when we start out), my view is that we should train developers the way we train creative people like poets and artists. People may say,"Well, that sounds really nuts." But what do people do when they're being trained, for example, to get a Master of Fine Arts in poetry? They study great works of poetry. Do we do that in our software engineering disciplines? No. You don't look at the source code for great pieces of software. Or look at the architecture of great pieces of software. You don't look at their design. You don't study the lives of great software designers. So, you don't study the literature of the thing you're trying to build.

Second, MFA programs create a context in which you're creating while reflecting on it. For example, you write poetry while reading and critiquing other poetry, and while working with mentors who are looking at what you're doing, helping you think about what you're doing and working with you on your revisions. Then you go into writers' workshops and continue the whole process, and write many, many poems under supervision in a critical context, and with mentorship. We don't do that with software.

I was talking to Mark Strand, who is one of the first poets who mentored me, and he said, more or less, that how good you are depends on how many poems you've written in your life. About two and a half years ago, I started writing a poem a day, and I've gotten way better since I started doing that. And so, I've probably written about 1000 poems in my life so far, almost all of them in the last two years.

Compare that to how many programs someone has written before they're considered a software developer or engineer. Have they written 1000? No, they've probably written 50. So, the idea behind the MFA in software is that if we want to get good at writing software, we have to practice it, we have to have a critical literature, and we have to have a critical context.

That, more or less, is a take on programming as a creative discipline. It is the classical education approach applied to computing, and it is a beautiful and wonderful idea. The part of me that wrote a lot of poems as a young man, the part that enjoys role-playing games and fiddling with software after everyone is asleep, really likes that approach.

But the part of me that goes to work every day, and looks at building robust systems that work, realizes that that approach to training programmers will only work for a small number: it's too difficult, time-consuming, and expensive. The immediate input is money and hard work, the output gain is in the far future, and is more variable and nebulous. Only a very small proportion of IT people could benefit — the future leaders and mentors. It is important, but not comprehensive, towards solving the software crisis.

(Background note: the software crisis is a term for the problem that it is nearly impossible to write programs that are not buggy, and even techniques like OO programming and formal design cannot help you past a few million lines, by which point the program is essentially guaranteed to be unrunnable because of bugs.)

Another approach is looking at programming languages. I had an epiphany a few weeks ago, wherein I realized that a large common element of many of the design problems being encountered in projects I work with is not that the developers are not good at figuring out how to correctly factor business logic, but that business logic is not an object or a characteristic of an object, so expressing it as objects is fundamentally wrong. Object orientation is just too simple to provide the correct abstractions beyond the model layer and the display-interaction part of the GUI (which is inherently physical). It cannot express business logic (processes, procedures, constraints, timing sequences, coupled rule sets and the like) appropriately.

In looking around for people already working on this, one of my colleagues pointed me to Sun's "metaphors" concept. I think that, from an enterprise software point of view, fixing the problem at this level — ending the artificial separation of design and coding, and allowing units of execution to run anywhere in an organic fashion (delegating deployment and execution decisions by rule set rather than by fixed implementation, for example) — would be a great step forwards. The problem that this attacks is that reusability is hard, currently. Take a simple idea of an object, and put it into a radically different infrastructure than what it was built for, and it often fails. Consider a system with a singleton logger, running on two different systems. Which gets the singleton, and how does the other communicate with it? Or do they each get one singleton, and hope nothing breaks?

In the end, I think that there are three fundamental changes that have to happen before software creation can become robust and inexpensive: a better job must be done training developers, system administrators and technical management, particularly at the beginning of their careers; a better programming paradigm must be developed to simplify the creation and maintenance of large-scale, distributed mixed behavioral/data-centric applications; and we must figure out a way to get people to actually start using the various tools that have become available over the past few decades to simplify the process of system development and integration.

On that last point, two of the little-discussed reasons why engineering software is not like engineering a bridge, are that project managers over bridges are usually qualified engineers in their own right, and that a project manager building a bridge would never dream of saying something like, "We're behind schedule, so we're leaving out half the supports on this one span to meet our deadlines." That happens — all the time — in software development.

Posted by jeff at February 28, 2007 2:59 PM

Trackback Pings

TrackBack URL for this entry:
http://www.caerdroia.org/MT/mt-tb.cgi/2460

Comments

I think you may be grabbing ahold of the wrong end of the stick.

The analogy to make for software development is not creative writing but advertising. The reason is simple: software development is yoked to specific (mostly proprietary) development environments.

In studying the craft of advertising one, generally, does not study the works of great copywriters nor the creation of great advertising campaigns of the past. One studies much the same sorts of things as one does in learning software development and design: ways of expressing yourself effectively, identifying customer needs and wants, etc.

Posted by: Dave Schuler [TypeKey Profile Page] at March 9, 2007 1:30 PM

Not I, sir, but Richard Gabriel. I was the one noting that this is a wonderful, but not entirely practical, view: only a small percentage of the programmers needed could be trained this way. They would be fantastic — the artist types generally are amazing programmers and designers — but too few, and too unresponsive to the needs of their customers (see a GUI created by a programmer's programmer, and you'll know what I mean). We need more engineers, skilled in their trade, and this won't get them.

That said, I don't think advertising is the proper example, either. The proper example, rather, is engineering. One does study great works of the past, but focuses mostly on the lessons of those works and, more importantly, the failed works from the past, rather than the works themselves.

Posted by: Jeff Medcalf at March 9, 2007 6:06 PM