30 November 03 - 13:16The trouble with projects

Projects in many industries go out of control. The problems that afflict projects are not peculiar to any one industry, to any one methodology, or to any one type of business. They stem from the very nature of project work. Project work is… well, what is it really ? Think about your answer for a moment.

On occasion, I've asked colleagues why they choose to organize their software work as "projects", as opposed to other forms of organization. The response was : "What other forms of organization ?" Like Moliere's gentleman who had been talking prose all his life without even knowing it, many of us, including project managers, never question why software development should take the form of "projects". The term is part of our final vocabulary.

Project managers who have trained as such may be familiar with the definitions given in the literature on project management, but these focus on a limited set of operational characteristics (projects are aimed at one-time outcomes, involve a defined start date and expected duration, multiple people from several disciplines, etc.). This is a bit like describing laughter as "rhythmic, vocalized, expiratory and involuntary actions" – somewhat accurate but entirely unenlightening.

What that literature is really about is a set of operational guidelines predicated on some understanding of the notion of project, which understanding is never opened up for inspection; this accounts for why so much of the literature is contradictory.

When we take a step back, we recognize that projects are but one form of human activity, that which is preoccupied with bringing about a chosen future. The attitude is summed up by Alan Kay's famous line: "The best way to predict the future is to invent it."

This is what projects are: the power of human invention harnessed to circumvent our tragically limited ability to predict the future. All the facts at our disposal are about the past; we have no facts about the future. It is a different country. In Searching for Certainty, mathematician John L. Casti surveys the performance of the finest tools of scientific prediction available. His findings are not encouraging. After celestial mechanics, where science does indeed have a fine record for prediction, our next best discipline is weather prediction. If you live in a temperate climate, as I do, you've probably had more than one occasion to complain about the quirks of weather forecasts; and long-range predictions are notoriously sensitive to such small and unpredictable events as the beat of a butterfly's wing.

What is surprising, then, is not that so many projects fail; what is surprising is that some projects succeed, in the face of so much uncertainty about the future. What's worse, in today's world the future holds much more uncertainty than it used to in simpler times - in large part due to the disturbing effects of the technologies we have introduced in order to better invent, and therefore predict, our chosen futures. (And one prediction we can safely make is that the uncertainty will keep growing.)

Put these two elements together: the high risk of failure inherent in project work, and the obliviousness of many to the exact nature of the work they undertake when on projects. We have here the recipe for tragedy, which has been described as "a drama in which a character (usually a good and noble person of high rank) is brought to a disastrous end in his or her confrontation with a superior force (fortune, the gods, social forces, universal values)."

That's why so many project are tragedies.

- Management - three comments / No trackbacks - § ¶

28 November 03 - 10:26Introspection - journals, logbooks, fieldstones and blogs

While I'm not a prolific or an extremely consistent blogger, I manage to write with some regularity. With a single exception, however, I have never been able to keep a journal or diary. That exception was a four-month consulting engagement last summer, where I used a diary to record my reactions to the work, to the team I was working with, and to the environment generally.

For over three years I did keep a logbook, where I listed the various engineering tasks I work on through the day, or other activities such as meetings during my on-and-off "manager" periods, and the time spent on them. Nothing personal.

I have never ever used this for purposes of introspection. My only objectives with this practice, both of which have been well met, were as follows : first, to record what I worked on as objective elements to use in all-too-frequent confrontational conversations with my managers; second, as a rough measurement of the breakdown of my working time according to activities (coding, debugging meetings), or according to projects when I worked on multiple projects, or according to user stories.

If found the logbook to be a very inexpensive practice, consuming no more than a few minutes per day, and one that paid for itself, if only in time saved when filling time sheets in positions where were mandatory (though the data was never actually used).

Collecting fieldstones is a technique that I used for a while, which yielded more occasions for introspection. It is much more free-form : I carried with me a pen and some index cards (stolen from XP planning materials), and whenever a thought crossed my mind that struck me as particularly true or revealing, I stoped whatever I was doing to write it down. (This usually happened on the bus or in the street or somewhere equally awkward.)

On the other hand, very few of these have turned out to be worth keeping when I reexamined them.

I'm trying a different technique now - looking at messages I sent to mailing lists, other online fora, friends, etc. a long time ago - a year or more - and extracting from the longer, more detailed ones any insights, lessons or revelations that still seem relevant. The present entry is one example. This form of deferred introspection seems to work better for me than instant introspection.

The technique has some useful characteristics: if I felt motivated enough at one point to write a lengthy explanation of something to some person or group of persons, then that must be a topic that has value for me. If I can still agree with what I said one year later, then my thoughts on the topic are stable enough to be worth extracting. Finally, if I can conceive of ripping them out of the original context and reusing them in a totally different context, there might be some broader relevance or usefulness to the thoughts.

- Private Life Of The Brain - No comments / No trackbacks - § ¶

26 November 03 - 17:11Management Panzaism

How a person sees the world is determined by the context that person operates in. Gregory Bateson's double bind is an application of this principle - if you live in a world where contradictory messages are commonplace, you will become a schizophrenic.

People in management positions have recurring pressures, strains and stresses operating on them which constitute their context. If there are regularities in these pressures, we can expect regularities in how managers perceive the world.

I suspect that a successful manager needs to believe that the projects and responsibilities they take on are doable, indeed that they are doable as a matter of course given the appropriate training and the appropriate process. A manager who had constant doubts on this score would be unable to perform the functions expected of her.

The manager's occupational disease must therefore be a form of Panzaism, which is the opposite of Quixotism. (Yes, I still have Quixote on my mind.) If you are a victim of Quixotism, you see the world as full of hostile giants where other people only see ordinary windmills. With Panzaism, you tend to see ordinary windmills (doable projects) when most people's reality includes hostile giants (complexity, uncertain futures, insufficient training, unproven technology, etc.).

Actually, it's more likely that managers are prone to both types of disease, in cyclic swings similar to manic-depressive syndromes. Thus the project management scenario where at the start of a project, "there's nothing to worry about", and team members' gnawing worries about the project aren't listened to; and when the (scheduled) end of the project comes around, the frantic scrabbling to make up for lost time, debug everything, and "work harder" to make the deadline. All of which is just tilting at windmills, as the project team members know.

Of course a better statement of this would be relative; that is, it would assess the manager's context as more conducive to Panzaism, relative to the worker's context. From the point of view of the manager, it's the workers who have a skewed view of things.

- Management - No comments / No trackbacks - § ¶

25 November 03 - 09:48Deadlines

Once I was asked to lead a team in slightly difficult circumstances. Working out of a remote site, the team had two developers and a more senior person serving as a functional consultant, and who didn't have management's trust. Management felt visibility into the project was very poor.

I asked these people to a meeting to discuss the "time" component of the project. I asked many questions; among those I remember : "what do you think the timeframes for the various parts of this project will be", "what do you see yourselves doing on this project six months from now", "how do you see staffing evolve over the course of the project".

The team's mission was get ready for maintenance of an application which had been developed by a different firm and purchased by my employer. Only one developer was left from the original team; he was the only person with any detailed knowledge of the application and its source code; and, as it happened, the original team had produced no technical documentation whatsoever during development.

This meeting was happening sometime in november, and B., the developer in question, was leaving for another job on january 25.

It was the most amazing thing how this date affected the team's perception of time aspects of the project. Their involvement with the project, as they perceived it, was divided into two sharply separated phases : "before january 25", and they felt quite certain about what they would be doing for the period between then and that date; and "after january 25", which could have been the 99th century for all we managed to say about what would happen then.

Faced with uncertainty about a project, people will look at things that seem most certain to them, and organize their thinking around those things.

The analogy that comes to mind is from a driving lesson, where I was taught that the car goes where you look : if you make the mistake of looking directly at an obstacle it's almost a certainty that you will hit the obstacle - intead you must fix your eye on a point beyond the obstacle on an avoidance trajectory.

Deadlines. Lovely term, very accurate.

- Management - one comment / No trackbacks - § ¶

24 November 03 - 17:35Project momentum

A Terry Gilliam quote from the famous Project Management Movie :

Making a film is essentially about two things: belief and momentum.

It's always terribly hard to restart a project. You don't stop a project - it's just not something that works. A project stopped is often a project killed. (Do you have counterexamples ?)

Why is that ?

If a project was just a work breakdown structure, and a set of human resources with defined skills, stopping and restarting a project would be just like stopping and restarting a car. Perhaps stopping and restarting a project is more like dissecting a frog then putting it back together. That just doesn't work.

- Management - No comments / No trackbacks - § ¶

20 November 03 - 14:21The Library of Babel

The universe (which others call the Library) is composed of an indefinite and perhaps infinite number of hexagonal galleries, with vast air shafts between, surrounded by very low railings. From any of the hexagons one can see, interminably, the upper and lower floors. The distribution of the galleries is invariable. Twenty shelves, five long shelves per side, cover all the sides except two; their height, which is the distance from floor to ceiling, scarcely exceeds that of a normal bookcase. One of the free sides leads to a narrow hallway which opens onto another gallery, identical to the first and to all the rest. To the left and right of the hallway there are two very small closets. In the first, one may sleep standing up; in the other, satisfy one's fecal necessities. Also through here passes a spiral stairway, which sinks abysmally and soars upwards to remote distances. In the hallway there is a mirror which faithfully duplicates all appearances.

There are five shelves for each of the hexagon's walls; each shelf contains thirty-five books of uniform format; each book is of four hundred and ten pages; each page, of forty lines, each line, of some eighty letters which are black in color. [...] All the books, no matter how diverse they might be, are made up of the same elements: the space, the period, the comma, the twenty-two letters of the alphabet. [...] In the vast Library there are no two identical books.

Jorge Luis Borges' haunting short story provides a crucial insight into the problem of design: the vastness and order of "design space". Daniel Dennett elaborates in the idea quite a bit in Darwin's Dangerous Idea.

In abstract, the Library is simply the combinatorial space of all combinations of length P from a set of N possible symbols. Borges' genius is to have given concrete values to P and N, and to have superimposed a concrete spatial vision over the mind-numbing vastness of the resulting number (10 to the 1,823,342).

The Library encompasses every conceivable text, meaningful or not. The number of books which contain some meaningful text in some language is still unbelievably large, as is the smaller number of books consisting solely of meaningful text in one given language. For any given book, there is a still freakishly, unimaginably large number of books which differ from it slightly enough (a comma, a few letters) as to make no noticeable difference. For any idea, there exist a stupendous number of books expressing that idea - and as many expressing its exact opposite.

This is the metaphor appropriate for understanding what software development consists of. All conceivable programs exist somewhere in the Library of Babel, or a version of it slightly modified to allow for the ASCII encodings we favor over Borges' original alphabet. Nearly all of them are meaningless jumbles of characters, but an astronomical number are programs which would compile if given to some compiler. For any given set of requirements, there is a freakishly, unimaginably large number of programs which implement these requirements.

The writing of any program, like the writing of any text, is a non-random exploration through a molecule-thin slice of this vastness that is design space, which if it is successful results in finding one solitary point (one book, one program) the effects of which are appropriate when read in an appropriate context.

For an idea of what "appropriate" might mean, and of how strategies for searching points in this vast space are formulated, one must consider a third kind of design space; that which encompasses all possible strings of DNA. This is the design space which has been explored for millions of years by evolution under natural selection.

Dennett's book is a superb discussion of how to think about nature's strategies for exploring this immensity of design space, and how they relate to (in fact, build up) our own minds' strategies for exploring analogous spaces. To anyone involved in such explorations - any writer, any programmer, any designer - these ideas are simply indispensable.

- Software Development - four comments / No trackbacks - § ¶

19 November 03 - 12:48From blog to print

Esther recently noted that Tim Van Tongeren's blog entry on information radiators had become an article in STQE Magazine. Encouraging news, to writers or aspiring writers who use their blog for "zeroth drafts". (I'm also having a blog entry of mine grow up into an article, to be published shortly if all goes well.)

At the AYE conference two weeks ago I attended Johanna and Naomi's session on building writing skill and confidence. It was the first time I'd heard the expression "zeroth draft".

Briefly, the story is this. Writing is not only a matter of talent - it involves a lot of editing "grunt work". Reading the works of accomplished writers, many readers might think "I can't possibly do that." But, says Johanna, if the same readers could see what these accomplished writers produce as their first draft, they would think instead, "Gee, I could probably do that." Your writing does not have to be perfect on the first try, but you do have to write about subjects you have a passion for. That can be difficult - being passionate implies that you're liable fumble quite a bit, and your fumblings might not get past the internal censor that most writers carry around with them.

One technique to produce a zeroth draft, one with quality expectations set even lower than for a first draft, is the timeboxed freewriting session. Set a timer to go off in five or ten minutes (or just go by your watch), then put pen to paper and force yourself to keep writing until the time is up. Dale has also blogged about the power of that technique, and related ones. The results may surprise you.

Freewriting "zeroth drafts" are not meant to be preserved. Sometimes they're just there to get ideas "out of your system". In the AYE writing session/workshop we symbolically ripped up the result of our first freewriting exercise. Just this once, though, I thought I'd follow Johanna's example and post the result of one five-minute timebox from last week, as I was waiting for the teacher to show up for a practical in my CORBA class.

The blackboards are no longer black but universally green now, and you might take this as a sign that the university really has changed; but it hasn't. Here I sit, still a vessel waiting to be filled with knowledge when the prof comes in; still in the passive position of a dutiful wife, the legs of my mind spread. In a moment we will begin our "practical" class on remote programming, which will consist of faking a spasm of understanding as we solve in our notebooks and on the green blackboard a problem meant for the keyboard; made for the inimitable consummation that is seeing a program actually run.

You can tell, reading this, that there's a lot more where this came from. That surprised me - I had suspected but not known the strength and nature of my feelings about studying.

- Blogversations - one comment / No trackbacks - § ¶

16 November 03 - 19:52There's no such thing as a conflict

Lisa Scheinkopfs' Thinking for a Change formally introduces the "thinking tools" associated with the Theory of Constraints. The tools presented are for the most part diagrams which have a lot in common with the perhaps better-known "fishbone diagrams" - tracing causes, effects and (that's the novel part) assumptions hidden in the cause-effect relationships. One is of particular interest, the Conflict Cloud diagram.

What makes the diagram valuable to me is not its technical merit as a tool, but rather one particular model that it embodies. According to ToC, there's no such thing as a real conflict, or at least we very rarely see one. The underlying motivations of people who are apparently in conflict often turn out, on closer examination, to be completely compatible. That is why there is only one box at the "top" of the cloud diagram.

In this view, conflict occurs for a reason that is a perennial source of ineffectiveness - we get "stuck on a solution", or on one particular definition of success, such as "on time and under budget". Conflict is a particular form of stuckness in which two people (or two distinct "camps") each get stuck on a different intermediate solution to a larger problem that they have lost sight of.

- Private Life Of The Brain - No comments / No trackbacks - § ¶

12 November 03 - 17:12What is the purpose of planning ?

What's at stake in the planning activities undertaken in your project ?

Some projects need to ship on time because they are subject to late penalties. Other projects need to ship on time because it is important to reach market before a competitor does. For still others, there really is a fixed date that must be hit (the software to drive the equipment for observations of the november 8th eclipse is worthless if it ships on the 9th).

Sometimes shipping on time isn't what's at stake in planning; it could be the accurate prediction of the budget to allocate to a research program, or the calibration of a larger project, etc.

It seems to me that a criticism one could have for Extreme Programming is that it does not clearly raise the issue of what's at stake in project planning. The XP "planning game" embodies a number of assumptions as to the objectives of planning activities, which assumptions are probably not true of all project contexts.

- Software Development - two comments / No trackbacks - § ¶

06 November 03 - 06:30AYE conference, day 3

More news from the AYE conference !

The morning session was about humor and its uses in workplace situations. In the first hour, Diane Gibson of the SEI gave us an overview of the research into this topic, including sociological, physiological and medical issues. Well-known author Naomi Karten then led an intense group discussion which ranged over such varied topics as introducing humor into the workplace, appropriate and inappropriate humor, effective aids to the deployment of humor, when and how to use humor in meetings, etc.

I'm kidding. Actually we all sat down on the floor with a bunch of silly toys and fooled around. And it was great fun !

My next and last session was Jerry Weinberg's "Enhancing your personal influence", in which we practiced being a consultant. Outstanding as usual.

Then, alas, came the time of goodbyes. Fortunately it's not actually over for me - with a number of participants of the Shape forum, we are meeting for a special post-conference day-long session.

I might not be blogging again until next week, what with the flight back home and recuperating from jet lag.

- Blogversations - No comments / No trackbacks - § ¶

05 November 03 - 07:01AYE conference, day 2

I'm blogging from Dave Smith's hotel room, on Dave Smith's laptop. That's been going on for two days now, and this is a good time to appreciate Dave here for being a totally nice fellow.

From Steve Smith (no relation to Dave) and Don Gray I learned a great game today, and from the game learned a good deal about tradeoffs affecting software quality, and how to gain an appreciation of these tradeoffs. Lesson number one is still that the fewer defects you put into the product, the sooner you'll end up shipping.

From Johanna and Naomi I gleaned a bunch of useful insights about writing. We also had many, many laughs. I learned more on the writing and publishing process later today at a Birds of a Feather session initiated by Bruce, where Jerry Weinberg shared tips from his long experience as an author.

This is all exhausting but immensely enjoyable.

- Blogversations - one comment / No trackbacks - § ¶

04 November 03 - 05:15Another AYE blogger

Johanna's list of bloggers at AYE is missing Bruce Eckel. Possibly others... not all bloggers here necessarily showed up for the BOF.

- Blogversations - No comments / No trackbacks - § ¶

04 November 03 - 05:06AYE conference, day 1

Originally "guest-posted" from Johanna's laptop at the blogging BOF:

I've attended two sessions so far - one on "Conscious choices for change", one led by Jean McLendon on Satir system coaching... but what has been nagging at me in the background is how annoying these electronic card keys are. It takes a few seconds to open a door with a regular key - with the electronic equivalent it takes three tries and five minutes, because you have to fit the card into the slot just so and I haven't gotten good at that yet. Why adopt a new system that's worse than the one we had before ? That's another topic that was covered at AYE - "What does it take to really improve things around here ?"

- Blogversations - one comment / No trackbacks - § ¶

03 November 03 - 06:30Live from AYE conference

After a 20-hour trip from Paris yesterday I've made it to Phoenix for the AYE conference. The conference started today with a warm-up tutorial on some of the commonly used models and vocabulary from the Satir/Weinberg approach to individual and group effectiveness. I didn't attend that; I was out sightseeing. I hooked up with another attendee, Fiona Charles, and we visited Paolo Soleri's offices at Cosanti, and the Desert Botanical Garden.

More later, perhaps. The conference hotel is magnificent but doesn't offer high-speed Internet access, a luxury I've grown used to.

- Blogversations - No comments / No trackbacks - § ¶