29 September 03 - 11:27Project schizophrenia

A recent Dilbert cartoon has the pointy-haired boss respond to an employee morale survey singling out as the top issue, "Not enough open and honest communication from management". The boss then delivers his open and honest assessment of the company and its employees. Not unexpectedly, this turns out to be so deeply cynical that we wonder if the PHB's reports were not indeed better off under the reign of "opaque and hypocritical communication". The PHB's parting shot: "If this doesn't work for you, let me know on the next employee morale survey".

Communication in businesses is often paradoxal in such Dilbertesque ways. Gregory Bateson's model of the double bind provides some insight into the underlying structure of such paradoxical situations.

Bateson's model was originally developed to account for the particular symptoms of schizophrenia. It sought to explain mental illness not merely as a vulnerability of the individual patient, but by looking at the context or system of communication having the most impact on the individual's life - most often the family. As adults, though, we spend more time at work than in the family; while we cannot expect the work to shape our thoughts as deeply as our early experiences with our parents did, we can perhaps expect communication at work to be responsible for various dysfunctions which affect only our work.

The double bind model is based in part on a theory of communication incorporating Betrrand Russell's hierarchy of "logical types": in communication, some messages are directly about things, but others are messages about messages - a different logical type. The PHB's "if this doesn't work for you" is a message about a message.

Paradoxes arise because it's hard to keep logical types straight. Many actions or verbal utterances contain more than one logical type of message. The gist of the double bind is that you are receiving, on distinct logical levels, contradictory but threatening messages from a figure of authority, and there is a prohibition on even discussing the contradiction. You can't win, you can't break even, and you can't get out of the game.

The "project" or "initiative" form of organization perhaps leads to more double bind situations that other forms, and among projects I would suppose that software projects are more vulnerable than others because software is all about communication; there more than anywhere else we find opportunities for mixing up (or keeping straight) communication and meta-communication.

A common issue is management keeping important information, e.g. about schedule, from project teams. (The converse happens as well, of course; many project teams keep information from their management.) I was asked recently about a project where management had negotiated one date with the customer, but had asked the project team to deliver on a date two months earlier, telling all sorts of scare stories about the customer pulling out if the date couldn't be met, while in private reasoning that "if they try to hit February there's some chance we might have something for April". Middle managers privy to this decision and its rationale were asked to keep it secret from the team.

If we look at this example in the light of the double bind model, we have: "We shall punish you if you don't make the February date" as what Bateson calls the primary injunction, and "Blame the customer for any punishment, and don't forget that we love you for shipping in April" as the secondary. The primary injunction is explicit, the secondary implicit: it is thus perfectly possible to ignore the paradox, pretend it doesn't exist. Secrecy seals the double bind, preventing those who might be able to resolve the contradiction, by making it explicit, from doing so.

We have here one situation in which the "double bind" structure applies, which might or might not be repeated in that organization.

In families, Bateson argues, the expected result of repeated double bind situations is schizophrenia, which he sees as the inability to correctly interpret the significance of messages from another, because of systematic exposition to messages carrying contrary meanings and which cannot be clarified. When schizophrenics hear the message "I love you", they cannot tell whether it is a joke, a threat, a manipulation, an exaggeration, or what it seems to be.

Similarly, I believe, some organizations systematically destroy their ability - the ability of their operational people - to give meaning to messages such as "This date is important", "We value this client", or "This project is vital".

The crucial thing about double binds is that their effect depend on the "escape routes" being sealed. A child, utterly dependent on her parents, may not be able to escape from dysfunctional communication. A slave may have no choice about recognizing the authority of an owner. An adult, who is neither, always has a choice about recognizing another's authority.

In organizations, we have a choice in the matter of speaking up about contradictions. Not doing so - from fear of being branded "negative" or "not a team player" - is submitting willingly to the double bind. Of course, it may be very difficult for an individual to recognize double bind situations and to select an appropriate response. One of the ways we give ourselves a choice in such matters is by relying on people outside the dysfunctional system - our family, our network of friends and supportive people, or professionals such as coaches and consultants.

- Software Development - No comments / No trackbacks - § ¶

25 September 03 - 14:46Curse of the pretend project

Once upon a time, there was a small software team at a large investment firm. The team was young and bright and eventually they produced a nice software package to automate trading operations. As is wont to happen, the nice software package eventually grew into a hulking monster of unmaintainable spaghetti code, whereupon the investment firm bought better software outside, disbanded the team and sold off the unmaintainable pile.

I happened to be somewhere on the receiving end of that deal.

My then-employer was the IT service company who bought the software. It was, for me at least, a mysterious decision. Some people who also had a look at the source code concurred. It seemed obvious to everyone who looked that it would be cheaper to rewrite the whole mess than bring it into tolerable shape. Mindful of admonitions that such decisions are usually big mistakes, I did provide a rough estimate of how much work would be involved. (It could have kept a team of four busy for over a year.)

The deal went through. Two sales consultants were to bring in new clients, sales to whom would provide funding for further development and maintenance.

I learned that a small team had been assembled to investigate the system, but was floundering for lack of direction. I was asked to manage the project. I declined: I was getting the hang of Extreme Programming at the time, and felt that it was more valuable practice if I stayed with new development projects rather than try to revive bad legacy.

The team floundered a little while longer. A few sales prospects materialized, but all would only commit to buy if we made some crucial modifications to the software to make it suited to their operations. I was ordered to manage the project. Gamely, I went in, established a nice two-week rhythm, started firming out plans for refactoring the system, and struck what felt like an appropriate balance between writing new tests to stabilize quality and adding new functionality.

Over the next couple months, one sales consultant was fired, the team was split up among two sites despite my recommendations that it should remain collocated, and a new project manager was put in my place while I was away on vacation. The sales prospects kept finding "must have" features that the product lacked. The market was dire. Eventually, the company announced bankruptcy and sold off its assets to another company, who rehired most of my colleagues. (I left shortly before that.)

I heard on the grapevine that a project was formed at the new company to sell the software to new clients, which sales were to provide funding for further development and maintenance. Then the one remaining sales consultant was fired and the team disbanded. There matters seemed to rest for a few months.

One day, out of the blue, I got a call asking if I could provide one or two answers to technical questions regarding product X. A new company, it turned out, had acquired the trading package, becoming its fourth successive owner. They were planning on making some sales to fund further development and maintenance, while a small team was tasked with investigating the system and getting ready for minor modifications should prospective clients ask for any. The person calling me was the project manager. (Actually, he was also the whole team.)

How long could this go on ? Forever, I'd bet.

Eventually, I learned that there is is a tactic that many companies in the IT service industry eventually hit on. You're trying to expand into, say, the financial sector. Clients will usually require that you have prior experience or familiarity with the sector. If you have no such experience, one way is to buy it outside, in the form of a software package. "Of course we understand your needs; we're a software vendor catering to investment firms since 199x."

Lesson learned: there are actual projects and there are pretend projects. A pretend project will never succeed; it will always be underfunded, understaffed, and badly managed. I hope to have the wisdom to always say "no" to pretend projects in the future, no matter how many times I'm asked or ordered.

- Software Development - No comments / No trackbacks - § ¶

18 September 03 - 23:01After Sobig, Swen

I came across this news piece. Seems on the money; at any rate I have upward of 20 pieces of email in my inbox from the past 4 hours with a 200K viral payload, as opposed to a couple a week usually. (And a hundred times as much spam, of course.)

It might not be Sobig - the subject headers don't match those I'm aware of. Mine are the "Network Security Patch" variety. At any rate, the next few days might be a good time to think twice about double-clicking any attachments. (I'm not worried for myself; I use Pegasus Mail.)

The Death of Email has been predicted. I used to be skeptical. These days I wonder.

Update: not Sobig, but something known as "Swen" or "Gibe". A spectacularly virulent breed at any rate - the total number of messages I've received, and safely deleted, has now risen to 130.

- The Universe And Everything - No comments / No trackbacks - § ¶

17 September 03 - 01:45Models shmodels

I can't spend one day without reading some article, blog entry, web page about the unified modeling language (UML), model-driven architecture (MDA), or "modeling visually" as a best practice. But I've been unable to find background documentation which says anything useful about the role of models and modeling in software development.

I've only ever had two real insights regarding models and modeling, but they are relevant and perhaps important enough to share here. A small preamble first.

If I think about what the term "model" evokes in general, outside of its software-specific meanings, I come up with two things. One is "scale model" - some physical, made-of-atoms object which in some way stands for another object. A cardboard model of a cathedral, a rods-and-balls model of an atom or a molecule. The other meaning is "conceptual model" - some set of abstract statements which in some way is about the real world or some aspect of it.

On the face of it, the meaning which is more appropriate in the context of software development is the theoretical one. Software is not physical, the kind of "modeling" which involves actual atoms does not seem to apply, except perhaps in one kind of situation. We do sometimes write software which "stands for" some other software that will be built on a different scale - we call that "prototyping".

Other than that, modeling in software must for the most part consist of conceptual modeling. A conceptual model is a set of statements about something, a description of the world; it is a set of symbols with a definite structure, which can make sense to someone. But then, so is any descriptive statement. What's different about a model ?

Insight number one, which I got from a colleague keen on UML, is that a model generally discards most of the information available about what is described. A model is a good one if, with what little information it retains, it still lets us derive useful statements about what it describes. A Newtonian model discards just about all the knowledge we have about physical objects: their color, texture, taste, fragilty, chemical properties, etc. All it retains is mass and location in some coordinate system. Yet it can be used to predict planetary motion to great accuracy.

In other words (George Box's words) : "All models are false, but some models are useful."

Insight number two is about the way we use models - and it's not one that can be explained in one short sentence. I owe it to Betty Edwards' "Drawing on the Right Side of the Brain". I bought the book because I'm not a very visual person, which perhaps explains my lack of excitement about UML documentation in general. It hasn't turned me into an artist or a UML fanatic yet, but I took at least one thing of great value from it - a splendid metaphor for how models work. This is the "viewfinder" tool Edwards introduces at some point, a clear pane of glass or plastic with horizontal and vertical guidelines drawn across its center.

Most models work precisely the way the viewfinder works. They are something interposed between our unaided sight and reality. The lines they draw permit us to make distinctions; this thing is in the upper left quadrant, that thing in the lower right. These statements are not literally "true" of the things we see; they are only true statements about the relationships between the model and reality. But by using the viewfinder we can find out important structural relationships between things in the scene we are trying to draw. These two simple lines let us achieve what Edwards says is the key to drawing well: drawing what we actually see, not what we think we see, after having processed the scene through a variety of symbolic filters loaded with all sorts of preconceptions ("faces are round", "ceilings and walls are at right angles to each other").

Another parallel: most people, once they draw well, rely less and less on the viewfinder. (Or so I'm told. I still can't draw worth a red cent.) So it is with models: the more useful ones we "internalize" until we come to rarely, if ever, use them consciously.

For a great example of an actual model that looks very much like a viewfinder, see this entry by Brian Marick.

Taken together, these bits of insight make me extremely puzzled that we call a "model" a UML document, or more generally a picture which can be transformed to any extent into code. (This isn't to say that a UML diagram has no value. In particular, I think UML can be good for learning one's way around a given piece of software. Perhaps UML makes for good maps. A map can with some justification be called a model, but certainly not all models are maps, and we should use the more precise term.)

Perhaps there is some confusion resulting from the "discarding most information" aspect. It's true that design diagrams discard most of the detailed (and nevertheless important) information about code, just as it's true that a model discards most information about what it describes. This is the property of abstraction. But just because diagrams and models have abstraction in common isn't enough to call diagrams models.

Abstraction is important, and something I've thought about as well - but that'll have to wait until later entries.

- default - No comments / No trackbacks - § ¶

05 September 03 - 13:56Of types and classes

Jim writes: "I haven't seen a discussion of "type" in the OO stuff that involves any single meaning for the word."

Mostly an OO language, any OO language, is an embodiment of somebody's preconceptions about "typing" in the broad sense - how "things" in the world are to be "classified". It's taken me years of having my brain beaten upon by postmodern texts, but I've finally grokked that there simply cannot be One Right Way of classifying things. So there cannot be a single meaning for the word "type".

In Java, the preconceptions that rule are that the labeling of types has more importance than the types themselves - that's the strong typing preconception. Hence interfaces. They are a way of saying, formally, "This is a set of methods that belong together" and "This class implements this predefined set of methods".

By contrast, in Smalltalk you don't spend much time predefining and looking up labels of types, making sure that a class belongs to a particular type, and so on. In Smalltalk "type" just means having certain expectations of classes that take part in some behaviour.

I remember that interfaces made a big impression on me when I started learning Java, presumably because of previous exposure to Eiffel. Now that I have also learned Smalltalk they're less of a big deal.

Between Java and Smalltalk I got exposed to LambdaMOO, which has some of the characteristics of Self : "class" isn't a concept at all in there, there is only "object". The notion of "type" there is closer to biology's - it's all about who's descended from whom.

- Object Lessons - two comments / No trackbacks - § ¶

04 September 03 - 18:48Domino methodology

Ron Jeffries calls Extreme Programming "holistic". However close you look - one iteration, one day, one hour, perhaps even as little as a few minutes - there is a complete software development process going on, including analysis, design, testing, and coding. (And why not... deployment ? At one time I worked on a Web application which I managed to deploy to the live server several times a day - whenever I made a modification and the tests all passed, basically.) Perhaps "holographic" or "homogenous" would be better terms.

Perhaps we also need a better term for the sequential aspects of "Waterfall", if we want to finally put an end to the confusion that surrounds that name.

When I started out developing software, the image that came to mind whenever I did things in that way - doing some preparatory work to separate out the aspects of a problem, then implementing separate pieces of the solution in sequence, then finally integrating the whole thing - was "setting up dominoes to fall".

For some problems, things have to be done in phases, and cannot be homogenous. I suspect that, as in the case of dominoes, that is because the process matters more than the result. Obviously, if what you're interested in is how a domino-toppling tapestry looks like when all the dominoes have fallen over, you could use a homogenous process: lay down a few dominoes here, a few there, rearrange them to get closer to the desired effect.

There would be no point whatsoever to this, of course. The point of domino toppling is that things should happen in a certain way, and that the more dominoes you have, the larger and the more complex your layout, the more of a risk you're taking that one mistake, one small thing going awry somewhere will ruin the whole effect. That risk, that thrill, is why people enjoy domino toppling.

In developing software, are we interested primarily in the results (the product or system) or in the process ? Offhand, it is tempting to answer that with "the product" : isn't all that matters obviously the system as delivered, in use by its intended user population or being commercially exploited ? Yet I suspect that things are far from that simple.

- Software Development - one comment / No trackbacks - § ¶

03 September 03 - 17:25Control: a metaphor

A slightly difficult read, but well worth the effort.

Perhaps I can best make this clear by considering [...] the matter of "control" and the whole related complex suggested by such words as manipulation, spontaneity, free will and technique.

I think you will agree with me that there is no area in which false premises regarding the nature of the self and its relation to others can be so surely productive of destruction and ugliness as this area of ideas about control. A human being in relation with another has very limited control over what happens in that relationship. He is a part of a two-person unit, and the control which any part can have over the whole is strictly limited.

[...] What I have contributed to this discussion is the notion that the contrast between part and whole, whenever this contrast appears in the realm of communication, is simply a contrast in logical typing. The whole is always in a metarelationship with its parts. As in logic the proposition can never determine the meta proposition, so also in matters of control the smaller context can never determine the larger.

[...] This gives us a vew of the world which is still almost unexplored. But some of its complexities may be suggested by a very crude and imperfect analogy. I think that the functioning of such hierarchies may be compared to the business of trying to back a truck to which one or more trailers are attached. Each segmentation of such a system denotes a reversal of sign, and each added segment denotes a drastic decrease in the amount of control that can be exerted by the driver of the truck. If the system is parallel to the right-hand side of the road, and he wants the trailer immediately behind him to approach the right-hand side, he must turn his front wheels to the left. This will guide the rear of the truck away from the right-hand side of the road so that the front of the trailer is pulled over to its left. This will now cause the rear of the trailer to point to the right. And so on.

As anybody who has attempted this will know, the amount of available control falls off rapidly. To back a truck with one trailer is already difficult because there is only a lmited range of angles within which control can be exerted. If the trailer is in line, or almost in line, with the truck the control is easy, but as the angle between trailer and truck diminishes, a point is reached at which control is lost and the attempt to exert it only results in jackknifing of the system. When we consider the problem of controlling a second trailer, the threshold for jackknifing is drastically reduced, and control becomes, therefore, almost negligible.

As I see it, the world is made up of a very complex network (rather than a chain) of entities which have this sort of relationship to each other, but with this difference, that many of the entities have their own supplies of energy and perhaps even their own ideas of where they would like to go.

In such a world the problems of control become more akin to art than to science, not merely because we tend to think of the difficult and the unpredictable as contexts for art but also because the results of error are likely to be ugliness.

Let me then conclude with a warning that we [...] would do well to hold back our eagerness to control that world which we so imperfectly understand. The fact of our imperfect understanding should not be allowed to feed our anxiety and so increase the need to control.

From Gregory Bateson, "Minimal Requirements for a Theory of Schizophrenia", in Steps to an Ecology of Mind. (Hugely recommended.)

You may be wondering what theories of schizophrenia have to do with project management. Gregory Bateson originated the so-called "double bind" theory of schizophrenia (see also here). In a nutshell the double bind is a situation in which communications among a small group get mixed up in a systematic manner, and the ensuing contradictions can literally drive people crazy. Does that sound familiar ? Situations conducive to double binds are extremely common in project contexts. I may have more to say on the topic.

- Private Life Of The Brain - one comment / No trackbacks - § ¶