28 May 04 - 21:48Are you playing tug-of-war ?

Because it's the easiest thing to observe and measure, controlling effort tends to be the path of least resistance for a manager. We choose policies tending to ensure that no one slacks off, that everybody is working as hard as they can. The picture we might have in mind is rowers in a boat, or better a tug-of-war team - people pulling on a rope with different strengths, but each pulling as hard as they can.

The problem with such policies is that, in software development, it routinely happens that effort correlates not at all with value delivered. People work very hard and end up having nothing of value to show for it. Each time that happens, we are baffled - how could that happen ? It happens because we pictured the team as a tug-of-war team, and that is not the appropriate model for the work that a software team does. Such work requires striking a balance between contradictory directions, rather than everyone pulling hard in the same direction.

Picture instead a team - say, six people - arranged in a circle around a round manhole. Each person on the team is holding on to one end of a rope, the other end being attached to a four- or five-foot pole which dangles into the manhole. The team's objective is that the lower end of the pole should NOT, under any circumstance, touch the walls of the manhole, while keeping the upper end as high as possible.

This is constructed as an exercise in balance. The team's collective responsibility is to keep the pole centered. If someone pulls too hard on their end of the rope, they will cause the team to fail just as surely as if they failed to pull hard enough. "Work" in this metaphor is achieved by pulling hard. "Value" is achieved by keeping the pole dead center and high.

The objective cannot be achieved by any one individual "giving it his all", or any two, and so on. Three pulling hard enough may be able to compensate for the other three not pulling at all, if they were placed in the proper symmetry - but that would be such a waste. Similarly, the team could maintain control by exerting varying amounts of effort - everyone pulling as hard as they could would be one way, if they were all more or less matched in strength; but you'd get just as good a result if everyone agreed to pull as lightly as they could given the weight of the pole, taking care to compensate for small variations.

Manage value, not effort.

- Management - three comments / No trackbacks - § ¶

27 May 04 - 18:26Hiring programmers

Someone recently commented on my awareness of writing skills. It's true that I consider myself a writer more than a programmer... I've met way too many people who were billed as "programmers" but whose writing skills - writing English, or French as it mostly happens - were so impaired as to cast serious doubt on their ability to perform their main job. I certainly have no aspirations to the Académie Française, but that's what my criteria sometimes felt like in comparison to the absurdly low standards I've encountered for hiring technical people.

You want to know how to hire a programmer - don't even bother with technical questions in the first interview; have them write a one-page essay (preferably at home, using their choice of tools). Too many spelling errors - no hire; they will write bugs and not fix them. Monolithic block of text without paragraphs - no hire; they won't structure their programs. No introduction or conclusion - no hire; they won't know how to start or finish.

On the other hand, if you see a proper logical flow, maybe even a main title and section headings, and if there's a single coherent story being told - grab' em. Do hit them with a technical challenge, it is programming skill you're after - but you have a much better shot at a good programmer if they can write well to start with.

And if they fail the technical test, keep their resume around - you can probably use them in some other job.

- Management - eleven comments / No trackbacks - § ¶

25 May 04 - 17:09It's all about people - or is it ?

From Esther comes a great story reminding us that successful work relationships depend not only on intrinsic "qualities" of the persons involved, but also on how these persons' qualities and preferences mesh together.

You sometimes hear that managing a team is "all about the people", but that's a gross oversimplification. It's about managing relationships between people, which is a much more complex proposition as soon as teams grow in size: the number of possible binary relationships between people grows roughly as the square of the number of people.

A two-person team has one relationship to manage. A five-person team has ten. That's how many there are in my household - and take it from this father of three, that's quite enough complexity to make life interesting. An eight-person team has twenty-eight; no wonder that seems to be the upper limit of effectiveness for most teams. What is at work is the Square Law of Computation, which constrains so many human activities, particularly intellectual.

For large efforts, a well-known way to cope with the Square Law is to break down the work into smaller chunks, and similarly break down the team into smaller teams. One strategy consists of fairly large "teams of teams", coordinating the efforts of fairly large units - close to the upper limit, around eight or so. Another tactic is to have people pair up - an eight-person team can be organized as four pairs, who at any given time have only six "channels" of communication among them to manage, down from the original twenty-eight; close to a five-fold reduction in complexity.

Note that these two tactics address the problem of work-related communications between people, but not the more general problem of matching people's personalities, attitude and preferences. People on a team do more than coordinate work - they also give support and respect to each other in various ways; whatever the N that measures the number of relationships, you can multiply that by the number of kinds of relationships that have to be managed. Though in terms of the Big-O notation used in the complexity theory which we computer geeks favor to discuss such things, that's still O(n²) - meaning, roughly proportional to the square of the number of people involved.

HR nightmare ? You bet.

- Management - one comment / No trackbacks - § ¶

23 May 04 - 21:26Five dead in... schedule pressure ?

Occasionally I still find myself having the odd conversation with someone, usually but not always in a management position, who avers that in some cases there's just no two ways about it - to ship your project on time you grit your teeth, square your shoulders, and go through a few weeks or months of overtime.

"That's what happens when you get behind on a government contract" was the latest I heard in defense of the sanctity of The Deadline. "Oh, okay then", was my reply, "I hope these guys were at least paid overtime rate then." This was yesterday. Reading the news today made me wish I could go back and give myself a swift kick in the ass as I was writing that.

Earlier today the roof of the newest terminal at Charles de Gaulle airport collapsed, killing five and injuring three. That's a stone's throw from where I live, for you readers in other countries.

Terminal 2E opened last year, a week late, with some equipments and services not fully operational. The official reason cited for the delay was that the region's commission on security had been unable to collect all the required certification documents by opening day. Actually, what happened was that a light detached from the roof and fell almost at the feet of the officials, architects and engineers from the commission who had been touring the building a few days before the scheduled date.

With a total cost of 750M euros, it was crucial that the new terminal, scheduled to process 10 million passengers per year, started paying for itself as soon as possible. At the time, several unions had been protesting work conditions on the site of construction, noting in particular that contractors were under significant pressure to complete the work on schedule. "Forced march" was the phrase used back then.

A phrase are more familiar with in the software industry is "death march" - meaning prolonged overtime resulting from intense schedule pressure on a project. The phrase takes on a slightly different meaning in the wake of the tragedy at Terminal 2E, just as it does on every occasion of a highly visible reminder of the results of schedule pressure. The latest was a bit over a year ago - the space shuttle Columbia.

In the intervening year, the news here has been particularly rife with the good results of the government's latest safe driving campaign. Massive installation of photo radars to get drivers to ease up on the gas pedal was the latest maneuver in this campaign. The message, a simple one, seems to finally be getting across: speed kills.

What is it going to take before our own industry finally fucking gets it ?

- Management - two comments / No trackbacks - § ¶

21 May 04 - 21:54Object dojo

If I want to learn Judo, I will enroll at the nearest dojo, and show up for one hour every week for the next two years, at the end of which I may opt for a more assiduous course of study to progress in the art. Years of further training might be rewarded with a black belt, which is merely the sign of ascent to a different stage of learning. No master ever stops learning.

If I want to learn object programming, my employer will pack me off to a three-day Java course picked from this year's issue of a big training firm's catalog.

What's wrong with this picture ?

Dave Thomas has got the right idea with his Kata, but it needs to be taken further. What I want is an object dojo.

- Software Development - three comments / No trackbacks - § ¶

20 May 04 - 16:25A monopoly on rationality

In a recent discussion a classic David Parnas paper turned up, "A rational design process: how and why to fake it".

A perfectly rational person is one who always has a good reason for what he does. Each step taken can be shown to be the best way to get to a well defined goal.

I am profoundly impressed with how the word "rational" gets hijacked to refer to a very narrow, very specific conception of rationality - the Cartesian conception, broadly. This hijacking comes out in places like the title of this paper and the above quote.

What we do is "rational" to the extent that it is based on real-world data (as opposed to wish fulfillment, living in our dreams) and pragmatically useful models of how the world works (as opposed to linear-reasoning, "A causes B" simplifications, unless of course these happen to be appropriate). "Rational" does not have to mean sticking to Cartesian precepts.

As used in the above quote, "rational" is synonymous with "armed with full, atomic and prioritized knowledge of particulars, systematically arranged in deductive patterns from observational data to conclusions". This is so rarely the case, but a lot more tractable mathematically than the usual situation of bounded rationality.

Humand minds have peculiar limitations. Some of these limitations result in obscure and messy processes such as intuition being more effective than the clearer, more pristine deductive reasoning. If we acknowledge these limitations, we will see that there need not be any "faking" involved in structuring our work along lines other than suggested by Descartes' precepts - that it is rational to acknowledge these limitations. Descartes did not have a monopoly on rationality.

- default - three comments / No trackbacks - § ¶

18 May 04 - 14:02Problem-solving books

I'm scanning my bookshelves for material to use in constructing workshops on general problem-solving skills, techniques and practices...

• Are Your Lights On, Jerry Weinberg, Donald Gause
• Exploring Requirements, Jerry Weinberg, Donald Gause
• Thinking For A Change, Lisa Scheinkopf
• Smart Thinking For Crazy Times, Ian Mitroff
• Getting To Yes, Roger Fisher and William Ury

The common thread in all these books is techniques for problem exploration and collaborative problem-solving, including when negotiation is difficult.

"Are your lights on" is a playful exploration of the general "problem solving" frame, including a very valuable definition of the term "problem" as "a difference between things as desired and things as perceived".

"Exploring requirements" is a more focused work on the discussions that happen around what the problem is. It starts with the "killer issue" of requirements, which is ambiguity. It includes a fantastic technique called "context-free questions". It has some advice on meetings effectiveness and some techniques for brainstorming, paving the way for a general requirements framework (which is probably overkill in agile contexts).

"Thinking for a change" introduces some diagramming tools from the "Theory of Constraints" discipline, which are particularly effective due to their emphasis on hidden assumptions; the Conflict Cloud in particular is of huge value, suggesting that in most project situations there is no such thing as a conflict, which I have found a powerful frame for resolving apparent conflicts. The book is somewhat long-winded but I still recommend it.

"Smart Thinking" includes a discussion of common causes of error, which is a very nice complement to the other topics above. It goes over a lot of the same ground as "Lights" and "Exploring" but in a different package; not as good a book but still recommended.

"Getting to Yes" is the reference book on win-win negotiations, including such great tools as the BATNA (Best Alternative To Negotiated Agreement), the "Hard on the problem, soft on the people" frame or the "Interests not positions" frame. Can be very handy when discussing paradoxical (or conflicting) requirements or constraints on time/scope/quality.

- Private Life Of The Brain - one comment / No trackbacks - § ¶

17 May 04 - 11:48Process as theatre

Robert Austin's latest book, Artful Making, presents an extended metaphor of agile design processes in general (with a firm nod in the direction of Extreme Programming and agile software development) as akin to the production of a play for the theatre.

Reading the book recently provided a frame which got triggered when someone on the Extreme Programming list asked what "revisions" had been made to the method since Kent Beck's book first came out.

The question is a bit like asking what revisions were made to Hamlet since it first came out. The answer has to be either "none" or "as many as the number of times the play was staged". Austin makes that point forcefully - it's not the script that gets revised; rather, the play consists of the script, which is fixed, and the production, which changes not just with each cast and director but also (within a more constrained range of variation) from one show to the next.

If you were to observe an Extreme Programming project today, you should expect to see something rather different from the first few XP projects; the point of an "artful" process is precisely not repeatability. But they would still be using the exact same script.

- Software Development - four comments / No trackbacks - § ¶

13 May 04 - 15:29Modeling design risk

In discussions of Extreme Programming, the YAGNI principle often turns out to be the subject of heated controversy. That is the principle which says we should code an application one requirement at a time, in sequential order, and that design elements should be introduced "just in time" - just when we are about to implement the requirement which calls for that design element. We should not introduce design elements ahead of time on the grounds that they support a requirement that we expect to be implementing later on.

This is a good test case for the idea that methodologies are risk models. We should not expect to find that either advocates or critics of Extreme Programming are "right"; rather, we will find that in both cases the person arguing for "designing for the future" or "delaying all design decisions" has in mind a model of the important risks involved in creating software, and a strategy which addresses these risks.

The people advancing either idea are not in conflict (there is no such thing as a conflict). Rather, the disconnect reflects either different experiences with software projects, which can be resolved by looking at which projects the one at hand is most like; or the unthinking use of a model which is not appropriate to the project at hand, which could be resolved by testing (perhaps through some manner of measurement) which model best describes that project.

In general two major risks that people are concerned with on software projects are requirements risk (i.e. volatility of the requirements, so that by the end of the project something different is needed from what was stated at the outset), and design risk (the internal structure of the software does not support the evolutions that are necessary during creation of the software, or in maintenance once it has been deployed).

Typically, even critics of XP recognize that in projects where the requirements are highly volatile, the YAGNI principle makes sense; it allows the software to remain flexible and respond to unforeseen changes throughout development. But critics argue that requirements are often less volatile than is claimed. If we assume that the requirements are completely stable, they say, YAGNI no longer makes sense, so we should "temper" our use of YAGNI depending on how stable the requirements are. The conclusion is often in favor of "light" (or "right-sized")up-front design.

Thus the implication of arguing for "up front" design, i.e. in advance of implementation, is that doing so mitigates design risk. Let's leave requirements risk aside, then, and focus on design risk. I will try to identify what model is held by people who argue for "right-sized" up-front design.

The way up-front design mitigates risk, in the model, is that we can (somehow) check for conceptual consistency among the various design elements. Design risk is mitigated by the detection (and resolution) of inconsistencies. (I'll take "missing design elements" to be a special case of inconsistent design elements.)

For instance, we decided to put data in an RDBMS to address the "searches are fast" requirement, and we decided to put data in XML because of the "ability to faithfully represent our semi-regular business data" requirement. (Let's not make too much of the example - I'm just giving an example to make concrete the meaning of "design".)

The problem with this risk model for design risk is that it is incomplete - it ignores important considerations and simplifies too much. In particular, a design decision can turn out to be inconsistent, not just with another design decision, but also (and perhaps predominantly) with what would be called an implementation decision.

I need to state this a little more accurately. The design decisions that an early high-level review of the requirements will get you are likely to be homogenous in level of abstraction. And I prefer to call "implementation" decisions low-level design decisions - it's design all the way down to me. So the equivalent statement becomes: the classic model of design risk includes inconsistencies between design elements at the same level of abstraction, but (implicitly) assumes that high level design elements will not be called into question by unforeseen lower-level design elements.

The opposite turns out to be true (or at any rate XP assumes it to be true). The effect is close to what Joel Spolsky "The Law of Leaky Abstractions". (Joel's name for it is somewhat unfortunate and his model has weaknesses, but the meme has gained some traction in the blog-reading public; I mean to use it only for illustration.)

At the same time, the early high-level design decisions may well turn out to exert a more constraining influence on later design decisions, i.e. lower-level ones, leading to inconsistencies we cannot back out of.

If we add this into the above model of design risk, then we see that design risk is increased the longer design decisions are delayed relative to corresponding "implementation" (lower-level design) decisions; thus, designing early increases design risk.

In the presence of static requirements, XP would still address design risk by insisting that design elements should be introduced one requirement at a time, in order of the requirements having higher business value. (We may still make an effort of detecting inconsistencies ahead of time, but in XP this is not a systematic effort; it's expected to happen "in passing" as it were.) Design elements are introduced in cyclic order of levels: High, Medium, Low, High, Medium, Low. "Upward" conflicts are detected and resolved sooner; moreover, this also tends to drive the design toward a more modular region where conflicts are generally easier to resolve.

The net effect of this more complete risk model (or so XP argues) is that we should prefer delaying design decisions, even in the presence of stable requirements.

- Software Development - four comments / No trackbacks - § ¶

12 May 04 - 16:14Motivational disasters

Monday evening, Emmanuel and I held a short rehearsal for the workshop we are presenting at XP2004, on self-organizing teams.

The workshop is organized, for good reasons, around a series of exercises that have their origins in the theatre; they are adaptation of a director's training material. When the rehearsal ended and we sat around discussing the experience, someone's chance remark about "motivation" triggered a memory long buried, and I told the following story.

Back in the heady dot-com days, I was employed by "the leading portal site in France". A couple months before the IPO, a rumor started circulating, soon followed by an official confirmation: all company personnel - around 80 at that time, if I remember correctly - would be invited to an off-site event, to last an entire day. No other substantial information was leaked. We knew that the offsite would take place close by, in Paris. In the corridors we passed the head of HR, grinning like a cat who has swallowed a small defenceless creature, but she wouldn't say anything more.

Some among us were grumbling that our workload left us little time to spend one whole day in some sort of corporate boondoggle, some (I was among them) were mildly interested and excited at the prospect; we were sure to enjoy at least a somewhat luxurious free lunch, and quite possibly some form of entertainment.

When the day came, we all dutifully showed up at an office building which looked a close cousin of our office building. Whatever the Big Surprise was, the setting wasn't it. In the lobby, we were told that we had been "assigned" to groups. I followed instructions to the room where "my group" was already mostly convened, about 8 in all. The assignments were obviously intended to associate people from different departments - there was one girl from HR (who seemed to have a slightly better idea what to expect), two guys in Sales, and so on.

I was greeted by a pair of people - one guy, one girl - who looked slightly out of place, and who introduced themselves as "HR consultants", and told us that they would lead us through various fun activities. We were made to stand up in front of the group, and recite answers to corny questions such as "What do you do", "what do you like", "what do you dislike". For the most part we played the thing straight, though I remember one guy's answer to that last was "What I dislike - um, the practice of sodomy". My own line was "I dislike having my arm twisted". (Looking back, perhaps we were both attempting to make veiled allusions to the proceedings. I certainly was, but I never found out what my colleague had meant.)

We may or may not have gone through other such "exercises" before lunch; my memory of the whole morning is fuzzy after the first twenty minutes or so.

In the afternoon, we were told that we would plan, script, rehearse and finally enact a "playlet" - a skit or short scene - we were to take turns at doing this with the audience being the rest of the company. We were allowed to choose from a short list of "interesting" topics, which were all in some way related to the firm's mission statement or IPO-related issues; I'm making these up (except the last one), but the following list probably gives a fair idea:

• Leading the industry in Community building
• The best and brightest Internet brand
• Stock Options & easy money
• ...etc...

My group picked "easy money". We were heavily prompted by the "HR consultants"; upon questioning these had turned out to be theatre students recruited as "extras" by the actual HR consulting firm retained for the event (I never knew who they were, probably Andersen or someone like that). They told us stuff like "you should portray an emotion; how do you feel about making easy money from your stock options ?", and "what would be a good metaphor for the greed you are feeling ?" I'm hazy on the details, but I remember we did something involving farm animals. I was a pig, symbolizing Greed.

You're probably nauseated enough by now; I shan't go on. I know from ulterior conversations that a number of us "resigned" then and there; they may not have left the company immediately after, but their mind was made up that this was a ship bound to sink.

In terms of motivation, the whole thing was a disaster. Actually, we were never told that the point of the entire mess had been motivation; we could only guess. The purpose was never explained, before, during, or after. My best guess is that this was just something foisted off upon us by the "consultants" as a standard feature of the pre-IPO grooming of an IPO-worthy company, and never questioned by the powers that were.

The contrast with the aforementioned "theatrical" workshop (which I perhaps might not have attempted if I had explicitly recalled this episode) is striking. In the workshop rehearsal, we provided a context of safety; in the beginning, we made clear that no one was under any obligation to perform the exercises. We intend to do this in the actual workshop, and I do this in every single one of my training or consulting activities. Such a simple "ground rule" was never offered in the corporate event. In the workshop, we make our purpose clear, and we make clear what we expect to get from the event and what we hope the participants will get out of it.

Most importantly, we make no attempt whatsoever to "motivate" people. Motivation is something intrinsic: an activity, a project interests you, or it does not. I suspect that all attempts to inject motivation from the outside, as it were, are bound to fail; at any rate, attempts to instill motivation by a stock "activity" chosen from an HR consultancy's catalogue certainly are.

- Management - three comments / No trackbacks - § ¶

10 May 04 - 13:58Projects are about desire

To be the kind of person it takes to start a project, you have to want something badly enough to surmount the natural tendency to just go with things as they are (or as they go, i.e. sometimes getting a bit worse every day, but not so you'd notice). That "something" can take many different forms - more fame, more money, more power, the realization of a nifty new idea, a more just world, a safer one...

Without particularly strong desires of one sort or another, I don't think there could be any "projects", software or otherwise.

- Private Life Of The Brain - three comments / No trackbacks - § ¶

09 May 04 - 14:06Teaching and learning with games

I have a particular fondness for games, and they work particularly well in my experience both as a learner and as a trainer. In the former capacity I particularly enjoyed Senge's Beer Game, Deming's Red Beads Game, a similar game of Jerry Weinberg's called the Quality Game (see here), and am looking forward to the XP Game; in the latter capacity, among others I ran a fun thing now called "Forty-nine minutes inside the Fridge", inspired by Dietrich Dörner's book The Logic of Failure.

The downside of games is that it can take a lot of effort to frame as a game something that you want to teach, as opposed to, say, a bunch of Powerpoint slides. You have to work out rules, scoring, timing, allowable numbers of participants, material such as tokens, cards etc. If you can spare the time and effort, though, dreaming up a game is well worth the effort.

Why do games work ? When used appropriately, games are a sub-category of experiential training. Experiential learning puts the learner in charge of tailoring the content to his or her needs and background, rather than the teacher or trainer. This contrasts with the "transfer of knowledge" approach, in which an expert attempts to "pour" the substance of her knowledge into the the student's head, mostly intact, or at least with modifications largely under the expert's control.

When the material taught applies to unique and complex situations (such as software development projects), such "tailoring" according to the student's special needs and particular background becomes ever more important. The experiential approach is a better fit in these cases than the "knowledge transfer" approach.

An important idea in experiential training is that there needn't be One Correct Answer to every question; rather, the process of coming up with an answer that fits the situation is the richest part of any learning activity.

Games exhibit these two elements, usually to a high degree: there usually is more than one correct "move" in any given situation, though of course some moves are more correct - or more skillful - than others. Players exhibit their mastery of the situation over a sequence of moves and not by a single move. This makes games a good fit for training that is designed to be experiential.

The tailoring of content in experiential learning happens through reflection on experience. "Students" do something, then think back on what happened for them while doing. They formulate various explanations for whatever happened that surprised them. This usually leads to improved mastery over the situations to which the training is relevant.

What distinguishes experiential training, explicitly offered as such, from everyday work (or life) is that the trainer provides a context which is particularly rich in opportunities to be surprised, and which sets apart explicit periods for reflection. The trainer also provides safety from ridicule, because we are often at our most ridiculous while learning new things.

One possible problem with experiential training is that you can't always just say, "Let's sit down and do something, then see what we've learned from it". People are rarely comfortable with this little structure. Framing an experiential learning activity as a game provides more structure, and provides an incentive for going through the various activities involved. Games are a socially acceptable pretext for the requisite "suspension of seriousness".

- Private Life Of The Brain - three comments / No trackbacks - § ¶

02 May 04 - 07:31On Intuition

Il nous faut une faculté qui nous fasse voir le but de loin, et cette faculté, c'est l'intuition.
-- Henri Poincaré

(We need some ability which allows us to envision the objective from afar, and this ability is intuition.)

- Private Life Of The Brain - No comments / No trackbacks - § ¶