29 April 03 - 16:14One Letter Away

The theme of "emergence" is something of a hot topic among Agile advocates, possibly even a buzzword in the broader high-tech community. See for instance this, that or this other viewpoint.

Today, in a mailing list post, a fellow software professional mentioned something about a project where work was done in "emergence mode". My brain got stuck for a moment, because my initial reaction had been a diffuse approval - sure, emergence is good - while my colleague's tone was clearly disapproving.

Then the penny dropped that he'd been talking about "emergency mode" - something that, indeed, I generally disapprove of, evoking as it does such related ideas as firefighting, multitasking, overtime, etc.

I find it a strange and ominous coincidence that "emergence" and "emergency", two words relevant to project management and carrying opposite values, happen to differ by only one letter.

- The Universe And Everything - one comment / No trackbacks - § ¶

28 April 03 - 13:51Certifications, degrees, teamwork

A while ago, Brian linked to Cem Kaner's Final Exam for Software Testing 2 Class. I've been thinking of doing the exercises in that, if I find the time, to see if I'd pass.

Certifications and exams are a touchy topic for me, because I am underendowed in the "official recognition of educational efforts" (aka degree) department, something which I have had to deal with in steering the course of my career. The last bit of paper I got in exchange for learning something was back in 1988; the french Baccalauréat. Recently I was asked to produce it for some administrative reason, and realized I couldn't even do that - I seem to have mislaid the actual physical thing.

So at best I'm only able to prove that I was learning new stuff until 1988. As far as the official system is concerned, the past fifteen years are an educational blank for me.

Recently something was passed into French law which ought to be very useful to me: a "validation" deal whereby people who've had a professional career longer than five years can apply for a degree, without having to display proof of the prerequisite degrees and in some cases without having to sit through an actual course. In essence, a way of officially recognizing that I haven't been at an intellectual standstill for fifteen years.

Well, last time I inquired, the people at the university's information desk laughed out loud. They told me that people have to sit through five years of boredom and mind-numbing paperwork to get their Master's degree; and they don't get paid for any of it. If the university started handing out degrees to people who have fun working on real problems, and get paid for that on top of it, the actual students would be rioting in the streets. So that was that for the time being. I'll be trying again this year, in case the university people are more aware of the new laws now. I'm ever the hopeful person.

Anyway, there was something in the text of Cem's exam which didn't sit well with me:

It is essential that you work on this exam ALONE with no input or assistance from other people. You MAY NOT discuss your progress or results with other students in the class.

Some time after I'd left the official education circuit, I came to the conclusion that it hadn't been that much of a loss, since it had utterly failed to equip me with the skills that were to prove most crucial in my work as a software professional.

Among the chief such skills were those related to teamwork, and working in collaboration with others. I'm pretty sure that collaboration and teamwork are as important for a tester as they are for a software developer.

And yet, I find in Cem's "skills-based" exam one which could very well be passed by a high-functioning autist. Color me puzzled.

- Software Development - No comments / No trackbacks - § ¶

27 April 03 - 19:16Power programmers

Continuing my inquiry into programmer productivity, I came across this article on "the fabled 10x programmers of yore". (Via Why Smalltalk.)

I'll quote a little from the article to show how little it does to allay the suspicions I outlined, or the additional concerns Frank listed when he chimed in on the topic.

I kept track of which programmers were contributing the most to the code we delivered to our customers. [...] Almost 75 percent of the final code came from just 5 percent of the programmers. And just about 100 percent of the "best" code (few initial defects, excellent efficiency) came from this same small group.

From that quote, Parkinson's measure of productivity seems to be nothing more sophisticated than how many lines of code each programmer wrote. As a metric, LOC leaves much to be desired.

One of my favorite design activities is refactoring; I once took part in a project to "debug" a program, in the course of which the line count shrank from 40,000 to 30,000 while keeping functionality essentially unchanged (we added a few features). By Parkinson's standards I would rank as a negative productivity programmer ! From my point of view, I had eliminated much redundancy and made the program easier to maintain in future - a significant, if less easily measurable, contribution to the overall value of the program.

Parkinson's reasoning gets interesting:

If just a few people were writing most of the code, what did the others do? Maybe if I could just identify the right people, I could get by without the rest. [But] there was some work Power Programmers just wouldn't do—testing, documentation, release management and so on—that needs to get done if projects are to be commercially successful. The less accomplished programmers often ended up with in these roles.

I find it somewhat scary to think of the quality of the work that must have been performed in crucial areas such as testing, if testing "ended up" being the province of "less accomplished programmers" because the Power People looked down their noses at such lowly assignments.

It would probably be more reasonable to factor into "productivity" measures all of the activities that contribute to the value of a piece of software, and not just the production of code: negotiating requirements, establishing testability, improving design, ensuring proper integration.

There is a "smoking gun" pointing to the fallacy of the whole thing, and it can be found early in the article:

[...] the productivity of the teams didn't vary much (and everyone worked about as hard as everyone else)

This, to me, is the one interesting finding from all that data, and it goes completely counter to what we might naively deduce from an apparent "10x" effect. The "productivity" of teams is actually pretty constant across an entire organization, which suggests that "Power Programmers" in fact have very little effect on how fast the whole system, from one end of the chain to the other, is capable of churning out software.

- Software Development - one comment / No trackbacks - § ¶

24 April 03 - 19:50A software craftsman

My friend Dave Astels has started a blog.

I met Dave at XP2002. Dave is a software craftsman - the first I've met to present himself as such. I had been aware, vaguely, of the notion of software craftsmanship, and subsequently read more about it in Pete McBreen's book of the same name.

I also met Dave's apprentice, Pat. It was quite something to directly witness the relationship between a master craftsman and an apprentice; it's definitely different from mere mentoring or teaching or training. I feel sure that an apprenticeship would do me a lot of good, if a suitable master craftsman could be found.

The "craftsmanship" paradigm makes for a very interesting contrast with the dominant "engineering" paradigm. On the other hand, I wouldn't want to have to choose to describe myself as either a craftsman or an engineer.

How about you - how would you describe yourself ? Craftsman or engineer ?

- Blogversations - No comments / No trackbacks - § ¶

23 April 03 - 20:22My work

What is it that I do ? What is my work, and how does it differ from, say, an airline pilot's ?

I'm a programmer. (Some might say "just" a programmer. Others might prefer coder, or developer - they're all pretty much the same to me; close enough for rock'n'roll, as the saying goes.) But just what does that mean ?

If you had asked me, say, ten years ago, I might have answered this : "I'm a programmer. I work with computers. I relate well with computers."

Today, this would strike me as a sterile and demeaning description; nor is it all that accurate. I have changed, and computers have changed, to be sure. But the fact is that I was never interested in computers as physical stuff, in the hardware. I lived some of my happiest programming moments when I was writing server-side code to run on a box that happened to sit somewhere in California, from my lonely office in Paris. You can't get much more remote from the hardware than that.

How about "I work with software" ? That isn't very satisfactory either. I discovered something about myself while I was sitting in that one-person office in Paris, coding while my partner in San Francisco was running the actual business. I discovered that, even though I thought of myself as shy and introverted, I liked my work best when I was around people. In fact, I discovered that I needed to work with teams.

What grabs me about computers and software is the conversations. I view software primarily as a linguistic medium. People express themselves through software - through writing it, and also through using it. Whenever I use software, and whenever I write software, I am joining a conversation mediated by computers.

Such conversations have their peculiarities; for instance, they often involve people who are not physically present. (This might be one reason why this kind of conversation is particularly appealing to me, as a shy, introverted person. Or it might not.) Still, I find that the conversations are most enjoyable when I am at least occasionally, and preferably often, in direct interaction with the people involved.

This seems to be suggesting some answers to the question, "What is my work ?". For instance, on one level at least, my work is different from the airline pilot's in that her (or most likely his) work does not primarily consist of conducting a conversation, but of operating an aircraft - reading instruments, working controls. I suspect that this might be somewhat naive - instruments and controls probably constitute as much of a linguistic medium as software does; and of course a pilot does engage in conversations, such as with air traffic controllers - but let's just say that the pilot's work does not consist as obviously of engaging in conversations as mine does.

Ultimately, what characterizes the conversations I engage in is the medium, and the kind of people I have conversations with. So, one part of a good description might read as follows: "My work is to engage in conversations in large part mediated through software and computers, with people whose work also consists of such conversations." This isn't all there is to my work - there are large parts missing from the description - but it's one of the key parts.

What is your work ?

- The Universe And Everything - No comments / No trackbacks - § ¶

22 April 03 - 20:47Hammers and hammering

We do not relate to things primarily through having representations of them. In driving a nail with a hammer (as opposed to thinking about a hammer), I need not make use of any explicit representation of the hammer. My ability to act comes from familiarity with hammering, not my knowledge of a hammer. This [Heidegger's] skepticism concerning mental representations is in strong opposition to current approaches in cognitive psychology [...].

Found in Understanding Computers and Cognition, and slightly related to this.

- default - No comments / No trackbacks - § ¶

21 April 03 - 22:11Programmer productivity

There's a thread currently kicking around on the XP mailing list about the "1x - 20x postulate", the assertion usually stated as "some developers are twenty times as productive as the others". (The factors given vary; I have seen "ten" most often, but occasionally you'll see this given as "a hundred times".)

I feel deeply uneasy whenever I hear someone make this assertion. Most often this is due to the context, though I also have more substantial reasons for wondering about the validity of this "observation".

The context thing is this - most often the "postulate" is advanced as support for a claim that strikes me as totally unrelated. As a case in point, the thread on the XP list was spawned when one person stated that since there are these one to twenty productivity differences, one shouldn't treat developers on a project as mere mechanical "resources", replaceable and pluggable at will.

I happen to agree that developers aren't "Field Replaceable Units", and that no effective manager can afford to treat people as such. But it seems to me that the "postulate" lends no support to that position - indeed, I think the opposite is true : if there was a substantiated basis to the "one to twenty" claims, such that one could actually measure a programmer's productivity, then it would be a lot easier to treat developers as fungible resources. Remove one "ten" developer, replace her with two "fives"; if you can scrounge a few thousand dollars (or euros) in salary costs in the operation, the project is no worse off and you're a hero to Accounting and higher management.

But this brings me to the substantial reasons for my uneasiness. First of all, we have no objective measure of productivity - possibly because rarely even bother to try measuring the value delivered by software projects. Lines of code and function points just don't cut it, as one poster on the XP list remarked, if they are buggy code or the wrong function.

Assuming the postulate to be true, what does it tell us about programmers' varying productivities over time ? How long does it take for a "one" to become a "twenty" ? Can that be done ? If you are a "twenty" in Java, does that correlate to your score in other languages ?

Then there is the question of the intrinsic usefulness of an observation about the extremes of a range, without any further details of the distribution. If I tell you that contemporary adult humans range in size from 60 to 270 centimeters (a factor of more than four between shortest and tallest), have you learned anything that will help you manage populations of typical human adults ?

Most important to me, the "postulate" is never stated in terms which would have it make sense to me: in terms of one person's contribution to the productivity of a team.

I'll probably come back to this topic. Is your team composed of ones, or of twenties ? Do you care ?

- Software Development - two comments / No trackbacks - § ¶

20 April 03 - 18:04Information leaks

I can't imagine that people would find this Shark Tank story at all funny, unless they were in the testing business.

In a similar vein Brian has posted an entry recently about how one should be careful about "background information" when writing tests.

I'm grappling with similar issues in my current work. I've been finding many tests which pass in the developer's environment, but fail as soon as they're checked out of CVS and run in some other environment, such as the "official" box where QA runs non-regression. This is because some information that the test relies on has "leaked" from the text of the test, where it belongs, and has passed on unnoticed to the developer's environment variables, local files, or other "personal" information stores.

- Software Development - No comments / No trackbacks - § ¶

17 April 03 - 23:26Credibility

I'm a developer turned consultant; but my current assignment has me working within a QA team. This is my first position in QA, and I don't have much of a background in that. From that decidedly underpriviledged standpoint, I want to take a stab at "what QA is all about".

Many development organizations comprise a person or team responsible for generating requirements; a development team who is to implement something which meets those requirements; and a QA team. Requirements are the "raw material" of a development process; and the development team transforms these into finished product. But what is QA's responsibility ?

Right now I see the role of QA as generating credibility. It's a commonplace that you can't sell anything if you're not credible. ("Would you buy a used car from this salesman ?") A development organization must be able to issue credible assessments of its products' quality. If it says "this feature works", and the feature doesn't in fact work, it loses credibility.

Software users and customers accept, or tolerate, that the software they purchase does not always work correctly. But they lose trust in the product and its vendor very quickly if, on reporting problems, they are told that the problems are fixed, and keep experiencing the problems after taking delivery of the fix.

People who are confident are more credible, and that is why I think automated tests as Extreme Programming recommends them make lots of sense. They are a way for a development team to be very confident in their assessments of the quality of their product.

I've also been thinking about this QA role from the point of view of the Theory of Constraints, spurred on by Hal, Joe and Frank.

In many development organizations I've seen, customers lose confidence in the vendor because of faulty fixes, or failed promises of various kinds - missed delivery schedules, etc. Market demand for credibility exceeds QA's capacity to provide it. That makes QA a bottleneck, or "constraint", according to TOC. I'm wondering how one might apply TOC's "five focusing steps" to such a situation.

- Software Development - three comments / No trackbacks - § ¶

16 April 03 - 23:05Projects, pictures, patterns

An information radiator is a publicly visible display of information relevant to a project or a process. Information conveyed by the radiator will be circulated within teams, or within larger groups, by "convection currents" of information.

I have been using the following information radiator to circulate the status of a suite of acceptance tests which the team has trouble automating:



The bottom right of the whiteboard displays red or green for each of the acceptance test scripts. Ideally, the test scripts should be managed from a testing framework similar to JUnit, so that at the press of a button the framework will display a red bar if some tests are failing, or green if all tests are passing. The team has been having difficulties writing this framework, with the result that the test scripts are not being run regularly. What I'm doing is running the tests manually, as often as possible, and updating the whiteboard to reflect the tests' status.

Tim Van Tongeren and Esther Derby have commented on the importance of "visualizing" project status - making things visible that are normally hidden. This strikes me as a good example, but I think there's more to the topic than meets the eye.

In a recent entry, Keith suggests using as an information radiator a cork board, to manage feature requests written on index cards. The suggestion that struck me as particularly insightful was that one might be able to spot "patterns and order" in clusters of cards, as people moved those around to reflect groupings or feature similarities.

There are those of us who argue that index cards are "nice, but really a database works much better"; who would use Excel and Lotus Notes to manage the project's information. And there are those who hold out for the low-tech solutions, and who invariably lose the impassioned arguments about which works best, because it's really not much work to enter into Excel or Notes what we would scribble down on index cards or Post-It notes or whatever. It's more readable, it's searchable, etc.

The problem is precisely that with software tools, all of the data lines up just so. All of your Excel rows are neatly aligned, every report looks just the same.

When moving index cards or Post-It notes around, scribbling on a whiteboard, sticking colored pins on a cork board, and so on, the inherent messiness adds a huge amount of "analog" information to whatever is on an information radiator, over and above what could be faithfully recorded in electronic form.

Awareness of subtle patterns is a wired-in feature of our brains, and so I'd bet that we are able to read a lot more from a messy, hand-drawn or hand-ordered display of project information than we could from a database.

- Software Development - five comments / No trackbacks - § ¶

15 April 03 - 18:24Attention ecology

George Por has posted a thought-provoking entry on "attention ecology".

George contrasts "attention ecology" with "attention management". The latter looks very similar to the model I use explicitly when I have to make decisions about how to use my time or focus my attention.

The model says there are only 24 hours in a given day, and all we can do is decide how we are going to spend them. If we divide them up, at random, into a thousand little things, chances are we're not going to do any of these thousand things properly. Rather, we need to know what the important things are in each of our days. We need to put aside sufficient attention to each important thing, so that each gets really and completely done.

George Por calls this "managing the slices" of a pie diagram. It's basically what the parable of the rocks and sand recommends, or Mark Forster's ideas about "how to get everything done". Or, from a different angle, it's why Agile software processes recommend timeboxing.

What I found provoking was the notion developed in a further entry that, even if "managing the slices" is the name of the game for a single person, larger numbers may be able to "stretch" their attentional budget.

This reminded me of the claims regarding pair programming and flow. For instance, when a pair has gotten into its rhythm, it is much more resilient to interruptions than a single person would be.

How are you managing your attention ? Have you experienced "shared attention" in pairs or in teams ?

- Private Life Of The Brain - No comments / No trackbacks - § ¶

14 April 03 - 19:12When tools are too versatile

I experienced a sinking feeling a few days back, browsing through the acceptance test scripts for a project I'm currently consulting with - a "Oh, no" kind of feeling.

The team's test scripts rely on Ant tasks that allow for coding rudimentary logic: loops, conditional branches, exception processing.

Ant is a build tool. Its purpose in life isn't to be a scripting language - in fact, the Ant tasks mentioned above are actually part of a distinct project, and "kept separate from core Ant for ideological purity", to quote the Ant documentation.

There is a mismatch between the tool and what the team has been using it for. Now, the team was initially interested in Ant because of its cross-platform portability and integration with JUnit, both of which are relevant in our context. So the mismatch reflects badly on neither the tool nor the team. In fact, they have been concerned for a while now about how complex it was to develop the acceptance test scripts...

Mistakes in selecting tools do happen. But a problem peculiar to software tools is precisely how flexible, how extensible our tools are. If I need a screwdriver and I select a hammer, I probably won't get anywhere and will eventually realize my error. The problem with software is that we are able to turn a hammer into a screwdriver.

When I started my programming career, my reaction to such protean tools was usually "Oh, neat !". Today, I find that experience has turned me around completely - I no longer care much that X can do Y; and when I find that someone has been using X to do Y... "Oh, no !"

What tools do you know that are more powerful than they should be ?

- Software Development - one comment / No trackbacks - § ¶

11 April 03 - 09:21Intervals

My cable modem broke down one week ago. It was only replaced this morning. I try to spend as little time as possible on non-work-related use of the Internet at work, so I was effectively without an Internet connection for a week.

I wrote about batches recently, and the incident made me realize that there is a flip side to batch sizes - the intervals between two instances of a recurring task.

I usually spend about one hour each day on reading emails, updated Web sites, blogs with new content, and technical newsgroups. From day to day the "size" of the task remains about the same.

What's interesting is that this morning, I got mostly caught up on one week's accumulated email, etc. - in one hour. Of course, I didn't spend as much time on each item as I would have; I made fewer and swifter decisions as to whether each item was something I should read, archive, or delete.

The expenditure of effort was the same, for a larger batch size, but also for a much reduced "quality" of the task performed.

Do we encounter similar effects on projects ?

- Software Development - No comments / No trackbacks - § ¶

07 April 03 - 14:52Autopoietic networks

I think that the phrase Alan and Frank are looking for is "autopoietic networks". At any rate, it's an interesting phrase to know when pondering what is being pondered; out of Stuart Kauffmann's "At Home in The Universe".

- Blogversations - one comment / No trackbacks - § ¶

06 April 03 - 16:23Interacting with systems

Every component of a system contributes to that system's overall dynamic.

Among other consequences, this means that every part of a system is a potential "entry point" for exerting influence on that system.

This is one notable reason I can't agree with people who - of the teams they work with, of their employers, of their families - say "I can't change anything, even if I wanted to - the problem isn't with me, it's with the system".

At the very worst, each of us is one obvious place to start exerting influence on the systems we find ourselves in, if we're looking to change these systems. Or we can look to the elements of the system closest to us - coworkers, bosses, spouses, kids.

A real problem is that it's not obvious what shape or form our influence should take, in order to move these systems in directions that suit our purposes. But the locus of that problem is quite accessible : ourselves, our attitudes, our knowledge about systems in general and the particular systems we want to influence.

- The Universe And Everything - No comments / No trackbacks - § ¶

03 April 03 - 12:47Screening out

There's something about Johanna's advice on attracting suitable candidates, when hiring technical people, that doesn't sit well with me. Points #3 and #4 in particular.

Johanna talks about "people screening themselves out", and suggests that when advertising an open position, the ad should list the "requirements" of the position such that only suitable candidates apply. She further advises working with contract recruiters to handle the "screening out" for you.

The Pragmatic Programmers wiki has a wonderful skit on how this can go awry.

I've had two relevant experiences recently, one pleasant and one not. In one case, I spotted an ad calling for a "QA Engineer" experienced in JUnit and Agile methodologies. As an XP coach in a country that has yet to see much demand for such, I was naturally curious.

The ad had been placed by a contract recruiter; I inquired; I was turned down because my profile did not fit the "QA Engineer" label - i.e., I was obviously not a tester. I insisted politely, the recruiter (quite a friendly person) agreed to trust me and forward my information to the hiring organization, and a few weeks (and some meetings) later we have closed on a short-term consulting deal.

The unpleasant experience was with a recruiter who contacted me on the basis of my postings to a technical newsgroup. I sent my CV, and back in a flash comes the reply that I am "not qualified for any of our positions, all of which have a hard requirement for at least a master's degree". Why is this person contacting potential recruits on the basis of technical mastery if that isn't one of the primary requirements ?

I think that hiring shouldn't be a matter of "screening out" from an inevitable deluge of candidates, too often on the basis of criteria which have little to do with what really matters, such as degrees or "checkbox" skills. As I've alluded to in my thoughts on the job non-market, hiring or being hired is a commercial transaction; that involves a conversation, a negotiation. Recruit solves a problem for hirer (such as a capacity shortfall), for a fee.

The best way to start the conversation is for the hirer to explain the problem they have, and expect the recruit to prove they can solve the problem. What means of proof the recruit wants to line up is part of the solution, not part of the requirements; as in XP, the solution should be up to the provider, and the problem belong firmly to the customer.

- The Universe And Everything - three comments / No trackbacks - § ¶

02 April 03 - 12:50Talking to objects

Keith's MemoRanda quotes, at length, from a post by Chris Uppal on his "Great Leap Forward" to Smalltalk.

Chris touches on a topic I, too, could wax emotional about: the deep, nearly numinous satisfaction of engaging objects in their organic ecosystem, the Smalltalk image.

Reading Chris' post, or Keith's commented excerpts, might give you an idea of what it is like to code in this manner. The sad truth, though, is that all too likely you will only resonate with the descriptions if you are already familiar with the Smalltalk experience. Or so I suspect; I'd love to be wrong, may Chris' prose have the power to convert masses.

Still, it gave me pause when I read the following.

"Even if I designed and wrote the classes, once an object has been created it has an independent existence. I can talk to it, I can ask it to perform operations, but it is separate from me. In a sense it is "my" object, but it is only "mine" in the same way that a pet, or a rose bush, or a table, could be "mine". [...] The way you "talk" [to objects] is by sending messages written in the Smalltalk programming language, but that's almost incidental. The important thing is that you are communicating with them using an interactive text-based medium, like using an IRC [chat] channel."

These excerpts reminded me of my own Great Leap - and that it had came about through contact with an environment not directly related to Smalltalk. An environment, in fact, which even readers less than enthusiastic with the prospect of learning a "professional" language - one little used by businesses to boot - might be curious to explore and have fun with.

The environment was LambdaMOO. It's a chat room - and an entire world of fun and discoveries. I can't begin to describe it. Check it out.

A warning, though. You might become a better programmer as a result.

- Object Lessons - No comments / No trackbacks - § ¶

01 April 03 - 11:28Mathoms

I'm rereading The Lord Of The Rings.

A mathom is what Hobbits called "anything that [they] had no immediate use for, but were unwilling to throw away". I want to hijack the term and add it to the software developer's jargon.

As a software developer, I have a tendency to be a mathom-keeper. (Not only as a software developer, if the truth be told.) I used to have source code in my projects that I had no immediate use for, but was unwilling to throw away. I kept commented out code - methods or method fragments. I kept source files that were still in the project tree but are no longer compiled or linked to.

I have acquired the habit of relegating source code mathoms where they belong : to the software developer's mathom-house (so Hobbits called their museums), also known as the "version control system". I delete commented-out methods (indeed, I no longer comment out what I don't need). I remove unused files from the project tree.

This turns out to be very effective in reducing the complexity of a project, and as a consequence the opportunities for defects and mistakes of all kinds. Mathom-free code is easier to understand and maintain.

Can you remove a few mathoms from your project today ?

- Software Development - No comments / No trackbacks - § ¶