Oct 31, 2000, 11:23 UTC
Managing Projects the Open Source Way
Oct 31, 2000, 11:23 UTC
(20 Talkback[s]) (1059 reads)
(Other stories by
Monty R. Manley
Begin at the Beginning
"A beginning is a very delicate time."
--Princess Irulan, from Frank Herbert's Dune
There are two words which don't get much play in Linux development projects:
Requirements and Design. Not surprisingly, it is the lack of these two things
which causes most of the problems in open-source software. Too many developers
in the Linux space start programming before they have a good idea of what their
project functions and capabilities are. In other words, they start in the
middle, not at the beginning.
An overwhelming tendency in the Linux world is to focus on the act of
programming as the sine qua non of the development process. The discussion is
always about the code: how to write it, how to maintain it, when to release it,
toolkits to build on, languages to use. The various discussions almost never
turn on issues like top-level design, user requirements, and life-cycle
There is an unspoken but clear belief among many in the Linux development
community that OSS development is fundamentally different than closed-source or
commercial software development. There is a feeling that the rules are
different, that Linux is somehow exempted from traditional development
pitfalls. This is obviously not true, and it has led to an increasing spate of
problems in Linux and other free OSes.
A project's success hinges on whether it has been properly begun. Many
high-profile open-source projects (Mozilla particularly) have been burned by a
lack of attention to high-level requirements and design. Beginning an OSS
project properly means that a developer must, first and foremost, avoid writing
code too soon!
The Value of Requirements And Design
Col. Kurtz: "Are my methods unsound, Willard?"
Capt. Willard: "I don't see any method at all, sir."
-- "Apocalypse Now" llard?"
Many OSS developers get exasperated when they are asked to gather requirements,
do top-level design, and produce a functional specification prior to writing
code. The reason for this is simple: it is not-coding, and therefore a waste of
time. The rationale that code is the end-all and be-all of development infects
developers with the idea that if they're not coding, they're not progressing.
This could not be further from the truth.
The gathering of requirements is an essential first step in any software
project, whether closed-source or open. How can you write a program if you
don't know what it is supposed to do? Unfortunately, when developers start
coding without having any kind of requirements in hand, the functional
specification remains open-ended: the software can never be finished because
there is no finish-line. Worse, maintainers who come afterward no roadmap to
Producing good requirements means being able to ask good questions. Bad
requirements are vague, wishy-washy, or too broad: "This software will allow
the user to do his job." Good requirements are specific and fairly atomic:
"This software must allow the user to pass information x to system y in format
d via the network, and this transfer must happen in real or near-real time."
(Not perfect, but you get the idea.)
As you might guess, gathering requirements means talking to your users.
As open-source software moves into the mainstream, the user-base becomes less
technically savvy -- they are the prototypical end-users who use software as a
tool. They are not programmers or administrators, but simply folks using the
software to get a job done. These folks usually know a lot more about their
needs than you do, so it is important to get their input. This give-and-take
process can be difficult in OSS projects (your users might be spread out all
over the globe!), which is why it's important to set up mailing-lists, message
boards, or web forums to collect input.
Requirements also help to identify issues like localization and accessibility.
Will this software be used by non-English speaking people? Are alternate
character-sets required? Will seeing-impaired individuals be able to use the
software? Are there artifacts in the software which might offend others (such
as icons or bitmap pictures)? Your users may bring some of these issues up, but
it's your job to make sure the questions are answered (even if the answer is
Once you have a fairly complete set of requirements in hand, the next step is
to produce a top-level design. Again, it's still too early to write code! The
top-level design is simply a kind of schematic -- a flowchart, box-diagram, or
simply a few descriptive paragraphs -- which boils down the essential functions
of your program. It identifies inputs, outputs, and the main processing
modules. This design can help you to identify and problems like bottlenecks and
missing I/O right up front.
For larger projects, it's usually a good idea to follow the top-level design
with a detailed design for each module in your software. It serves the same
function as the top-level design, but it helps to modularize the software and
make the flow of data more efficient.
Lastly, the task is to turn the design into functional specifications. This is
what you or your team members will write code from: the software blueprint, so
to speak. These documents are so critically important to any of non-trivial
software that it boggles the mind that so few Linux people use them. "Seat of
the pants" programming may be fine for one-offs and learning exercises, but if
you're at all concerned with longevity and maintainability, functional
specifications are absolutely essential.
Be aware, though: functional specifications are not code. They are descriptions
of code, with perhaps some difficult processing in pseudocode. It is the job of
the programmer to turn the functional specification into code.
If all this sounds like the project lead does more paperwork than code...well,
that's often the case. Many excellent project leads never write a line of code
because managing the project is a full-time job. On larger projects, the
requirements/design phase can sometimes encompass more than 50% of the entire
process. So you can see that leaving these steps out can mean an almighty big
hole in the project itself.
Building And Managing the Team
"We few, we happy few, we band of brothers."
-- W. Shakespeare, "King Henry V"
To say that a good team is critical to the success of an OSS project is
what is known as a BGO: a blinding glimpse of the obvious. But many OSS
projects fail because of friction within the teams, and this friction can be
alleviated somewhat if some simple ground rules are established up front.
0. The ship can have only one captain. As a rule, OSS projects do best when one
person is the clear leader of a team and makes the big decisions (design
changes, release dates, and so on).
- The project founder and the project lead do not necessarily have to be
the same person. In fact, the founder of a project is often too close to
the issue at hand to make objective judgments, or may lack some project
management skills. This is a big problem in teams of more than a few
people. The first thing a founder should do when starting a new project
is find a good project lead if he does not feel he can do the job
- Each team member should know the rules and responsibilities. Many larger
teams often designate roles to team-members: buildmaster, webmaster, and
so on. If a team-member cannot meet his responsibilities for whatever
reason, an alternate must be designated to pick up the slack.
- Establish a community. Since OSS developers often live in very different
parts of the world, it's important that the project establish some
"place" -- an IRC channel, a web-forum, or a mailing-list -- where the
developers can meet and exchange ideas. Many OSS teams never cohere
because the developers never really feel part of a team. Nothing beats a
face-to-face meeting, and this should be done if at all possible.
- Give everyone a goal. OSS projects tend to just laze along with no
delivery dates or milestones. This tends to lull some developers. Giving
developers target-dates for delivery of code is valuable because it
gives them a target to hit -- even if they miss, it provides good
variance data for the next target.
- Make sure every team member knows that he or she is valued. Some
projects are dominated by "prima donnas" or outspoken programmers who
tend to overwhelm others. This can lead to hurt feelings, flame-wars,
and even project-forks. Everyone needs to be recognized from time to
time; even an e-mailed "attaboy" for a good patch can do a world of
- Don't carry dead weight for too long. Not all programmers are created
equal, and some team members just might be the wrong fit for the
project. Their coding skills might not be up to par, they might have
personal issues, or may simply have philosophical differences with the
rest of the team. The project lead needs to stay on top of such
situations. Sometimes, the best solution for all concerned is to remove
a team member. This should be done only by the project lead, and only
after all other alternatives have
The above items are common sense, but it's surprising how few projects make an
effort to build good teams. Good teams don't just happen: they are built and
Testing Is Your Job
"About suffering they were never wrong,
The Old Masters: how well they understood
Its human position; how it takes place
While someone else is eating or opening a
window or just walking dully along..."
-- W. H. Auden, "Musee des Beaux Arts"
Crashes are bad. Obscure, inappropriate, or improper software behavior is bad.
Security breaches are bad. The good news is, good unit testing can alleviate
about 98% of these problems. The bad news is, not many Linux developers bother
to test their code in a methodical fashion.
Many large projects have test plans as an integral part of the detailed design.
As modules are completed, they are run against the test-plans; if they pass,
then by definition they meet the functional specification. The cannot be
considered as release-quality until it passes the test-plans.
In the Linux community, however, testing has devolved onto the users: the so
called "release early, release often" doctrine. The thinking goes that getting
code into the public eye right away means that bugs will get fixed faster and
the program will thus mature much faster. This whole concept was fine in the
days when Linux developers and Linux users were pretty much the same group of
people, but now that Linux is in the mainstream, this attitude needs to change.
Nowadays, the users are probably just folks trying to get something done.
Making them learn to read code or parse obscure initialization files is just
perverse. We must assume that our users' time is at least as valuable as our
own, and anything which wastes that valuable time is bad and wrong. Testing is
a way to save the user time; we find and fix the bugs so they don't have to.
Testing also has more immediate benefits. If each developer is responsible for
unit-testing modules and making sure they pass formal test plans, then the
design can be verified incrementally and problems found and fixed much sooner.
An error in design amplifies the longer it goes unfixed, so it pays off to find
it early. It saves a lot of hair-pulling later.
Finally, a careful unit-test can uncover such security exploits as buffer
overruns and dangling pointers.
A good test of each module should include the following:
- Boundary conditions. Always check maximum buffer lengths, data
conversions, upper/lower boundary limits, and so on.
- Inappropriate behavior. Even if the program works correctly when the
user follows the rules, users often make mistakes: they enter the wrong
value, click on the wrong button, or forget to enter some piece of
crucial data. What happens if the user enters a nonsense our
out-of-range value? What if the software receives an
incorrectly-formatted message? What if a file-write or database-update
- Check for graceful failure. Software should never just crash and leave
no indication of what happened. Whenever possible, the program should
fail gracefully and give the user some idea of what happened.
- Standards conformance. If the program has a UI, does it conform to a
standard? Are the fonts, button placement, etc. conformant? Do the GUI
elements in this module act the same as in other modules? If the program
has no UI (if it is a daemon, for example), does it communicate with the
system in standard ways? Does it enter and exit gracefully? Does it
"play nice" with the other daemons on the system?
Most developers know these things, but rarely put them to use in debugging.
Debugging is tedious, boring, and often maddening, but it is also essential to
robust software. And the above rules apply to integration and system testing as
well as unit testing, just on a larger scale.
A Project Lives (Or Dies) On Documentation
"It does not do harm to the mystery to know a little about it."
-- Richard Feynman
A programmer's favorite refrain to a complaining user is "RTFM" (read the fine
manual). Unfortunately, the OSS world largely lacks the manuals the user is
supposed to read!
Most programmers hate writing documentation. There are lots of reasons for
this, but in the end it boils down to either a lack of writing skills,
laziness, or both. (Most often both.) Linux is notorious for shoddy,
incomplete, out-of-date, and inaccurate documentation, and with good reason.
Whereas commercial companies can hire technical writers to produce documents,
Linux has to rely on the developers themselves, but they often just don't
Why is documentation important? What is "documentation" anyway? Does that mean
commenting code in a rational way? Does it mean writing good manpages where
appropriate? Does it mean writing API documentation? Should the developer
really be responsible for all that stuff?
Documentation is really just a way of detailing and illuminating the subject at
hand. Everyone needs documentation. Maintainers - even those who are expert
programmers - cannot dive into a massive pile of code without some higher-level
document to help light the way. Project leads need documentation to track
progress and determine how well the project is meeting the top-level design.
It's almost a given that users will require documentation to install and use
the program effectively.
Things that are self-evident are also self-documenting, but precious few things
in software projects are so simple. Writing documentation is an integral part
of writing software; it's just that programmers tend not to extend the
documentation beyond the source code itself. This makes maintenance very
difficult, and raises the bar for new developers too high.
Documentation should be integrated into every step of the development process.
At least three documents are essential: a developer's guide (to explain the
API, naming conventions, toolkit issues, and so forth), a user's guide (a
comprehensive guide for users on how to operate the software), and the
Requirements/Design document specified earlier. These documents should be kept
with the code-tree in whatever change-control system you're using (most often
Bear in mind that these documents must be "living documents" which change and
adapt as the project itself does. If the project lead spreads out
responsibility for documentation to every member of the team, it tends to get
done more consistently. For example, one rule might be that if a module is
updated by programmer A, then programmer A is also responsible for updating
both the developer's guide and the user's guide.
A project without documentation is almost guaranteed to fail.
documentation isn't something that will magically get done at some later time.
It's not an intrusion in your development process ("I don't have time to write
docs!" I can hear a programmer whine). It is an integral and vital part of
development, and should be treated as such.
OSS Lifecycle Management
"Thales said there was no difference between life and death.
'Why, then,' said some one to him, 'do not you die?' "Because,"
said he, 'it does make no difference.'"
-- Diogenes Laertius (circa 200 A.D.)
Most software lives far longer and plays larger roles than its developers ever
anticipated. That's why Linux developers need to take a longer view and bring
some sanity to their release procedures. Linux has made a mockery of the
Alpha/Beta/Release Candidate/Release cycle.
Let's state it plainly: Alpha is not Beta. Beta is not Release. Repeat that
several times until it sticks. Then mumble it under your breath all day long
until it sounds like a children's nonsense rhyme. Keep saying it until you
dream it and hear ghostly echoes of it all the time.
Feature-incomplete (or API-incomplete) software is often referred to as
"pre-Alpha", and generally, it should never live outside the development team.
Alpha software is feature-complete but (perhaps) only partially functional.
Beta software is feature-complete and functional, but is in the testing cycle.
Release-candidate software indicates that software has one or more beta-cycle
tests and is thought to be ready for production use.
Linux developers seem to be at a loss as to how to designate software as Alpha,
Beta, or Release. The whole "release early and often" motto has turned formal
release methodology on its head. What we need is a commitment to release
software in a more regularized way - the users need this information to plan
for deployments and future upgrades.
Part of this commitment means giving at least ballpark delivery-dates and then
meeting them. Users need to have at least a general idea of when software will
be usable. And developers have a double role now: not only to work on the next
major revision of software, but to respond to bugs in the current release and
provide patches. They can juggle these tasks better by being able to prioritize
A key issue in this process is responsiveness. Simply saying, "I'll do it when
I get around to it" won't do at all. It doesn't matter if you're working on the
project for fun, are doing it at night for no pay, and have a day-job that
takes all your energy. When you make software available to the general public,
there is an implicit promise of quality: "This software will do what we say it
will." Responding to user concerns in a timely way reinforces the ideal of OSS
as a better way than closed-source software.
Understand, responding to a problem and fixing the problem are not the same
thing: a fix may take a long time to implement, but the user will be happier
knowing that his or her concerns have been heard.
Learning From Failure
"We learn wisdom from failure much more than from success. We often
discover what will do by finding out what will not do; and
probably he who never made a mistake never made a discovery."
-- Samuel Smiles (1816 - 1904)
Sometimes, despite our best efforts, projects fail. There are as many reasons
for this as there are stars in the sky: bad design, bad management, immature
technology, lack of developer talent...the list goes on and on. But the one
constant is that usually we can look back on a project and identify something
that was the main contributor to the failure. Hindsight, as they say, is always
The first rule is to avoid placing blame (even on yourself!). No one is
perfect; the whole software industry has a pretty lousy batting average. At
least you're in good company!
Secondly, recognize that the failure was probably as much administrative as it
was technical: a lack of design, focus, or goals in the project itself. Often
the hardest thing to change about developers is the mindset -- unlearning bad
habits and learning good ones.
Last, do a postmortem on the failed project and see if it can be resurrected
-- often, the underlying idea is sound but simply needs to
If there's one point I want to hammer home, it's this: don't write code too
early. It's a guaranteed way to run into big trouble later
Good software is rarely the result of brilliant programming, despite the mythos
that has grown up around the activity. Most often, good software results from
good design, good planning, and strict attention to detail: all those things
programmers find boring and not-fun. So, as developers, we have a choice to
make: have lots of fun producing mediocre (or downright bad) software, or put
up with some tedium and produce good software.
The choice is ours.