For the better part of two decades now, I've been in the software industry as a developer, engineering manager, product manager, and now as an entrepreneur. Over the last six years, I've been leading software development teams using so-called Agile techniques.
When I first started with Agile, the bleeding edge literature was on Extreme Programming, which at the time I found a bit too, well, extreme for my projects, but with a lot of useful ideas that resonated with my experience. Later I discovered Scrum, which added some project management principles that really helped solidify my thinking on the subject.
What really sold me on the concepts of Agile methods was the emphasis on rapid feedback and people over process. My experiences up until that point told me that various methodologies I had tried (Information Engineering, Strutured Development, RAD, Rational Unified Process, etc.) only seemed to work as well as the individuals on the team.
I saw that a small team of good people seemed to outperform the most disciplined process, toolset, or philosophy. A bad team usually failed to produce a good result, regardless of what magic process was applied.
But what I also noticed was that even a great team that wasn't focused on delivering a solution incrementally could also fail.
About 10 years ago I was on a team with outstanding developers building a workflow management product. We built a solid architecture, complete with a portable persistence layer, cross-platform GUI widgets, and cutting edge object-oriented design.
As months went by we were making great progress from the bottom up. The widget set was portable across Windows, OS/2, and Motif platforms. The data store could be hosted in an object database, Oracle or DB/2. We had the coolest "smart object" framework for managing memory. Even the basics of a GUI modeling tool for designing interactive workflows, and a workflow runtime engine and API.
But as time passed, management started getting nervous. Where was the product? Did it really make sense to keep working on this stuff when Web development was becoming the new hot topic?
Since we were so focused on building the rich, robust infrastructure and frameworks for what we "knew" would be needed, we didn't have much to show when the VP of Engineering and CTO would come by looking for a demonstration. It probably didn't help matters that my manager at the time would aggressively refuse to demonstrate incomplete software.
Now, we were all experienced developers with project successes under our belts. But we were determined to not repeat the problems of previous development efforts. In hindsight it's easy for me (and probably for you) to see what we did wrong, but it seemed perfectly reasonable at the time to design the system up front, complete with a fully portable, flexible, robust, cutting-edge infrastructure design.
Ultimately the project was canceled after about nine months of work. Purportedly this was due to the desire to focus resources on the then emerging "Internet fad." In truth, I believe that our failure to demonstrate consistent progress in the form of working software was the death blow. It's not that we were necessarily wrong for wanting to build great software, it's more that we forgot about the need to DELIVER.
But what's all of this have to do with methodologies and small teams? Simply this - methodogies are no good without talented people. And talented people alone can flounder without some structure to guide their efforts.
Agile methods embrace individuals and interactions, while providing easy to understand structure and practices to guide a team. For example, frequent releases for feedback might have saved my workflow project from the chopping block. Certainly it would have focused the team on prioritizing things like working software over world-class engineering infrastructure.
The combination of small teams with lightweight, disciplined process is a force to be reckoned with. Even large projects can benefit from dividing the work between small feature teams, as long as the work is coordinated.
Over time, I've noticed that as small teams become proficient in delivering software, sometimes the process scaffolding gets torn away, and to an outside observer, it seems as if the team isn't using a process at all. In reality, the team is operating so efficiently that informal communications, minimal documentation and impromptu meetings are good enough.
This is where teams just starting with Agile methods can get into trouble. An inexperienced team may well be served by using a little more structure to get started, and adjust as needed later based on real world feedback. A professional circus performer can walk a tightrope without a net and make it look easy. But would you bet your life trying to do the same?
As Agile methods go mainstream, I'd hate to see projects try to adopt the techniques without an awareness of the original context - small teams working with frequent feedback, close communcations, and respect for each other and their customers.
This doesn't mean that Agile can't be scaled to large organizations - I've personally been involved in doing this successfully. But methodologies are only part of the story - we can't forget about the human side. Smaller teams of about 3 to 8 people are a better way to collaborate.
And given a reasonable degree of structure, it is the people that ultimately make the difference between a successful project and another disappointing statistic.[Digg this article]
For more on agile tools and techniques: http://www.extremeplanner.com
, project management
Get your copy of the new book! Agile Thinking: Leading Successful Software Project and Teams