Agile Project Planning

ExtremePlanner: Agile Project Management for Distributed Software Teams
Click For Your Free Online Trial

Monday, June 19, 2006

Managing Defects on an Agile Software Team

I've been thinking about one of the under-discussed issues with Agile software development, which is how a team actually manages defects discovered after release.

Much of the discussion has centered around preventing defects in the first place, through Test Driven Development, Customer Acceptance Tests, and other three-word phrases intended to improve quality. These are all extremely valuable, but I've never seen or heard of a software team that didn't let a defect out into the wild.

So, I'll describe a process I've used for managing defects, and I hope you'll share yours as well.

Carl the Customer sends an email with an issue he's been having with the software. At this point, it doesn't matter whether it's a software bug, a documentation problem, or a user error, I just want to understand what Carl is trying to do, and why he is frustrated.

For example, Carl could say "I want to rescehdule all appointments to next month, but the software makes me do one at a time." This is clearly an enhancement to the product in software terms, but to Carl, it may as well be a defect, since he can't do what he wants, and it hurts.

In another scenario, Carl might say "I tried to reschedule an appointment to next year, but I get a message that says Internal Error -376." OK, we seem to have a software defect now. If the software can't handle rescheduling items more than 365 days away, maybe it's just a bad error message, or if the spec says it should be able to anything, maybe it's a true failure. Either way, Carl is in pain.

So my approach is to treat all of these situations similarly, regardless of the label you might apply, "bug", "enhancement", "spec problem", "cosmetic issue", "feature request".

In a typical agile process, there is a "backlog" of features, enhancements, and defects. As part of the plannning process, the Customer/Product Owner/Decider will prioritize the items on the backlog for the next release.

Now, in the case of defects, teams have often resorted to a DIFFERENT backlog of "bugs", and managed it in some other manner. I believe this can be problematic, especially if the Customer is not involved in choosing the priority of these items. Without customer involvement, the team may be spending large chunks of the development budget on issues that aren't critical or destabilizing to the codebase.

Note that I don't consider problems found while developing or testing new code to fall into this category. Why? Because the Customer has already prioritized that new code in the form of a user story, and it's not done until you've got approriate tests that all pass. So we're strictly referring to issues found in the wild.

There is one challenging aspect to this approach. As a developer, project manager, and a leader, I am driven to produce the highest quality software possible. It bothers me to see unfixed bugs in any system I am involved with. But there is a fine line between conscientious quality management, and obsessive, wasteful fixing of every defect that arrives.

As an example, I've had defects on my backlog that become irrelevant when we implement a new feature, or modify an existing one. The time we would have spent fixing those would have been wasted because: 1) The customer wasn't impacted by them, 2) The features affected were deprecated or removed, or 3) The code was replaced by a new approach or implementation.

So it's important to consider the opportunity cost to your customer of excessive defect fixing, and let them be the judge of what's important, with your guidance. Whether or not you choose to fix a defect, it is still useful to try to understand the root cause in all cases - you may be sitting on a time bomb if you don't know why or how this defect got into the system.

As an example, I once encounted a defect that would show up once every 2 weeks in production, but when it did, the system needed to be restarted. This was painful, and we tried to find the root cause, but were unable. After a few months of this, it hurt enough to invest the time in fully researching it. After a couple of weeks of one developer's time during a slow period, we were able to isolate the problem to an issue in the underlying platform software (a specific Java library).

In this case, it was still worth the effort to understand where the problem was rooted, even though ultimately the code was not within our control (we did ultimately find a tolerable workaround). So even though I'm advocating prioritization of defects, some due diligence of the root cause is important so that you can communicate the impact to your customer, as well as gain a better understanding of what the fix would require.

Even within this process, my teams have certainly fixed small problems without prioritizing, but there is a slippery slope here. We were occasionally burned by seemingly inconsequential fixes that changed the behavior of some obscure part of the system, even though all of the tests passed. That's another part of the cost/benefit equation. Fixes, just like new features, can introduce new problems into the system. But unlike new features, which typically just break themselves, defect fixes can break critical existing functions in subtle ways. Although solid test coverage can minimize this kind of regression, it can and does happen.

I believe that holistic management of your backlog, including new features, enhancements, and defects, is the most consistent way to stay on top of priorities, and effectively deliver quality software that meets your customer's needs.

How do you manage your defects?

For more on agile tools and techniques:
(Tags: , , )

Get your copy of the new book! Agile Thinking: Leading Successful Software Project and Teams

Friday, June 09, 2006

Software Misconceptions Revisited

My last post on common misconceptions about managing bugs generated quite a few responses, so I thought a followup was in order.

Some thought that my contention that many believe that all bugs should be fixed was simply a "straw man" argument. In other words, I couldn't possibly believe that anyone really thinks that.

I guess that's the problem with generalizations, so I'll be more specific.

I've worked with quite a few people who DID think exactly that, and it affected many of the decisions at those companies, and not always for the better.

In particular, one QA manager routinely prevented software from shipping because the CTO had declared a "Zero-defect policy". This meant that in theory, we couldn't ship the new version until the defect tracking system had no open items.

As you might expect, each team spent the next few days tranferring bugs logged against their areas to other potentially responsible parties. Without telling them.

This also kicked off lots of heated arguments with the QA team as to what exactly constituted a defect. I can't prove it, but I think the entire QA team got free lunches for a week. This seemed to correlate with a new level of flexibility re: "bug " versus "enhancement."

Moral of that story: zero-defect policies usually just force bugs underground. Instead of having no known bugs, you wind up with no DOCUMENTED bugs that everyone knows about.

By contrast, another company I worked for was obsessed with features. There was no time to fix bugs, we had to get out the latest features NOW! The development team was very talented, and my some miracle we were able to keep delivering. The system quality did start to show signs of fraying, and the stability was somewhat less than rock-solid.

I've seen these patterns over and over again, both in companies I've worked in, and those that my friends and colleagues have. It's a form of corporate unconsciousness, where things get done, but no one really is sure how or why.

This isn't just a management problem though, I've seen unconscious behavior at the developer level also. Unplanned "tweaks" and "cleanups" of perfectly functioning code that caused major problems in production. An extra feature added in that no one asked for, which destabilizes the system or confuses users.

Anyway, that's enough on that for awhile, but I wanted to get you thinking about ways we can develop software more consciously. It's not just about fixing bugs or not fixing bugs.

It's about approaching development with the same care we approach a by-the-pound salad bar. Load up on the high-value, tasty items and leave the heavy, bad-tasting things where they lie.

For more on agile tools and techniques:
(Tags: , , )

Get your copy of the new book! Agile Thinking: Leading Successful Software Project and Teams

Sunday, June 04, 2006

The Biggest Misconception in Software Development

Software development is a tricky profession. It seems like every year a new methodology is born, a new technology rises, and the industry annoints a new hero.

C++, then Java, then C#, now Ruby.

Procedural progamming, OOP, Component-based development, and now SOA, whatever that means.

Waterfall process, then iterative development and RAD, then Rational Unified Process, and now Extreme Programming, Scrum, and agile methods.

But one thing that hasn't changed much is the idea that software should be bug-free. Zero defects. Elegant perfection.

Sure, we all understand that perfection is just an ideal, but we still should strive to fix every bug that we know about, right?

Well, it's this idea that is the biggest misconception in the software industry. Not all bugs should be blindly fixed without considering the consequences.

Consider an example.

You create a simple web store that can display products, add them to a shopping cart, and accept payment. After it goes live, you discover a bug in the product catalog than prevents products with hi-resolution images from displaying properly.

Now, only 2 products out of my 100 have hi-res images, and these aren't very popular products anyway.

You can do one of two things:

1. Ignore the bug for now as a low priority item, and focus on adding new features to the store to boost sales.

2. Work on the bug, make sure nothing else breaks, and redeploy the store.

If there are important new features that your customers are clamoring for (or that will prevent you from going out of business next month), you need to carfeully prioritize this work.

Unfortunately, it's not as if fixing this bug can be done in isolation. What if the fix introduces another, more serious problem? Is the risk of adding more serious bugs worth the benefit of fixing this one? Is the effort spent on this issue better applied to another more pressing feature?

Your approach then, has to take into account both risk, frequency, and severity. How many people are affected and how likely are they to encounter the problem, and how severe are the effects of encountering it? If you decide to fix it, how likely are you to introduce another problem inadvertently?

In the scenario above, very few customers will see the problem, as it only affect 2 unpopular products. Perhaps the fix will be in a delicate area of the code, and may cause the rest of the products to be incorrectly displayed. That would constitute a high-risk fix, with low frequency, and a low severity. Probably not worth doing right now.

On the other hand, a bug in the pricing engine that gives customers a 70% discount on all purchases would be much more severe, and could demand immediate attention. Pretty clear value proposition for that one.

Now, I'm not saying that as software professionals, we shouldn't strive to write clean, well-tested, error-free code. In fact, there are many approaches that are becoming popular, such as Test-Driven Development, Continuous Integration, and even Pair-Progamming that can help developers keep their code performing well.

But the reality is that as long as there is software, there will be defects. As part of a responsible product development effort, those defects should be prioritized along with the features, and the decision to work on them should be well-reasoned.

I've worked with many developers (and testers) that think this is just wrong, and that it's unprofessional to leave any known bugs in your software. But like anything else in life, it's all about tradeoffs. Obviously if all of the known bugs are trivial, low-risk changes, it's reasonable to fix them. But obsessive bug-fixing can actually cause more problems than it solves.

So fix your bugs, but make a conscious decision about them. If the benefits aren't there, you may be better served leaving a few until later. You might be avoiding even worse ones by doing so.

For more on agile tools and techniques:
(Tags: , ,

Get your copy of the new book! Agile Thinking: Leading Successful Software Project and Teams