Requirements Development Techniques and Agile Projects

By Ed Willis
December 9, 2009

This article is about developing requirements for agile projects. I've used a number of different techniques and have recently happened upon one that's been the best of the bunch for me - hopefully it will prove useful to you as well. Along the way, I'll sketch out a couple of other techniques for comparison.

I'd like to thank Jon Tarry both for being my collaborator on the technique described below and for reviewing this article.

Likely the most common method for developing requirements I've seen is what I'd call the "Illuminati" approach. In it, a senior person writes a requirements document and pulls in other senior people as needed. This can be fast, providing all those senior people are actually able to speak to the complete scope of the project and find the time to do so. In practice though, my experience has been that this approach is normally not at all fast - partly because neither of those provisos tends to pan out. In addition, because so few people have been involved in the requirements development process, the eventual review and acceptance of the requirements will typically take quite a while. Similarly, this approach does a pretty poor job of informing everyone who needs to understand the requirements because most of the stakeholders will only get the document - they won't have developed as deep an understanding as those who actually participated in the requirements development.

At this stage in my career, I'd be hard-pressed to use this approach.

The first game-changing process for requirements development I came across was Joint Application Development (JAD). Now JAD is more of a set of principles perhaps, or maybe a set of methods than it is a rigidly defined procedure - either way, I should take some time to outline what we meant when we spoke of JAD for requirements development because people in my experience are rarely talking about precisely the same thing when they refer to JAD. In particular, the technique I'll describe does not develop screen mockups or any other design artifacts but is rather solely focused on developing requirements.

Our JAD process starts off with a chartering session. Senior management and project leaders come together to develop, in real-time, a charter for the JAD. Likely the most important things that get decided at that meeting are:

  • They agree, in rough measure, what the main features the product will include are
  • Perhaps more importantly, they agree what major features will be out of scope
  • They assign a JAD facilitator to run the sessions - ideally also a scribe
  • They define a core team including product management and senior development people who will be in every JAD session - this core group should be 3-5 people and will provide leadership and continuity to the requirements development effort
  • For each major theme in the requirements' scope, they figure out what other Subject Matter Experts (SMEs) should be involved and schedule JAD sessions for each theme - taking care not to have more than 7 +/- 2 people in any one session
  • They agree on the template to use for each requirement - typically, these are fairly comprehensive and include things like description, priority, pre- and post-conditions, performance requirements and other non-functional requirements, affected user roles, scenarios, constraints, and so on
  • Not necessarily in the charter session but in advance of the first JAD sessions, they develop a context diagram - a context diagram shows the product in its context of use with all major interactions depicted; it is used to assess the "done-ness" of each theme-specific JAD session
  • Similarly before any JAD sessions are held but not necessarily in the charter session itself, they define user profiles that are relevant to the requirements themes

From there, you get started with the individual, theme-specific JAD sessions. Each of those works more or less like so:

  • In advance, the facilitator works with the core team to sketch out an agenda of topics relevant to that theme with a roughly 15 minute granularity
  • At the JAD session, the facilitator, scribe, core team and invited SMEs meet and collaboratively develop each requirement proposed by the team in real-time
  • The facilitator uses the agenda to move the discussion along
  • As each topic is discussed, the facilitator helps the team decide what it wants to say and the scribe captures the content in the agreed-on template
  • All of this is done in real-time with the document projected for all to see as it is being developed
  • The invitees use the context diagram to assess whether or not they've said everything they need to about each theme by asking "have we addressed everything we need to about this theme with respect to each interaction?"

Once all scheduled JAD sessions have been completed, the core team polishes the document and sets priorities in light of the complete set of requirements. Lastly, a review of the requirements is done - this is normally pretty painless because most people have already contributed.

The strengths of this process are pretty broad. Overall, we were pretty happy with it. The buy-in on the requirements is much, much higher given the high level of involvement from the wider set of stakeholders. Similarly many more people being exposed to the requirements development process means that, across the team, there is a much broader and deeper understanding of the requirements. The collaborative nature of the requirements development work builds in a free-form, continuous review of the requirements similar to what pair programming does for code - ultimately leading to better requirements earlier in the process. Lastly, the proactive planning and involvement of all relevant SMEs helps the team build both more correct and more complete requirements.

For waterfall/plan-driven/"Big Design Up Front" projects, this is the method I'd prefer for requirements development. That said, it's been a long time since I was involved in a project like that :)

The main weakness of the process is its performance - the JAD sessions take a while. Some of this is just perception. It feels longer because so many people are involved. But the truth is, it can take a while. In my experience, with a reasonably comprehensive set of requirements attributes to consider for each requirement, we were doing pretty well to develop four requirements an hour. Much of that is due to the serial handling of each requirement - the group proposes it, the scribe records it, the team reviews it and then we move on to the next requirement. Sometimes the tension in the room can be almost palpable as people grow impatient waiting to talk about specific topics that are near and dear to their hearts. At four requirements per hour, if a project will ultimately have 100 requirements, you're looking at 25 hours of meeting time times the average number of attendees per session in terms of total effort in the JADs. Given that you're normally looking at your best and brightest involved in these sessions, people can and do complain about the dent the sessions put in their schedules. Also it's typically tough to do more than one two-hour session a day - that mainly due to people tending to lose focus in long meetings but also because two hours is about the limit people can accommodate given all their other responsibilities. Now don't get me wrong, this is all very much time well-spent. Steve McConnell notes in "Software Estimation: Demystifying the Black Art" that the two factors most correlated with project schedule or effort are requirements analyst capability and product complexity, so there's little doubt that the investment in requirements called for in the JAD process will be repaid in spades. But even with that noted, if there was a way to make the process a little less expensive, I think it would go a long way towards making it more appealing to the peopled involved.

I and a bunch of other like-minded people participate in a software development study group. We get a book and read a bit of it at a time, getting together in between to discuss what we've read. It was at that forum that I first came across "User Stories Applied" by Mike Cohn. An awesome read, this - we did a Net Promoter Score on it when we finished it and it scored 54%, which is excellent. Reading it put me in mind of something I heard McConnell say once about "Extreme Programming Explained" by Kent Beck - that he loved how confidently prescriptive it was. Cohn's book is like this also - he's sharing stuff that he knows works. There are lots of little changes you might want to make here and there to suit your specific needs, but at the end of the day, he's pretty sure that you're not going to go too far wrong if you just do exactly what he's saying. That assurance jumps right off the page. This is in pretty stark contrast to "Requirements by Collaboration", where the steps in the process are up for debate, as are the specific of how each step will work. I have no doubt that Ellen Gottesdiener would run an awe-inspiring requirements workshop - her knowledge and experience are certainly beyond reproach - but frankly after reading "Requirements by Collaboration", I felt less prepared to actually help teams develop requirements than I did before reading it because the world of requirements development techniques seemed much more murky and confusing than it had before. That may suggest that her book is more for experts in this area, which I don't pretend to be. In any event, "User Stories Applied" had me ready to roll up my sleeves and get busy when I'd finished reading.

Now I won't go into detail on what a user story is in this article - there are other resources to consult to learn that but for the purposes of this article, the main thing I should point out is that a user story is intended to be much less detailed and precise than the requirements templates discussed above. They are of form "as a ROLE I can do FEATURE so that RATIONALE". The details of the story are deferred until the user story is actually about to be developed. Even then, the details are intended to be developed in face to face discussions and only really captured in tests. So the detail that the JAD process outlined above isn't developed straight away but rather is plumbed only when the team is about to use that information.

The approach to developing the stories Cohn presents works as follows - note I'm not giving it to you in gory detail here but rather calling out the salient points that will motivate comparison.

There's a session to develop the user roles (the expected types of users who will interact with the system) involving the facilitator, customer(s) and the development team.

  • At it, everyone works in parallel, writing roles on cards, saying their names aloud and taping them to a wall
  • When people have run out of ideas for roles, the participants merge them by putting potentially overlapped roles physically on top of one another
  • The facilitator helps the team determine, for each set of overlapped cards, whether or not they really need to be merged into one role or not and what the resulting roles should be
  • Finally, descriptions for each role are developed involving any interesting or useful attributes desired by the team (for example, level of expertise of the role, frequency that they would use the software and so on)

With respect to developing the user stories, Cohn presents a number of techniques: interviews, questionnaires, observation and story-writing workshops - I'll focus on that last one in this article (it's also, in Cohn's estimation, "the most effective way to quickly trawl for stories"). His user story workshop involves these steps:

  • A low-fidelity prototype is developed that maps high-level interactions with the envisioned software
  • The prototype is refined on the fly during the workshop as the participants determine what they think users will want to do with the software
  • For a "green-field" application development project, you'd start with an empty box labeled as the main screen of the application and consider one of your roles and ask "what should this person be able to do from the main screen of the application"
  • The resulting discussion is captured as both the user stories and a mapping of them to the emerging application architecture
  • As the discussion continues a broader picture of the stories emerges and the group is encouraged to split the application into new views to allow the stories to be organized sensibly
  • Additional stories are identified using leading questions like "what would the user want to do next", "what additional information could the user need at this point", and so on

One key thing to note is that we're back to serial handling of the stories as was the case in the JAD sessions above. Actually when I read this book the first time, I misremembered what I'd read - I thought the same free-form, parallel process used for the user roles development was also used in the story development. It was only on a second reading in the book club that I realized what he was actually saying. I can't speak for Mr. Cohn but the main motivation I could see for not using the same approach for developing the stories is that you'd expect many, many stories to be developed but not so many user roles. I expected the culling and consolidation exercise would be pretty daunting if you had large numbers of stories. That put many of us in the book club in the mind of trying the same, very parallel approach on stories all related to the same, fairly distinct, theme of requirements. So that's what we did - almost a hybrid of the last two approaches discussed in this article but definitely rooted in Cohn's technique:

  • Define themes of requirements - in user story terminology, these would be epics or themes - in the same chartering exercise detailed above in the JAD discussion
  • Take care to invite the right people to each theme - product owner, SMEs and development team members
  • Perhaps refine the agenda to identify sub-themes to further focus the discussion
  • Turn the participants loose on a theme or sub-theme - tell them to write stories on sticky notes as fast as they can, reading them aloud as they go
  • When they've run out of ideas, they review the set of stories and put stories on top of one another if they believe they are candidates for consolidation
  • The facilitator then helps the team decide how to disposition each pile of stories - leave them separate or consolidate subsets of them into new stories

The first time we tried this, we spent 90 minutes developing and consolidating stories and finished with 20 stories - much, much faster than the JADs we'd done previously. The second time we tried this, we developed 12 stories in 45 minutes. Again, much faster than we'd seen previously - not surprising given the parallel nature of the story authoring and the reduced detail expectations inherent in stories, but when all is said and done, if you're developing requirements for an agile project, this is a great way to do so very rapidly.

In short, we were happy with the performance of this process, and the people involved enjoyed it also. I think this would be my opener in any future requirements development work on agile projects. I hope it proves helpful to you in your work also.

You might also be interested in:

News Topics

Recommended for You

Got a Question?