Tag Archives: customer

Intensifying Stories Job Aid

"Intensifying stories" is an attempt to identify what can make a story (feature, capability) more potent.

I've identified key features in a number of systems (perhaps 50), identified a rule or concept beneath those features, then generalized that to apply to a number of other systems.

Intensifying Stories Job Aid (PDF)

This is no master list (nor do I expect any to exist), but I hope this list is helpful for people working to create highly-valuable capabilities in systems.

Intensifying Stories: Running with the Winners

For a given story headline, many interpretations are possible. These vary in their quality and sophistication. I call this the intensity or depth of a story. Story intensity is a knob you can control, dialing it down for earlier delivery, dialing it up for a more complex product.

For example, consider the story "System Cross-Sells Item." The simplest version could suggest the same item to everybody. The next version could have the system suggest a bestseller in the same category as a shopping-cart item. The most sophisticated version might consider past purchases from this and millions of other users, take into account the customer's wish-list, and favor high-margin items.

Lowering Intensity

The most common way to move the intensity dial is to lower intensity. This is the domain of various story-splitting strategies such as 0-1-Many or Happy Path First. These techniques are usually used to pull out the most valuable parts of a story, or to help ensure it can be implemented in a short time-frame (e.g., an iteration).

Story splitting is an important tool, but in the rest of this article we'll focus on going the other way.

Why Intensify?

The ability to intensify is valuable because we are uncertain about what is most valuable. We have hypotheses, but reality can often surprise us.

Think of it as placing bets. Suppose you have six alternatives. You can bet everything on one choice, or you can place small bets on each alternative with followup bets when you see which one is winning. Put this way, it seems clear we're better off starting with small bets, unless we're already sure about which will win.

The Lean Startup community has a related idea. They note that startups often pivot from one business model to another as they try to create a business. One pivot is called a zoom-in pivot: take a subset of the original product, and build a business on just that. An example is Flickr: it started as a game with a sharing component, but was zoomed in to focus on just the photo-sharing aspects.

Intensifying: Three Approaches

There are three common ways to intensify stories:

  1. Improve quality attributes
  2. Apply story-splitting techniques backwards
  3. Invent intensifiers that add in more options or capabilities

Improve Quality Attributes

The most straightforward way to intensify a story is to improve some quality attribute. Quality attributes, also known as non-functional requirements or "ilities," have a scale that tells how good something must be. Intensifying is just a matter of targeting a better number.

For example, suppose our system must support 125 transactions/second. We could intensify this by raising that to 500 transactions/second.

This form of intensifying is the least creative: take an attribute and ask for one 5x as good.

Be careful: some increases can be extremely expensive. A system that is 99% available can have 3.5 days of downtime per year; one with 99.9999% availability can have only ~30 seconds of downtime per year. A desktop machine might support the first one, but the second one might require a sophisticated and expensive multi-site setup.

Apply Story-Splitting in Reverse

For any story we've already split down, we can of course intensify it by combining everything back together.

But we can also use story splitting techniques as a way to generate new possibilities. For example, the 0-1-Many split says that when we have many of something, we can reduce it to 0 or 1 ways. To run this split backwards, we find something where we have only one of something, and generalize it to many.

For example, we may be able to save a document in a specific file format. We can look at this as a "one"; and look for a "many." In this case, that might mean supporting a variety of output formats.

Invent Intensifiers

The third approach is to find creative ways to improve a feature or set of features. There's no fixed set of ways to do this, though we can identify some common approaches.

Let's take a story "User finds matching item" to describe a searching process, and look at several ways to make this feature more sophisticated.

Add Control: We can provide options that let the user do a more precise or broader search:

  • Boolean operators
  • Proximity operators ("Agile within 3 words of Testing")
  • "Find more like this"

Add Intelligence: We get the system to automatically do more for us:

  • Automatically search for synonyms
  • Autofill with the most common searches
  • Automatically classify results

Manage History: Searching is rarely a one-time event; we can put the activity in more context by considering what has gone before.

  • Save searches for future use
  • Manage a list of previous searches or previous results

Manage Multiple Items: A user may have multiple searches active at once, and want to coordinate them.

  • Allow multiple search result windows
  • Compare results of two searches
  • Merge results from multiple searches

Conclusion

Intensity is a controllable characteristic of stories.

To lower intensity, standard story-splitting techniques apply.

To increase intensity, there are several approaches:

  • Improve quality attributes
  • Apply story-splitting in reverse
  • Invent intensifiers

By controlling the intensity of stories, we can better respond to the uncertainties of development and use.

Related Articles

Valuable Stories in the INVEST Model

Of all the attributes of the INVEST model, "Valuable" is the easiest one to, well, value. Who is against value?

We'll look at these key aspects:

  • What is value?
  • The importance of external impact
  • Value for whom?

What is Value?

Value depends on what we're trying to achieve. One old formula is IRACIS. (Gane & Sarson mentioned it in 1977 in Structured Systems Analysis, and didn't claim it was original with them.) IRACIS means:

  • Increase Revenue
  • Avoid Costs
  • Improve Service.

Increase Revenue: Add new features (or improve old ones) because somebody will pay more when they're present. 

Avoid Costs: Much software is written to help someone avoid spending money. For example, suppose you're writing software to support a call center: every second you save on a typical transaction means fewer total agents are needed, saving the company money. 

Improve Service: Some work is intended to improve existing capabilities. Consider Skype, the voice network: improving call quality is not a new feature, but it has value. (For example, more customers might stay with the service when call quality is higher.) 

IRACIS covers several types of value, but there are others:

Meet Regulations: The government may demand that we support certain capabilities (whether we want to or not). For example, credit card companies are required to support a "Do Not Call" list for customers who don't want new offers. If the company didn't provide the capability by a certain date, the government would shut down the company.

Build Reputation: Some things are done to increase our visibility in the marketplace. An example might be producing a free demo version of packaged software, to improve its marketing. In effect, these are an indirect way to increase revenue.

Create Options: Some things give us more flexibility in the future. For example, we may invest in database independence today, to give us the ability to quickly change databases in the future. The future is uncertain; options are insurance. 

Generate Information: Sometimes we need better information to help us make a good decision. For example, we might do an A-B test to tell us which color button sells more. XP-style spikes may fit this category as well.

Build Team: Sometimes a feature is chosen because it will help the team successfully bond, or learn important to the future.

Several of these values may apply at the same time. (There's nothing that makes this an exhaustive list, either.) Because multiple types of values are involved, making decisions is not easy: we have to trade across multiple dimensions. 

Valuing External Impact

Software is designed to accomplish something in the real world.

We'll lean on a classic analysis idea: describe the system's behavior as if the system is implemented with a perfect technology. Focus on the effects of the system in the world.

This helps clarify what are "real" stories: they start from outside the system and go in, or start inside and go outside. 

This also helps us avoid two problems:

  • "stories" that are about the solution we're using (the technology)
  • "stories" that are about the creators of the system, or what they want

If we frame stories so their impact is clear, product owners and users can understand what the stories bring, and make good choices about them. 

Value for Whom?

Who gets the benefit of the software we create? (One person can fill several of these roles, and this is not an exhaustive list.)

Users: The word "User" isn't the best, but we really are talking about the people who use the software. Sometimes the user may be indirect: with a call center, the agent is the direct user, and the customer talking to them is indirect. 

Purchasers: Purchasers are responsible for choosing and paying for the software. (Sometimes even these are separate roles.) Purchasers' needs often do not fully align with those of users. For example, the agents using call center software may not want to be monitored, but the purchaser of the system may require that capability.

Development Organizations: In some cases, the development organization has needs that are reflected in things like compliance to standards, use of default languages and architectures, and so on.

Sponsors: Sponsors are the people paying for the software being developed. They want some return on their investment. 

There can be other kinds of people who get value from software we develop. Part of the job of a development team is balancing the needs of various stakeholders.

Summary

We looked at what values is: IRACIS (Increase Revenue, Avoid Costs, Improve Service), as well as other things including Meeting Regulations, Generating Information, and Creating Options. 

We briefly explored the idea that good stories usually talk about what happens on the edge of the system: the effects of the software in the world.

Finally, we considered how various stakeholders benefit: users, purchasers, development organizations, and sponsors.

Value is important. It's surprisingly easy to get disconnected from it, so returning to the understanding of "What is value for this project?" is critical.

Related Material

Negotiable Stories in the INVEST Model

In the INVEST model for user stories, N is for Negotiable (and Negotiated). Negotiable hints at several important things about stories:

  • The importance of collaboration
  • Evolutionary design
  • Response to change

Collaboration

Why do firms exist? Why isn't everything done by individuals interacting in a marketplace? Nobel-prize winner Ronald Coase gave this answer: firms reduce the friction of working together. 

Working with individuals has costs: you have to find someone to work with, negotiate a contract, monitor performance carefully–and all these have a higher overhead compared to working with someone in the same firm. In effect, a company creates a zone where people can act in a higher-trust way (which often yields better results at a lower cost). 

The same dynamic, of course, plays out in software teams; teams that can act from trust and goodwill expect better results. Negotiable features take advantage of that trust: people can work together, share ideas, and jointly own the result. 

Evolutionary Design

High-level stories, written from the perspective of the actors that use the system, define capabilities of the system without over-constraining the implementation approach. This reflects a classic goal for requirements: specify what, not how. (Admittedly, the motto is better-loved than the practice.)

Consider an example: an online bookstore. (This is a company that sells stories and information printed onto pieces of paper, in a package known as a "book.") This system may have a requirement "Fulfillment sends book and receipt." At this level, we've specified our need but haven't committed to a  particular approach. Several implementations are possible:

  • A fulfillment clerk gets a note telling which items to send, picks them off the shelf, writes a receipt by hand, packages everything, and takes the accumulated packages to the delivery store every day.
  • The system generates a list of items to package, sorted by (warehouse) aisle and customer. A clerk takes this "pick list" and pushes a cart through the warehouse, picking up the items called for. A different clerk prints labels and receipts, packages the items, and leaves them where a shipper will pick them up. 
  • Items are pre-packaged and stored on smart shelves (related to the routing systems used for baggage at large airports). The shelves send the item to a labeler machine, which sends them to a sorter that crates them by zip code, for the shipper to pick up. 

Each of these approaches fulfills the requirement. (They vary in their non-functional characteristics, cost, etc.)

By keeping the story at a higher level, we leave room to negotiate: to work out a solution that takes everything into account as best we can. We can create a path that lets us evolve our solution, from basic to advanced form. 

Feedback

Waterfall development is sometimes described as "throw it over the wall": create a "perfect" description of a solution, feed it to one team for design, another for implementation, another for testing, and so on, with no real feedback between teams. But this approach assumes that you can not only correctly identify problems and solutions, but also communicate these in exactly the right way to trigger the right behavior in others. 

Some projects can work with this approach, or at least come close enough. But others are addressing "wicked problems" where any solution affects the perceived requirements in unpredictable ways. Our only hope in these situations is to intervene in some way, get feedback, and go from there.

Some teams can (or try to) create a big static backlog at the start of a project, then measure burndown until those items are all done. But this doesn't work well when feedback is needed.


Negotiable stories help even in ambiguous situations; we can work with high-level descriptions early, and build details as we go. By starting with stories at a high level, expanding details as necessary, and leaving room to adjust as we learn more, we can more easily evolve to a solution that balances all our needs.  

Related Material

Independent Stories in the INVEST Model

The INVEST model is a reminder of the important characteristics of user stories, and it starts with I for Independent.

Independent stories each describe different aspects of a system's capabilities. They are easier to work with because each one can be (mostly) understood, tracked, implemented, tested, etc. on its own. 

Agile software approaches are flexible, better able to pursue whatever is most valuable today, not constrained to follow a 6-month old guess about what would be most valuable today. Independent stories help make that true: rather than a "take it or leave it" lump, they let us focus on particular aspects of a system.

We would like a system's description to be consistent and complete. Independent stories help with that too: by avoiding overlap, they reduce places where descriptions contradict each other, and they make it easier to consider whether we've described everything we need.

Three common types of dependency: overlap, order, and containment

What makes stories dependent rather than independent? There are three common types of dependency: overlap (undesirable), order (mostly can be worked around), and containment (sometimes helpful). 

Overlap Dependency

Overlap is the most painful form of dependency. Imagine a set of underlying capabilities:
        {A, B, C, D, E, F}
with stories that cover various subsets:
        {A, B}
        {A, B, F}
        {B, C, D}
        {B, C, F}
        {B, E}
        {E, F}

Quick: what's the smallest set of stories that ensure that capabilities {A, B, C, D, E, F} are present? What about {A, B, C, E}? Can we get those and nothing else?

When stories overlap, it's hard to ensure that everything is covered at least once, and we risk confusion when things are covered more than once. 

Overlapping stories create confusion.

For example, consider an email system with the stories "User sends and receives messages" and "User sends and replies to messages." (Just seeing the word "and" in a story title can make you suspicious, but you really have to consider multiple stories to know if there's overlap.) Both stories mention sending a message. We can partition the stories differently to reduce overlap:

    User sends [new] message
    User receives message
    User replies to message

(Note that we're not concerned about "technical" overlap at this level: sending and replying to messages would presumably share a lot of technical tasks. How we design the system or schedule the work is not our primary concern when we're trying to understand the system's behavior.)

Order Dependency

A second common dependency is order dependency: "this story must be implemented before that one."

Order dependencies complicate a plan,
but we can usually eliminate them.

While there's no approach that guarantees it, order dependency tends to be something that is mostly harmless and can be worked around. There are several reasons for that:

  1. Some order dependencies flow from the nature of the problem. For example, a story "User re-sends a message" naturally follows "User sends message." Even if there is an order dependency we can't eliminate, it doesn't matter since the business will tend to schedule these stories in a way that reflects it. 
  2. Even when a dependency exists, there's only a 50/50 chance we'll want to schedule things in the "wrong" order.
  3. We can find clever ways to remove most of the order dependencies.

For example, a user might need an account before they can send email. That might make us think we need to implement the account management stories first (stories like "Admin creates account"). Instead, we could build in ("hard-code") the initial accounts. (You might look at this as "hard-coding" or you might think of it as "the skinniest possible version of account management"; either way, it's a lot less work.)

Why take that approach? Because we want to explore certain areas first. We consider both value and risk. On the value side, we may focus on the parts paying customers will value most (to attract them with an early delivery). On the risk side, we may find it important to address risks, thinking of them as negative value.  In our example, we may be concerned about poor usability as a risk. A few hard-coded accounts would be enough to let us explore the usability concerns. 

Containment Dependency

Containment dependency comes in when we organize stories hierarchically: "this story contains these others." Teams use different terms for this idea: you might hear talk of "themes, epics, and stories," "features and stories," "stories and sub-stories," etc. A hierarchy is an organizational tool; it can be used formally or informally. 

A good organization for describing a system is rarely the best organization for scheduling its implementation.

The biggest caveat about a hierarchical decomposition is that while it's a helpful strategy for organizing and understanding a large set of stories, it doesn't make a good scheduling strategy. It can encourage you to do a "depth-first" schedule: address this area, and when it's done, go the next area. But really, it's unlikely that the most valuable stories will all be in a single area. Rather, we benefit from first creating a minimal version of the whole system, then a fancier version (with the next most important feature), and so on. 

Bottom Line

Independent stories help both the business and technical sides of a project. From a business perspective, the project gets a simple model focused on the business goals, not over-constrained by technical dependencies. From a technical perspective, independent stories encourage a minimal implementation, and support design approaches that minimize and mange implementation dependencies. 

Related Material

"INVEST in Good Stories, and SMART Tasks" – the original article describing the INVEST model

Composing User Stories – eLearning from Industrial Logic

User Stories Applied, by Mike Cohn

Review – Agile Product Management with Scrum (Pichler)

Agile Product Management with Scrum Agile Product Management with Scrum: Creating Products that Customers Love, by Roman Pichler. Addison-Wesley, 2010.

This is a fairly easy read (about 120 pages) explaining the role of the Product Owner in Scrum. I'd describe the target as "someone preparing to fill the Scrum Product Owner role who already knows something about product management." There is a little material on product management techniques, but it's not the emphasis. 

This book is divided into six chapters, talking about the product owner role, envisioning the product, the product backlog, planning, the sprint meeting, and transitioning into the role. There's a good discussion of simplicity, and a little bit on handling this role on large projects.

I particularly liked that most chapters had a section on "Common Mistakes"; they gave me the sense of getting advice from someone who'd seen and worked through these things with real teams. 

Review – The Product Manager’s Handbook, by Linda Gorchels

The Product Manager’s Handbook, by Linda Gorchels. McGraw-Hill, 2005.

This is a basic book on product management (an area which has a shortage of material). It talks about the role of product managers, planning, maintaining a portfolio, new products, pricing, and marketing. There are some checklists and sample plans to help although I don’t see them as the main focus of the book. (For example, there’s a handy checklist on trade show responsibilities.) Most of the examples are about consumer-type products, but software makes a limited appearance. An OK intro, but I’m looking for better.

Review – Structured Systems Analysis: Tools and Techniques

Structured Systems Analysis: Tools and Techniques. Chris Gane and Trish Sarson. Prentice Hall, 1979.
This is one of the classic books on systems analysis: data flow diagrams, data dictionary, and so on appear. It does a decent job explaining these (though heavier on the tools than the techniques). The description of a data dictionary is one of the better ones I’ve seen. There’s a nice distinction between system and organizational objectives. This is the earliest reference I’ve seen to the IRACIS model: that work is done because it will Increase Revenue, Avoid Costs, and/or Improve Service. Their explanation of decision tables is excellent. For those who trace the history of agile ideas, Gane and Sarson view systems development as following Boehm’s spiral model: “In each case and at each level we build a skeleton, first logical and then physical, see how well the skeleton works, and then go back to put the flesh on the bones.” (This is from 1979, and 30 years later we’re still working on it.)


User Story Examples

This is a sample set of stories for a time management system.

I've been interested in personal productivity systems for a while, so when Mark Forster introduced a new system called AutoFocus, I decided to create an electronic implementation. The system is a sort of todo list, in a organized as a notebook where each page has a list of tasks to do. There are rules defining the currently active page and task, and how you move your focus as you complete tasks. 

I began by analyzing the system (e.g., building a flowchart and transition diagrams to look at how tasks and focus move). I wrote some basic cards, and a series of screen sketches. For example, a task page will look something like this:

|<   <<   *   >>   >|

Add new task: [_______________]

Active Tasks
[ ] task 1
[*] task 3

Completed Tasks
[x] task 2
[x] task 4

[Dismiss] [Force Focus] [Move Focus]

|<   <<   *   >>   >|

(The characters at the top and bottom of the page represent navigation arrows.)

Basics – Tasks

The first cluster of stories is centered around tasks and assumes all tasks are on the same page.

Show tasks

 

Add a task

 

Mark task done

Notice that with just the above stories, the system could be a basic todo list manager.

Mark task progress

Basics – Pages

The system is built around the metaphor of a notebook; we need navigation around the pages. The "current" page can mean two things: either the page you're currently looking at, or the one with a currently active task.

Multiple pages

 

Navigate – first, last, previous, and next

 

Navigate to focus page

 

Move the focus page "forward"  [can wrap]

By the "rules," you're normally supposed to move the focus page forward (wrapping to the beginning) as you complete tasks. But there's a "don't be stupid" clause that says you can work where you need to.

Force the focus page here

Friendly Feedback

I want to push the system to a  point where I can get some friendly feedback.

First, I need to pick up the last major action required by AutoFocus:

Dismiss open tasks on page

Then I need to put the system somewhere others can see it:

Deploy to sandbox

Users

I can't just have one notebook out there for the world to see; I need to introduce the notion of users. I actually started with a card that said "User Management" but I expanded it to these:

Register

 

Login

 

Logout

 

Forgot password

 

Update settings

Getting Real

What would it take for a beta test? I could probably defer "forgot password" and "update settings" but I have other work that would need to be done.

First, I'm asking users to trust the system and load in all their tasks. Paradoxically, I think if I make it less necessary to trust the system, people will trust it more. I'll do this by providing a means for them to export the tasks they've entered; then they know they can easily recover their work if they want to move on to a different system.

Backup/export tasks

Another aspect of trust is the look of the site. I have some OK default icons, but the site will shine better with a new set.

Polish stylesheet and images

It will have to be deployed to a public system:

Deploy

"Home page" is a bit of a placeholder card. The bare minimum is to provide a login page, but in the long run it will need some explanation of the system and an appeal to register.

Home page

I could start with these on the home page but probably want them somewhere else.

Terms and privacy statement

I haven't decided how to fund the site, and I don't need to know to start a beta test. But eventually I'll need a revenue model fancier than "pray Google offers millions." There might be other ways, but the next two stories are what sprang to mind first.

Ads

 

Charge fees

Here's another story that could be big. I'm starting with a web site, but people may want to use their mobile phone to manage their tasks. This could be as simple as a special stylesheet, or it could become a whole separate application.

Support mobile phones

 

Sparkling Touches

These next stories are ideas I have that fit in the basic framework. They could be early upgrades.

Delete a task

 

Edit a task

 

Revive a task

I don't know what all I mean by page status, but it includes the date and whether this is the focused page.

Page status

Some tasks should just automatically recur once you complete them (e.g., "email inbox to 0"). I could put a flag on the task, or maybe find a more sophisticated approach.

Auto-reschedule

Future Versions

I have a number of stories that aren't so well defined – the fuzzy ones.

People might want to pull out separate lists that aren't managed by the AutoFocus rules. I don't know if this needs to allow multiple user-created lists, or how that will work.

Lists

Once people get a lot of tasks, they may want to find a particular one:

Search

A tickler is a reminder system that lets you schedule tasks out in the future. ("43 folders" – one for each month and one for each day number in the month – is a classic paper form.) Some people just put tickler tasks in their calendar, but I think it would be handy to have a tool that automatically brings them in to the task list as appropriate. I want to make sure to include the notion of lead time – I might like to be reminded that today is my sister's birthday, but I really need 2 weeks notice so I can get her a card or gift. (And I want that reminder as a task, not a phone or email reminder.)

Tickler

Many tasks have a standard set of sub-tasks. I'd like to put them on a checklist and have them brought in on demand. The easy form is a "parallel" list – put all tasks on the list.

Checklist – parallel

The more classical form is to do the steps in order. When one step is completed, the next one should be scheduled.

Checklist – serial

People might like some sort of analysis, or it might be useful in supporting the system.

Reports/statistics

And I have a last one that's totally open. It's really more of a goal than a concrete story. It might be met by paid advertising, blogs, FAQs, help pages, email reminders, you name it.

Increase usage of the system

Analysis

I feel good about this set of stories. They capture the bulk of what I want the system to do.

I used a simple "headline" form for the story titles – just a short verb phrase. I know the domain, and it's generally obvious who the story is for and what it's for, so I didn't feel the need to put them in the "As a ____, I want to ____ so that ____" form.

The stories came in three batches – the early task and page stories (with a couple sparklers), user and deployment stories, and the others. I developed them in combination with repeated sketches and analysis.

Another way to look at stories is to look at them versus the INVEST acronym (from my earlier article "INVEST in Good Stories"):

  • I – Independent. The stories mostly stand alone from each other. The dependencies tend to be domain-related rather than technical. (For example, it doesn't make much sense to have a logout story before you have a login.)
  • N – Negotiable. The smaller stories are straightforward and well-defined (though they have some room on the details). Bigger stories have a lot of flexibility. Plus, this is a one-person project so there's not much problem there.
  • V – Valuable. Each story adds a clear bit of functionality to the system. The most technical story ("deploy") still makes sense to a product owner.
  • E – Estimable. I have a good sense of which stories are well defined; even for the fuzzy stories I can make educated guesses about the order of magnitude.
  • S – Small. Stories ready to schedule (closer to the top of the list) are generally hours to a few days work. That's appropriate. Later stories are bigger; that's ok too. When it gets time to implement them, I'll break them down into small stories as well.
  • T – Testable. Ready-to-schedule stories are testable; the fuzzier stories will need more explanation and analysis before we could write a test.

If I look at the list of stories as a release plan, the order is generally good. (There are a few stories that could move down in priority.) But one thing stands out: deployment could be moved much further to the top. It could be the second story: a simple list of active tasks might be useful to somebody (perhaps as a status report or proof of concept). It's easier to deploy when the system is still small. And it's a key to getting feedback.

Resources

Using Actuals with Estimates

I work with a group that’s been estimating in pair hours for a while. We’ll describe a story, everybody will write their estimate on a card, somebody collects the estimates, and we make an overall estimate. (The tricky part is that an estimate needs to include the tasks that aren’t yet known but that will be discovered.)

We also track actuals. Whenever it’s convenient, but definitely at the end of each day, the pair adds its tick marks onto the board.

We’ve used a couple approaches to coming up with "the" estimate. One was to have the longest-standing team member propose a consensus number => not consistently right. Then we tried taking the maximum estimate => usually too high.

Now, we’re keeping track of who made what estimate. We compare estimates to actuals and decide who was the "winner" – the one with the most accurate estimates that week. When it’s time to do new estimates, we let that person hear all the estimates, and they make the overall estimate we record.

It’s too early to say how well this works. The same winner showed up twice in the first month, which may be an early indicator.

My estimates? I’m almost always too optimistic. So I’m going to pay more attention to how the winner thinks about our tasks.

Review – The Toyota Product Development System

The Toyota Product Development System, James M. Morgan and Jeffrey K. Liker, Productivity Press, 2006.
Toyota's lean manufacturing system has had a lot of press, but lean influences their product development approach as well. This book emphasizes the system aspects of their approach. Along the way it explores a variety of practices and tools: set-based approaches, use of a chief engineer, alignment, and more. I preferred this book over Kennedy as it feels both broader and deeper. (Reviewed Nov., '06)

Agile '06 Workshop: Example-Based Specifications

This workshop was designed to explore Example-Based Specifications. Hosted by Brian Marick and Bill Wake.

Schedule:

  • 1 hour – One person plays customer, the rest of the team writes tests
  • 30 minute debrief
  • 1 hour – Small group discussions
  • 30 minutes – sharing and conclusions

Test Writing

Group 1: Domain – a statistical tool that evaluates inputs according to a formula, and plots the results.

Their test:

(A, B, and Y are inputs; b0, b1, b2, and b12 are outputs, produced by a process of matrix algebra.)

A B Y b0 b1 b2 b12
10 75 8 11.0 5.0 8.0 12.0
20 75 12
10 100 10
20 100 15

This is a textual representation of the graph

b0 b1 b2 b12 B-1 B-2 B+1 B+2
0 5 8 12 (1,1,RGB) (10,10,RGB) (10,1,RGB) (1,10,RGB)

Their expected graph (described here, though the team drew a picture):

  • a large X with each line in a different colors
  • labels +1 and -1 in the graph
  • a horizontal axis labeled "A" (also having labels -1 to +1)
  • a vertical axis ("Y")
  • a graph name ("B").

Group 2: Domain – a session scheduler, matching audiences to rooms

Test: Schedule a session -

Preconditions: user has logged in, room size < audience size

  • TU35 has iaudience 100
  • Room NICA has capacity 25
  • TU35 is not scheduled
  • NICA is available Monday 11-12:30

Execution:

  • User assigns TU35 to NICA at Monday 11-12:30
  • System displays a warning, "Room size mismatch… Continue or cancel?"
  • User selects "Continue"

Expected Results:

  • TU35 shows up scheduled in NICA, Monday 11-12:30

Group 3: Domain – a graphical tool for tracking and identifying criminal conspiracies. (Tool helps build a network of connections between people).

Their test:

Select template criminal
Create node
Create node
Select link template ?
Create link
Check count links 1
Delete node 1
Check count links 0
Create node policeman
Set first name Tom
Set last name Nassau
Create criminal
Set last name Soze
Select link template arrest
Create link Nassau to Soze
Check link exists
Select template policeman
Create node
Add role
Select node 1
Check has role policeman
Check has role drug dealer


Group 4: Domain – an expense approval manager

A GUI mockup:

    Manager approval:
    Purpose:

Type Date Description Amount Needs Receipt
    American Airlines SEA to MSP $600.00  
       

X

         

Issues: editing, are you sure?, manager approval, expense group

Test:

Type Date Vendor v Description Amount
Air tix 7/16/06 AA ORD to MSP 763.42

    Purpose: Trip to agile conference
    Manager approval:    Pointy-haired boss

An analysis of states:

New – Open, can’t pay, can’t approve, can’t close
Submitted – Open, can’t pay, can approve, can’t close
 

  Is Open Can Pay Can Approve Can Close
New        
Submitted        
Approved        
Paid        
Closed        

 


Group 5: A stock trading program

A screen mockup with annotations:

Start Time [   ] v : [   ] v 9:30-16:00 ET (market start to market end)
End time [   ] v : [   ] v
Stock Symbol [    ] 1-4 alpha
# Shares [    ] 1
Order Size [    ] 100->1 million (int) (+- 100)
  [Buy/Sell]  
Price $ [    ] (optional) numeric + 2 decimal optional
     
  [OK]    [Cancel]  

Test of algorithm:

  • Start time >= now >= 9:30 EST
  • End time > start time <= 16:00 EST
  • Distribution list (file) exists
  • Test invalid symbol entry
  • Test for valid symbol entry
  • Test all shares are traded if no limit price
  • Test trades don’t violate limit price
  • Buys or sells based on selection

Group 6: Insurance tracking program

Test for story "Insurer adds provider":

Test 1 – Sunny day

Add table:

Johnson, David name 15ch, 15ch
1200 Nicolette Dr addr1 40ch
12345 addr2 5d or 5d-4d
123456789 taxid 9 ch num
ALLINA network id must exist in db

Read: (same)

Test 2- Invalid zip code (and more like this…)

Johnson, David name 15ch, 15ch
1200 Nicolette Dr addr1 40ch
1234 addr2 5d or 5d-4d
123456789 taxid 9 ch num
ALLINA network id must exist in db

Expected error: "Invalid zip could should be 5 digits"

Test for story "Analyst approves pending claims < 60 days since claim made"

Test 1 – Sunny day

Populate database – set up claim 1

Date of claim 30 days ago
Member name Doe, Jane
Provider name Johnson, David
Diagnosis 769
Charge $500.00
Network id ALLINA

Step 1. Analyst views pending claims < 60 days (display claim A => OK)

Step 2. Analyst approves pending…

    Signal "ok" = submit
    Claim disappears from list
    Message "approved"

Step 3. Read claim, status "approved"


Group 7: Domain: Shipping company.

Test: 1. Customer ABC call to know where shipment with order #33 is. The system should answer, "Last stop was Tampa, FL."

 

Note Order # from Customer Request Answer from System
Truck left origin #33 last stop? Tampa, FL
Not in truck #34 last stop? nothing
Truck at destination #35 last stop? Daytona, FL
Truck at destination #35 arrived? yes
Not in truck #34 arrived? no
Not arrived #34 expected date? 26/7
Arrived #35 expected date? 25/7
Truck underway #33 arrived? no

Example context:

# Pick up origin Final drop destination Expected Date
#33 Tampa, FL Toronto, ONT 27/7
time 24/7 8 AM  
#34 Vancouver, BC Toronto, ONT 26/7
time  
#35 Toronto, ONT Daytona, FL 25/7
time 24/7 17:00 time 25/7 10:00

Test 2: I want the system to help me minimize empty truck displacement. For example, I want to be able to ask if there is an empty truck in Ontario on July 27. Arrival within 2 days.

Empty truck in? Shipment order  
Ontario/27/07 #33, #34 truck at city
Ontario/25/07   no truck because wrong date
Montreal/QC   no truck because wrong location

Debrief

We noticed these things from the experience of writing tests:

  • What to do is vague.
  • Developers tend to embellish.
  • Tests teach developers, but it’s a challenge to pick the right level.
  • It’s hard to turn descriptions into tests.
  • Tests (and collaboration) can help you discover new things.
  • The idea of a GUI became actions on a model (for the test).
  • Customers should come in prepared.
  • We need many questioners per answerer.
  • Someone came in late, and found that the examples helped them understand.

Topics

In small groups, we explored these topics:

  • Sufficient coverage: How do you know you’re done – what’s sufficient coverage?
  • Test styles: What is the relationship between example-based specifications and other styles of tests?
  • Product Owner: Techniques for interviewing product owners

Sufficient Coverage

  • When adding tests to a legacy system, how much do you "backfill"?
  • Should we just use "change detector" tests (record what the system currently does, knowing that it may change later, and may not even be correct).
  • Do we need all combinations?
  • Where do we fit example-based tests?

Test Styles

We suspected these things about example-based tests:

  • They may bring about exploratory testing, regression tests, better unit tests, acceptance tests. Load tests?
  • They provide a concise account of an edge case.
  • They serve to train new developers in a domain.
  • They provoke a certain style of conversation.
  • They may overwhelm developers with distracting detail (no "metaphor").
  • Alternatively, developers may ignore examples! It happens…

Product Owner

  • Use Sophie’s Rule – keep asking "why?"
  • Ask the end goal, define the business problem, define acceptance criteria
  • Discuss requirements "as if you were blind" (without reference to UI)
  • Need a customer who is is readily accessible
  • Helps to talk to end users
  • Programmers need to understand the domain
  • Let product owners ramble

General Notes / What Next

  • Mind maps have utility in these conversations.
  • Examples are a style of conversation – the easiest kind to get.
  • "We’re going to practice" – "process miniature" exercises could help.
  • A whiteboard and pointing are powerful ways to focus attention.

Extreme Test Makeover – Agile ’06

Brian Marick and I hosted "Extreme Test Makeover," where people could bring their laptops with code and tests, and have an experienced tester/programmer review them. Observations by participants:

  • Watij tests in Fit are too long/confusing to read for customer.
    • You could write it in JUnit instead of Fit
    • Break them up into small focused tests
  • Neat new delegate syntax (with .Net 2.0)
  • Descriptive variable names are good [even] for short term variables.
  • Keep tests focused on one purpose – If a test needs 3 things to work, create 3 tests
  • Generic isn't always useful.

Thanks to our participants and our experts: Bob Martin, Brian Button, Janet Gregory, JB Rainsberger, Jim Newkirk, Jim Shore, Lisa Crispin, Micah Martin, Randy Coulman, and Ward Cunningham.