Agile ’06 Workshop: Example-Based Specifications

This workshop was designed to explore Example-Based Specifications. Hosted by Brian Marick and Bill Wake.

Schedule:

• 1 hour – One person plays customer, the rest of the team writes tests
• 30 minute debrief
• 1 hour – Small group discussions
• 30 minutes – sharing and conclusions

Test Writing

Group 1: Domain – a statistical tool that evaluates inputs according to a formula, and plots the results.

Their test:

(A, B, and Y are inputs; b0, b1, b2, and b12 are outputs, produced by a process of matrix algebra.)

This is a textual representation of the graph

Their expected graph (described here, though the team drew a picture):

• a large X with each line in a different colors
• labels +1 and -1 in the graph
• a horizontal axis labeled “A” (also having labels -1 to +1)
• a vertical axis (“Y”)
• a graph name (“B”).

---

Group 2: Domain – a session scheduler, matching audiences to rooms

Test: Schedule a session –

Preconditions: user has logged in, room size < audience size

• TU35 has iaudience 100
• Room NICA has capacity 25
• TU35 is not scheduled
• NICA is available Monday 11-12:30

Execution:

• User assigns TU35 to NICA at Monday 11-12:30
• System displays a warning, “Room size mismatch… Continue or cancel?”
• User selects “Continue”

Expected Results:

• TU35 shows up scheduled in NICA, Monday 11-12:30

---

Group 3: Domain – a graphical tool for tracking and identifying criminal conspiracies. (Tool helps build a network of connections between people).

Their test:

---

Group 4: Domain – an expense approval manager

A GUI mockup:

    Manager approval:
    Purpose:

Issues: editing, are you sure?, manager approval, expense group

Test:

    Purpose: Trip to agile conference
    Manager approval:    Pointy-haired boss

An analysis of states:

New – Open, can’t pay, can’t approve, can’t close
Submitted – Open, can’t pay, can approve, can’t close
 

 

---

Group 5: A stock trading program

A screen mockup with annotations:

Test of algorithm:

• Start time >= now >= 9:30 EST
• End time > start time <= 16:00 EST
• Distribution list (file) exists
• Test invalid symbol entry
• Test for valid symbol entry
• Test all shares are traded if no limit price
• Test trades don’t violate limit price
• Buys or sells based on selection

---

Group 6: Insurance tracking program

Test for story “Insurer adds provider”:

Test 1 – Sunny day

Add table:

Read: (same)

Test 2- Invalid zip code (and more like this…)

Expected error: “Invalid zip could should be 5 digits”

Test for story “Analyst approves pending claims < 60 days since claim made”

Test 1 – Sunny day

Populate database – set up claim 1

Step 1. Analyst views pending claims < 60 days (display claim A => OK)

Step 2. Analyst approves pending…

    Signal “ok” = submit
    Claim disappears from list
    Message “approved”

Step 3. Read claim, status “approved”

---

Group 7: Domain: Shipping company.

Test: 1. Customer ABC call to know where shipment with order #33 is. The system should answer, “Last stop was Tampa, FL.”

 

Example context:

Test 2: I want the system to help me minimize empty truck displacement. For example, I want to be able to ask if there is an empty truck in Ontario on July 27. Arrival within 2 days.

Debrief

We noticed these things from the experience of writing tests:

• What to do is vague.
• Developers tend to embellish.
• Tests teach developers, but it’s a challenge to pick the right level.
• It’s hard to turn descriptions into tests.
• Tests (and collaboration) can help you discover new things.
• The idea of a GUI became actions on a model (for the test).
• Customers should come in prepared.
• We need many questioners per answerer.
• Someone came in late, and found that the examples helped them understand.

Topics

In small groups, we explored these topics:

• Sufficient coverage: How do you know you’re done – what’s sufficient coverage?
• Test styles: What is the relationship between example-based specifications and other styles of tests?
• Product Owner: Techniques for interviewing product owners

Sufficient Coverage

• When adding tests to a legacy system, how much do you “backfill”?
• Should we just use “change detector” tests (record what the system currently does, knowing that it may change later, and may not even be correct).
• Do we need all combinations?
• Where do we fit example-based tests?

Test Styles

We suspected these things about example-based tests:

• They may bring about exploratory testing, regression tests, better unit tests, acceptance tests. Load tests?
• They provide a concise account of an edge case.
• They serve to train new developers in a domain.
• They provoke a certain style of conversation.
• They may overwhelm developers with distracting detail (no “metaphor”).
• Alternatively, developers may ignore examples! It happens…

Product Owner

• Use Sophie’s Rule – keep asking “why?”
• Ask the end goal, define the business problem, define acceptance criteria
• Discuss requirements “as if you were blind” (without reference to UI)
• Need a customer who is is readily accessible
• Helps to talk to end users
• Programmers need to understand the domain
• Let product owners ramble

General Notes / What Next

• Mind maps have utility in these conversations.
• Examples are a style of conversation – the easiest kind to get.
• “We’re going to practice” – “process miniature” exercises could help.
• A whiteboard and pointing are powerful ways to focus attention.