Agile ’05 Conference Report

Posted on by

Part 1

The Agile ’05 conference was July 24-29, 2005, in Denver, Colorado, USA. There were about ten or twelve tracks at all times, so this report is necessarily only a limited bit. Usually I teach, but this time I was an organizer so I got to be more like an attendee in some ways.

Brian Marick and Bob Martin Keynote: Where were we, where are we, where are we going?

Brian Marick: We have different disciplines. We become multi-specialists. “A team of multi-specialists can knock the socks off a team of specialists.” But there’s no career path for multi-specialists – what conference would you go to?

Brian invited Jim Highsmith up to announce the formation of the Agile Project Leadership Network (APLN). Its focus is on applying agile principles to all projects, not just software. Relative to the Agile Alliance, this organization has consistent principles and intends to collaborate; it just has a different focus.

Bob Martin: “Value the flow of value.”

The software industry: was either no process or waterfall. Now we’re recognizing the failure of waterfall and the need to measure in-process work. It’s time to do “software at home.” Where we’re going is to resolve this challenge, with short cycles, feedback, testing, craftsmanship.

The agile community: The Scrum paper in ’95, XP in ’99: resonated with the community. But we were fractured into many agile methods; branding. Now, industry is learning to pull and mix practices; we’re at the moment of convergence. Future: head away from brands, strong focus on project management. At the end, we’ll have a definition of agile.

Users: Were small teams with a programming focus; limited acceptance tests, project managers “lost.” We are a community that has grown. We see 80-people teams often mixing Scrum and XP. 300-person teams exist. A company of thousands doing transitions. Future: we’ll grow. Agile project management will happen. Automated acceptance tests will be regarded as the measure of progress.

Brian Marick: There was a group of English cyberneticians. They made discoveries of performance, adaptation, surprise. But oops… no (single) institutional home, no acknowledged base body of techniques, no grooming of successors. Their tradition is gone. To avoid this, “we gotta take over a university department.” He announced the “Gordon Pask award”, to honor people whose recent contributions demonstrate their potential to be leaders of the field.

Open Space

There were a lot of topics; see the web site. One was “XP Rituals”, where we identified a number of fun things:

  • Breath mints
  • Haiku written on card before starting a task
  • Hacky sack at standup
  • Traffic light for builds
  • Darts when a pair finishes a task
  • “Story starting” gong
  • Story completion frog (noise-maker) or bell
  • Daily progress chart – two thermometers, showing days used and points locked in
  • Automatic break program
  • Smiley stickers for story cards

Metaphors, by Dave West

In 2001, metaphor was listed as an official XP practice (part of architecture); in 2005, it was not. Kent has said its redundant as a practice since you can’t help using it. (Dave argues it’s a learned skill.)

Metaphor has a lifecycle from poetry through use. Lakoff and Johnson say all thought is based on metaphor. Dave argues:

  • If you don’t choose consciously, you get one unconsciously.
  • Metaphor selection has a real effect on software design (architecture).
  • Metaphor is a skill you can develop.
  • Therefore, it should be an agile practice.

We have many default metaphors: machine, org chart, entity, dualism, computer science, software engineering, product manufacturing. There are alternatives: e.g., object as person.

To use metaphors well, learn: reflect, experiment, read widely, liberal arts, point of view.

Tim Lister: XP Denver Invited Talk

  • Managers are lonely – they don’t have a collegial environment.
  • QA is a bottleneck, and no wonder: it’s done late and in serial rather than in parallel.
  • Getting agreement on requirements is not neat. “Gathering requirements” sounds like they’re Easter eggs in the yard. But really, most requirements have to be invented.
  • Problem: people believe the customer.
  • It’s messy, so model and prototype. Don’t prototype solutions – prototype wrong things on purpose (so they can laugh at it).
  • Estimating is overwhelmed by expectations: skew (never early). Separate goals from estimates.
  • Software in general is uninteresting. “One right way” is anti-intellectual.
  • “Never lose the satisfying feeling of making something valuable, together with your colleagues.”

Part 2

Delivering APIs in an Agile Context, John Major

John Major described a project that was doing custom programming of lab workflows as part of the Human Genome Project. This was an environment with lots of churn: biology, instruments, J2EE.

Tensions:

  • Stable APIs vs. agile change
  • General features vs. “Do the simplest thing that could possibly work”
  • Frameworks vs. global refactoring
  • Framework design vs. features

The result was a platform API, a mix of data-driven (configuration) specification and custom code. The team had the attitude toward process, “We’ll try anything for a month.” There was lots of testing. They used Jemmy with Swing.

Unique practices:

  • Internal support: old and new features. Used the idea of “office hours” for support. Provided training. Tried having platform developers work on applications, but it didn’t work so well.
  • Lightweight code reviews. Rather than pairing, they required at least a 20-minute weekly code review with an office-mate.
  • Agile database schemas. Problem: RDBMS is rigid, but schemas evolve. Solution: build agile schema features and tools to manage change.

Lessons learned: Good:

  • Agile practices help keep technical debt low.
  • Build tools to support RDBMS and large codebase.
  • Pull in senior people to help design and adoption.

Bad:

  • Cost of absorbing API changes is paid by the application teams, but benefits accrue to the business.
  • It’s hard to get feature design right (to balance flexibility and focus).
  • The business had trouble understanding the costs of release management. (Branches made the whole thing even crazier; he described a “London subway” map they created to show all the paths.)

Ugly:

  • People issues – don’t go into denial.
  • Weren’t able to tell QA what the test suites did, so there was overlap between the automated tests and manual testing.
  • Be humble – the platform needs the app and vice versa.

Summary:

  • Build reusable platform technology
  • Use agile practices to cope with change
  • Work with “eyes wide open”

Part 3

Rachel Davies’ and Mike Hill Workshop on Informative Workspaces

Informative workspaces:

  • Team memory
  • Visible status (keep it fresh)
  • Automation – light and sound
  • Hawthorne effect
  • Track “puzzles”
  • Root cause analysis
  • Positive focus

Themes: Ownership: Own the space; collective ownership of communication, accountability.

Transparency: Be honest with yourself – show where you really are; peer pressure; reveal hidden/unknown problems.

Keep it Fresh: Drop stale charts. Let anybody update. Automation (e.g., build server status, test status). May use projects or status monitor.

Intra-/Inter-Team Communication: Visual progress => motivation. Whose team sees it.

Hokey-ness: Lack of ownership. Formality. Ownership – lead team to the charts. Time limit for new things. Cool is fun, but must be cool to team.

Kent Beck Open Space on Renewing the Fire

“The edge of knowing and not knowing” – Troy Frever.

What helps keep the fire? A lot of disucssion on being in the zone, in flow – but that’s only part of it. Many people crave novelty, mentoring, flow, living on the edge.

There’s a “game face” you put on.

Pollyanna Pixton Open Space on Organizational Change

There’s a “practice” level, for developers, managers, product owners. But there’s also an organizational change level.

“Continuous retrospective” – collect cards all week.

Leadership “versus” self organization.

Kent Beck Open Space on XP for Beginning Teams

He uses an Appreciative Inquiry approach. Take a practice – what does it mean for you? Acts as a “practice inkblot”.

An AI formulation:

  1. Remember a good (peak) situation
  2. Explore the circumstances that made it possible
  3. What is the meaning now?

We broke into pairs and did some mind-mapping of a practice.

Research Paper: A Case Study on the Impact of Scrum on Overtime and Customer Satisfaction – Mann and Maurer

Described a team that originally had variable sprints, then shifted its sprint size from 20 to 11 days at the customers’ request. Research tried to address, “Does Scrum provide a sustainable pace?” They found there was less overtime (both mean and variance) after Scrum was introduced in this organization. Customers were more satisfied. (Planning closer to delivery led to less misdirected development; playing with the product in the release cycle let them tweak its direction.) Scrum gave them better control and visibility.

Research Paper: An Environment for Collaborative Iteration Planning (Liu, Erdogmus, Maurer)

They used a traditional-looking table with a projected image, and wireless tablet PCs.The thought was that people would create and edit stories on a tablet, then organize and prioritize stories using finger and pens. This would give real-time information, as well as persistence.

The display was nice. One neat trick was that you could “push” a card toward somebody and it would glide at a realistic-looking rate (but never fall onto the floor:)

Existing tools are either cards, which are easy to work with but lack persistence, or planning software, which creates an unbalanced environment (somebody controls the keyboard) but which does have persistence.

The research effort is just beginning; they want to evaluate, “Is it useful? Is it usable? How does it work compared to existing tools?”

Part 4

Jeff Sutherland on Advanced Scrum

“Better, faster, cooler.” If this is interesting, see Jeff’s paper. I made very quick notes but it’s an interesting extension of the Scrum work and I plan to give it more study.

Scrum is out of development and into the whole company. This approach is for experienced ScrumMasters/developers only. See Lawrence Leach: “Eight Secrets to Supercharge Project Performance”, Adv. Project, Inc.

Productivity is measured as “features/$100K”

How can we (agile?) win?

One study shows outsourcing cuts costs by only 20% – significant, but not quite the same as cutting 80% or more of costs as some have promised.

The new approach: anticipatory, requires accurate analysis of process and automatic monitoring.

Type A, B, and C Scrum – A: isolated cycles (breaks between sprints), B: overlapping iterations (no handoffs), C: all at once. Consider the sprint level. Need to do pre-staging work for the next iteration inside this one (or else we’ll be tempted to fall back to type A).

Advanced scrum has multiple simultaneous concurrent sprints. All sprints release live software.

Changes:

  • MetaScrum for release planning
  • Variable-length sprints
  • Overlapping sprints for one team
  • Pre-stage product backlog
  • Daily scrum of scrums
  • Integrate product backlog and sprint backlog
  • Paperless project management and realtime reporting
  • Administrative overhead of 60 seconds/day/developer and 10 minutes/day/ScrumMaster.

One of the big challenges is having items in the backlog be ready. In their case, that means “intuitive for a doctor” – usable within one hour. This means stories need enough detail, and must have been prototyped with doctors. It forces the product owners to work with the developers to prototype things.

MetaScrum: weekly meeting of stakeholders. Led by the Product Owner. Use the three questions. All product decisions are made here. Release status. All decisions are communicated that day. Damage control plans are executed in the same day.

Iterations: three-month – major product releases (eliminate bugs from portfolio); 1-month sprint for new customer/upgrades (eliminate high-priority bugs); one-week sprint for critical issues. This gives them 45 production releases a year. “Assumes good practices. They’re using one code base, working at the tip.

Pre-staging: Overlap sprints, with no break between them. Work only on product backlog that is ready to go the sprint. Pre-staging can double throughput in sprints.

Prioritizing sprints. Constraint theory tells us we must have slack. This lets you optimize multiple overlapping sprints. The backlog is fully automated, with priority levels for the different length sprints (week, month, quarter). Developers focus on one task at a time in priority order. Completing weekly/monthly tasks let them get to the “fun” quarterly tasks.

Sprint backlog: Set to embrace change. Every sprint releases. Customer satisfaction is the top priority. Can restart – a MetaScrum decision.

Automated backlog: Via a bug tracking tool. For each task, the developer is asked: 1. For tasks touched today, how much time is invested in the task? 2. What percent is done? This provides enough data for management “micro-accounting”.

Part 5

Random notes from Open Space

Overheard: “Whenever I’ve been late, I’ve been yelled at by management. One time, we actually pulled it together and finished early. The president called me – not to praise me, but to say, ‘You sandbagged me.'”

Is XP Sustainable? Some things people do:

  • Daily: variety
  • Medium-term: cleanup projects
  • Gold cards (explicitly scheduled ‘free time’)
  • Slack

Rick Mugridge, Domain-Driven Design Patterns and Fit. Examples: queries vs. operations, entities vs. value objects, aggregates (key root objects), repository (turns into query or iterator). Can use a different test harness with the same fixture and same code, but control whether to connect to a “real” database via configuration.

A test writing pattern: create a setup, do some action, repeat the setup but show what has changed.

Norm Kerth: The Secrets of Leading from a Position of No Power

He showed clips of the film Gandhi, then debriefed to explore how Gandhi led. He points out that we can learn about leadership from studying history. Gandhi: “Without a journal of some kind, you cannot unite a community.”

Some principles: transparency, persistence, courage. An unjust law: don’t resist, but don’t comply either.

How did he do it? “He decided soemthing needed change–something important, and worthy of his effort.” He let go of assumed authority.

Inauthentic power = anointed power – (“I’m powerful because I’m a manager.”) – a weak form of power since it goes away if the position does. Authentic power – inside yourself.

Gandhi accepted the cost of punishment. “Better to lose your job for doing something, or for doing nothing?” He was respectful, sought commonality. He knew his support system: the law. He started small, finding people of like mind.

Characteristics of change agents:

  1. Ability to articulate a vision of where you are going (though it can change)
  2. Persistence
  3. Confidence – inner energy for the right thing
  4. Optimism – energy that you lend to someone else so they can gain confidence.

You can cultivate these!

“The most effective coach is sitting next to you, involved from the beginning to the end.”

Gandhi: it paid to advertise. Peaceful revolutions are the lasting ones – they’re really an evolution. Pick your battles – you don’t need to do everything yourself. Know your real mission.

Joshua Kerievsky: Commoditizing Agility

According to Josh, we’re right at the edge of Moore’s chasm, and need to make it easier to make that move.

He had statistics from a project showing productivity increases by a factor of 2.5, achieved in one year with an XP team.

“The agile community is thriving, but transitions are too slow and expensive.” Why? We’re not agile enough in transitions. Problem: serialized knowledge transfer. The shift is repetitive and exhausting.

A couple models: 16 days coaching for a 15-person community. Or, inhouse workshop with 3-6 month full-time coaching for a 20-30 person community. Then try to stretch to other communities, transferring experts out and future experts in. The Problem: people are trained technically but not so much in coaching.

The basics are taught manually. This gives inconsistent content and it doesn’t scale. Books don’t go far enough. Basic knowledge is fragmented. Books require dedicated study.

This marginalizes coaching time: the basics take away from the hard stuff. It burns out the coaches. So we need to commoditize the basics. Experts are in short supply. How many coaches have 3+ years experience? Internal experts are “too indispensable” to share.

Commoditization

“Products and services are so standardized that attributes are roughly the same.” A commodity market has declining prices and profit margins, more competition, and lower barriers to entry.

Ian Murdoch: “Open source and the commoditization of software”: Commoditization happens eventually. It’s unstoppable, but it’s good for all. There’s a cycle: Innovation leads to standardization [accessible] leads to commoditization (maybe) [affordable] leads to innovation. Innovation builds on a custom platform.

Commoditization can go two ways: Decommoditization is an incompatible innovation, or customization. But be careful with decommoditization – it can leave you out of the innovation loop.

What can be commoditized: basics, stories, planning, chartering, retrospectives, … What can’t: advanced practices, specialized things, “living agile values”, people issues.

Josh showed an example of Husqvarna’s sales training – a sort of animated comic book, with a model of the sales process built in. He showed a demo he made, using a screen capture tool.

Bottom line: Tomorrow we can have parallel processing for the basics, leaving more quality time for advanced topics. We can get quality, speedy, consistent knowledge transfer.

Workshop: TDD and Beyond, Astels and Marick

Brian Marick emphasized customer-facing tests, that help with project communication. We don’t want to do all the tests at once. “Doing Fit tests in a blob creates a bottleneck at the customer/QA level. We don’t need all tests, we just need one to get started.” He had the image of “programmers as baby birds” [feed me].

Rick Mugridge: We need to move toward declarative tests. “Procedural tests are a smell.”

Questions: “Do you have to have normal (JUnit) TDD in place before STDD?” “Do you need a strong customer to make it work?”

Roadblocks: Fit is a challenge because you need programmers and testers to make it work.

Fit Unification Summit

Friday (after the conference), a number of Fit implementors met for a Fit unification summit. If you’re interested in that, look for the fit-devl mailing list.

[Published in 5 parts, Aug. 23-27, 2005]