Tag Archives: scrum

Review – Agile Product Management with Scrum (Pichler)

Agile Product Management with Scrum Agile Product Management with Scrum: Creating Products that Customers Love, by Roman Pichler. Addison-Wesley, 2010.

This is a fairly easy read (about 120 pages) explaining the role of the Product Owner in Scrum. I'd describe the target as "someone preparing to fill the Scrum Product Owner role who already knows something about product management." There is a little material on product management techniques, but it's not the emphasis. 

This book is divided into six chapters, talking about the product owner role, envisioning the product, the product backlog, planning, the sprint meeting, and transitioning into the role. There's a good discussion of simplicity, and a little bit on handling this role on large projects.

I particularly liked that most chapters had a section on "Common Mistakes"; they gave me the sense of getting advice from someone who'd seen and worked through these things with real teams. 

100 Impediments

One of the most important things a ScrumMaster does is recognize and remove impediments.

This summary sheet includes 100 impediments (PDF) you may run into (though hopefully not all at once:).

I was inspired to start this list years ago from an article by Barry Boehm (which I've since lost and can't even find a reference for it).

I've found these articles that address impediments, and you may find their perspective useful as well:

[August, 2009.]
 

ScrumGathering ’07

The last ScrumGathering was held in Portland, OR, May 7-11.

On Tuesday, Mike Cohn and I taught a course centered around a series of case studies. Wednesday and Thursday were Open Space sessions.

The overall site for the Gathering is here". This is the result of a session on Games for Scrum, along with a particular planning game, AgileTripTik.

I saw three themes:

  • How can coaches/ScrumMasters effectively intervene?
  • Leadership/vision "versus" self-organization
  • "Beyond iterations" – moves outside generic Scrum approaches

ScrumGathering ’06, Open Space

The fall ScrumGathering is in Minneapolis, MN, this week. I’m here for the two-day open space and the trainer’s meeting. Groups are using http://scrumalliance.pbwiki.com as the conference proceedings. I’ll refer to those articles rather than repeat them here.

Product Owner – I hosted a session on challenges of being a product owner. We described various problems, and broke into subgroups to look at it from the perspectives of "patterns of product owners," "traps," and a "values/principles/practices framework."

Planning a Long Project, hosted by Kelly Weyrauch. We defined a long project as lasting more than a year, and considered a variety of issues that affect planning. Two key challenges are "dealing with changes in scope" and "being absolutely clear about how much is left to do."

Being a Manager, hosted by Jens Ostergaard. Looked at things managers do: remove impediments, set goals, coach/help with individual development, encourage innovation, provide resources/environment, block for the team. But the most important thing is to provide vision.

What is business value?. Hosted by Joe Little. Value to the customer versus benefit to the business. The Kano model – mandatory, linear, delighters. Value vs. cost.

Write a product backlog, hosted by Mike Cohn. We took a couple examples and broke them into stories. Mike talked about using the expected implementation cost as a driver for whether to make stories bigger or smaller. He also described using the back of the card to record conditions of satisfaction, high-level acceptance criteria.

Scrum for non-software applications. We identified a number of areas that use or can use Scrum. Then we explored some challenges and potential solutions for problems they might face.

In the afternoon, we did some closure exercises, to identify important areas going forward and to turn the energy into accomplishment.

I joined the Leadership group. We decided to survey scrum masters and trainers about how leadership is presented as a topic in the courses, to create a resource list of books and articles, and to gather success stories of people on Scrum teams who demonstrated leadership.

I had a fun couple of days, and took away some inspiration, some ideas to try, and some ideas for new articles. It’s always feels good to re-connect to the community.

Agile ’05 Conference Report

Part 1

The Agile '05 conference was July 24-29, 2005, in Denver, Colorado, USA. There were about ten or twelve tracks at all times, so this report is necessarily only a limited bit. Usually I teach, but this time I was an organizer so I got to be more like an attendee in some ways.

Brian Marick and Bob Martin Keynote: Where were we, where are we, where are we going?

Brian Marick: We have different disciplines. We become multi-specialists. "A team of multi-specialists can knock the socks off a team of specialists." But there's no career path for multi-specialists – what conference would you go to?

Brian invited Jim Highsmith up to announce the formation of the Agile Project Leadership Network (APLN). Its focus is on applying agile principles to all projects, not just software. Relative to the Agile Alliance, this organization has consistent principles and intends to collaborate; it just has a different focus.

Bob Martin: "Value the flow of value."

The software industry: was either no process or waterfall. Now we're recognizing the failure of waterfall and the need to measure in-process work. It's time to do "software at home." Where we're going is to resolve this challenge, with short cycles, feedback, testing, craftsmanship.

The agile community: The Scrum paper in '95, XP in '99: resonated with the community. But we were fractured into many agile methods; branding. Now, industry is learning to pull and mix practices; we're at the moment of convergence. Future: head away from brands, strong focus on project management. At the end, we'll have a definition of agile.

Users: Were small teams with a programming focus; limited acceptance tests, project managers "lost." We are a community that has grown. We see 80-people teams often mixing Scrum and XP. 300-person teams exist. A company of thousands doing transitions. Future: we'll grow. Agile project management will happen. Automated acceptance tests will be regarded as the measure of progress.

Brian Marick: There was a group of English cyberneticians. They made discoveries of performance, adaptation, surprise. But oops… no (single) institutional home, no acknowledged base body of techniques, no grooming of successors. Their tradition is gone. To avoid this, "we gotta take over a university department." He announced the "Gordon Pask award", to honor people whose recent contributions demonstrate their potential to be leaders of the field.

Open Space

There were a lot of topics; see the web site. One was "XP Rituals", where we identified a number of fun things:

  • Breath mints
  • Haiku written on card before starting a task
  • Hacky sack at standup
  • Traffic light for builds
  • Darts when a pair finishes a task
  • "Story starting" gong
  • Story completion frog (noise-maker) or bell
  • Daily progress chart – two thermometers, showing days used and points locked in
  • Automatic break program
  • Smiley stickers for story cards

Metaphors, by Dave West

In 2001, metaphor was listed as an official XP practice (part of architecture); in 2005, it was not. Kent has said its redundant as a practice since you can't help using it. (Dave argues it's a learned skill.)

Metaphor has a lifecycle from poetry through use. Lakoff and Johnson say all thought is based on metaphor. Dave argues:

  • If you don't choose consciously, you get one unconsciously.
  • Metaphor selection has a real effect on software design (architecture).
  • Metaphor is a skill you can develop.
  • Therefore, it should be an agile practice.

We have many default metaphors: machine, org chart, entity, dualism, computer science, software engineering, product manufacturing. There are alternatives: e.g., object as person.

To use metaphors well, learn: reflect, experiment, read widely, liberal arts, point of view.

Tim Lister: XP Denver Invited Talk

  • Managers are lonely – they don't have a collegial environment.
  • QA is a bottleneck, and no wonder: it's done late and in serial rather than in parallel.
  • Getting agreement on requirements is not neat. "Gathering requirements" sounds like they're Easter eggs in the yard. But really, most requirements have to be invented.
  • Problem: people believe the customer.
  • It's messy, so model and prototype. Don't prototype solutions – prototype wrong things on purpose (so they can laugh at it).
  • Estimating is overwhelmed by expectations: skew (never early). Separate goals from estimates.
  • Software in general is uninteresting. "One right way" is anti-intellectual.
  • "Never lose the satisfying feeling of making something valuable, together with your colleagues."

 

Part 2

Delivering APIs in an Agile Context, John Major

John Major described a project that was doing custom programming of lab workflows as part of the Human Genome Project. This was an environment with lots of churn: biology, instruments, J2EE.

Tensions:

  • Stable APIs vs. agile change
  • General features vs. "Do the simplest thing that could possibly work"
  • Frameworks vs. global refactoring
  • Framework design vs. features

The result was a platform API, a mix of data-driven (configuration) specification and custom code. The team had the attitude toward process, "We'll try anything for a month." There was lots of testing. They used Jemmy with Swing.

Unique practices:

  • Internal support: old and new features. Used the idea of "office hours" for support. Provided training. Tried having platform developers work on applications, but it didn't work so well.
  • Lightweight code reviews. Rather than pairing, they required at least a 20-minute weekly code review with an office-mate.
  • Agile database schemas. Problem: RDBMS is rigid, but schemas evolve. Solution: build agile schema features and tools to manage change.

Lessons learned: Good:

  • Agile practices help keep technical debt low.
  • Build tools to support RDBMS and large codebase.
  • Pull in senior people to help design and adoption.

Bad:

  • Cost of absorbing API changes is paid by the application teams, but benefits accrue to the business.
  • It's hard to get feature design right (to balance flexibility and focus).
  • The business had trouble understanding the costs of release management. (Branches made the whole thing even crazier; he described a "London subway" map they created to show all the paths.)

Ugly:

  • People issues – don't go into denial.
  • Weren't able to tell QA what the test suites did, so there was overlap between the automated tests and manual testing.
  • Be humble – the platform needs the app and vice versa.

Summary:

  • Build reusable platform technology
  • Use agile practices to cope with change
  • Work with "eyes wide open"

Part 3

Rachel Davies' and Mike Hill Workshop on Informative Workspaces

Informative workspaces:

  • Team memory
  • Visible status (keep it fresh)
  • Automation – light and sound
  • Hawthorne effect
  • Track "puzzles"
  • Root cause analysis
  • Positive focus

Themes: Ownership: Own the space; collective ownership of communication, accountability.

Transparency: Be honest with yourself – show where you really are; peer pressure; reveal hidden/unknown problems.

Keep it Fresh: Drop stale charts. Let anybody update. Automation (e.g., build server status, test status). May use projects or status monitor.

Intra-/Inter-Team Communication: Visual progress => motivation. Whose team sees it.

Hokey-ness: Lack of ownership. Formality. Ownership – lead team to the charts. Time limit for new things. Cool is fun, but must be cool to team.

Kent Beck Open Space on Renewing the Fire

"The edge of knowing and not knowing" – Troy Frever.

What helps keep the fire? A lot of disucssion on being in the zone, in flow – but that's only part of it. Many people crave novelty, mentoring, flow, living on the edge.

There's a "game face" you put on.

Pollyanna Pixton Open Space on Organizational Change

There's a "practice" level, for developers, managers, product owners. But there's also an organizational change level.

"Continuous retrospective" – collect cards all week.

Leadership "versus" self organization.

Kent Beck Open Space on XP for Beginning Teams

He uses an Appreciative Inquiry approach. Take a practice – what does it mean for you? Acts as a "practice inkblot".

An AI formulation:

  1. Remember a good (peak) situation
  2. Explore the circumstances that made it possible
  3. What is the meaning now?

We broke into pairs and did some mind-mapping of a practice.

Research Paper: A Case Study on the Impact of Scrum on Overtime and Customer Satisfaction – Mann and Maurer

Described a team that originally had variable sprints, then shifted its sprint size from 20 to 11 days at the customers' request. Research tried to address, "Does Scrum provide a sustainable pace?" They found there was less overtime (both mean and variance) after Scrum was introduced in this organization. Customers were more satisfied. (Planning closer to delivery led to less misdirected development; playing with the product in the release cycle let them tweak its direction.) Scrum gave them better control and visibility.

Research Paper: An Environment for Collaborative Iteration Planning (Liu, Erdogmus, Maurer)

They used a traditional-looking table with a projected image, and wireless tablet PCs.The thought was that people would create and edit stories on a tablet, then organize and prioritize stories using finger and pens. This would give real-time information, as well as persistence.

The display was nice. One neat trick was that you could "push" a card toward somebody and it would glide at a realistic-looking rate (but never fall onto the floor:)

Existing tools are either cards, which are easy to work with but lack persistence, or planning software, which creates an unbalanced environment (somebody controls the keyboard) but which does have persistence.

The research effort is just beginning; they want to evaluate, "Is it useful? Is it usable? How does it work compared to existing tools?"


Part 4

Jeff Sutherland on Advanced Scrum

"Better, faster, cooler." If this is interesting, see Jeff's paper. I made very quick notes but it's an interesting extension of the Scrum work and I plan to give it more study.

Scrum is out of development and into the whole company. This approach is for experienced ScrumMasters/developers only. See Lawrence Leach: "Eight Secrets to Supercharge Project Performance", Adv. Project, Inc.

Productivity is measured as "features/$100K"

How can we (agile?) win?

One study shows outsourcing cuts costs by only 20% – significant, but not quite the same as cutting 80% or more of costs as some have promised.

The new approach: anticipatory, requires accurate analysis of process and automatic monitoring.

Type A, B, and C Scrum – A: isolated cycles (breaks between sprints), B: overlapping iterations (no handoffs), C: all at once. Consider the sprint level. Need to do pre-staging work for the next iteration inside this one (or else we'll be tempted to fall back to type A).

Advanced scrum has multiple simultaneous concurrent sprints. All sprints release live software.

Changes:

  • MetaScrum for release planning
  • Variable-length sprints
  • Overlapping sprints for one team
  • Pre-stage product backlog
  • Daily scrum of scrums
  • Integrate product backlog and sprint backlog
  • Paperless project management and realtime reporting
  • Administrative overhead of 60 seconds/day/developer and 10 minutes/day/ScrumMaster.

One of the big challenges is having items in the backlog be ready. In their case, that means "intuitive for a doctor" – usable within one hour. This means stories need enough detail, and must have been prototyped with doctors. It forces the product owners to work with the developers to prototype things.

MetaScrum: weekly meeting of stakeholders. Led by the Product Owner. Use the three questions. All product decisions are made here. Release status. All decisions are communicated that day. Damage control plans are executed in the same day.

Iterations: three-month – major product releases (eliminate bugs from portfolio); 1-month sprint for new customer/upgrades (eliminate high-priority bugs); one-week sprint for critical issues. This gives them 45 production releases a year. "Assumes good practices. They're using one code base, working at the tip.

Pre-staging: Overlap sprints, with no break between them. Work only on product backlog that is ready to go the sprint. Pre-staging can double throughput in sprints.

Prioritizing sprints. Constraint theory tells us we must have slack. This lets you optimize multiple overlapping sprints. The backlog is fully automated, with priority levels for the different length sprints (week, month, quarter). Developers focus on one task at a time in priority order. Completing weekly/monthly tasks let them get to the "fun" quarterly tasks.

Sprint backlog: Set to embrace change. Every sprint releases. Customer satisfaction is the top priority. Can restart – a MetaScrum decision.

Automated backlog: Via a bug tracking tool. For each task, the developer is asked: 1. For tasks touched today, how much time is invested in the task? 2. What percent is done? This provides enough data for management "micro-accounting".


Part 5

Random notes from Open Space

Overheard: "Whenever I've been late, I've been yelled at by management. One time, we actually pulled it together and finished early. The president called me – not to praise me, but to say, 'You sandbagged me.'"

Is XP Sustainable? Some things people do:

  • Daily: variety
  • Medium-term: cleanup projects
  • Gold cards (explicitly scheduled 'free time')
  • Slack

Rick Mugridge, Domain-Driven Design Patterns and Fit. Examples: queries vs. operations, entities vs. value objects, aggregates (key root objects), repository (turns into query or iterator). Can use a different test harness with the same fixture and same code, but control whether to connect to a "real" database via configuration.

A test writing pattern: create a setup, do some action, repeat the setup but show what has changed.

Norm K erth: The Secrets of Leading from a Position of No Power

He showed clips of the film Gandhi, then debriefed to explore how Gandhi led. He points out that we can learn about leadership from studying history. Gandhi: "Without a journal of some kind, you cannot unite a community."

Some principles: transparency, persistence, courage. An unjust law: don't resist, but don't comply either.

How did he do it? "He decided soemthing needed change–something important, and worthy of his effort." He let go of assumed authority.

Inauthentic power = anointed power – ("I'm powerful because I'm a manager.") – a weak form of power since it goes away if the position does. Authentic power – inside yourself.

Gandhi accepted the cost of punishment. "Better to lose your job for doing something, or for doing nothing?" He was respectful, sought commonality. He knew his support system: the law. He started small, finding people of like mind.

Characteristics of change agents:

  1. Ability to articulate a vision of where you are going (though it can change)
  2. Persistence
  3. Confidence – inner energy for the right thing
  4. Optimism – energy that you lend to someone else so they can gain confidence.

You can cultivate these!

"The most effective coach is sitting next to you, involved from the beginning to the end."

Gandhi: it paid to advertise. Peaceful revolutions are the lasting ones – they're really an evolution. Pick your battles – you don't need to do everything yourself. Know your real mission.

Joshua Kerievsky: Commoditizing Agility

According to Josh, we're right at the edge of Moore's chasm, and need to make it easier to make that move.

He had statistics from a project showing productivity increases by a factor of 2.5, achieved in one year with an XP team.

"The agile community is thriving, but transitions are too slow and expensive." Why? We're not agile enough in transitions. Problem: serialized knowledge transfer. The shift is repetitive and exhausting.

A couple models: 16 days coaching for a 15-person community. Or, inhouse workshop with 3-6 month full-time coaching for a 20-30 person community. Then try to stretch to other communities, transferring experts out and future experts in. The Problem: people are trained technically but not so much in coaching.

The basics are taught manually. This gives inconsistent content and it doesn't scale. Books don't go far enough. Basic knowledge is fragmented. Books require dedicated study.

This marginalizes coaching time: the basics take away from the hard stuff. It burns out the coaches. So we need to commoditize the basics. Experts are in short supply. How many coaches have 3+ years experience? Internal experts are "too indispensable" to share.

Commoditization

"Products and services are so standardized that attributes are roughly the same." A commodity market has declining prices and profit margins, more competition, and lower barriers to entry.

Ian Murdoch: "Open source and the commoditization of software": Commoditization happens eventually. It's unstoppable, but it's good for all. There's a cycle: Innovation leads to standardization [accessible] leads to commoditization (maybe) [affordable] leads to innovation. Innovation builds on a custom platform.

Commoditization can go two ways: Decommoditization is an incompatible innovation, or customization. But be careful with decommoditization – it can leave you out of the innovation loop.

What can be commoditized: basics, stories, planning, chartering, retrospectives, … What can't: advanced practices, specialized things, "living agile values", people issues.

Josh showed an example of Husqvarna's sales training – a sort of animated comic book, with a model of the sales process built in. He showed a demo he made, using a screen capture tool.

Bottom line: Tomorrow we can have parallel processing for the basics, leaving more quality time for advanced topics. We can get quality, speedy, consistent knowledge transfer.

Workshop: TDD and Beyond, Astels and Marick

Brian Marick emphasized customer-facing tests, that help with project communication. We don't want to do all the tests at once. "Doing Fit tests in a blob creates a bottleneck at the customer/QA level. We don't need all tests, we just need one to get started." He had the image of "programmers as baby birds" [feed me].

Rick Mugridge: We need to move toward declarative tests. "Procedural tests are a smell."

Questions: "Do you have to have normal (JUnit) TDD in place before STDD?" "Do you need a strong customer to make it work?"

Roadblocks: Fit is a challenge because you need programmers and testers to make it work.

Fit Unification Summit

Friday (after the conference), a number of Fit implementors met for a Fit unification summit. If you're interested in that, look for the fit-devl mailing list.

[Published in 5 parts, Aug. 23-27, 2005]

 

 

 

Scrum Development on a Page

Scrum is an agile software process, centered around a self-organizing team and a 30-day development cycle. This page contains a one-page summary of its development process.

Scrum on a page. (Microsoft Word .doc format, Adobe .PDF format)

 

Related Articles: Extreme Programming on a Page.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License.

[Revised July, 2005. Previous version developed January, 2004. PDF format courtesy of Eric Chamberlain.]

Older version (2004) still available: PDF, DOC

Bill Wake is the author of the Refactoring Workbook, and Extreme Programming Explored. He works as a software coach and trainer. Contact him at William.Wake@acm.org or 1-804-519-6799.

Scrum Gathering Oct ’04

The Scrum gathering was a workshop gathered for a couple days in Denver, this past October. We worked in three groups: metrics, process, and facilitation. (Scrum is an agile process. I think of it as approximately the project management part of XP, though that's of course not fair to either one:)

I participated in the facilitation group. (The others were metrics and process.) We spent a lot of our time trying out a variety of simulations and other means to help teams understand what it means to do Scrum. I contributed two exercises: Push Line/Pull Line (a demonstration of lean manufacturing), and Second Agenda [now called "Scrum from Hell"] (a role-play of a standup meeting).

Esther Derby shared a debriefing framework she uses: What/Gut/So What/Now What? "What" asks for objective data about what happened. "Gut" asks for your emotional reaction. "So what" asks for your interpretation. And "Now what?" asks what you'll do next. This framework helps us avoid jumping to interpretation too early.

The process group identified challenges around cross-functional teams, the role of testing, and the challenges of leading or lagging behind (e.g., the analysts want to be an iteration ahead). The metrics group identified metrics as a means of making management comfortable, and created and highlighted various means of helping with that.

I enjoyed this workshop a lot. As always, it's good to meet people and put names with faces. It really helped me feel a sense of community.

On my way out of town, I passed a quotation on a building: "Stay firmly in your own path and dare; be wild two hours a day!" – Paul Gauguin.

Review – Agile Project Management with Scrum

Agile Project Management with Scrum, Ken Schwaber. Microsoft Press, 2004.
This book presents stories from several software teams adopting Scrum. It takes stories from several perspectives such as “The Product Owner” or “Scaling Projects using Scrum.” What I like best is that he describes the situation, and then tells you how the team addressed it, so you can practice your own thinking about how you’d improve projects. (Reviewed Nov., 2004)

Scrum from Hell

Goal: Practice standup meetings with a dysfunctional one.

Time: 15-20 minutes.

This exercise is for a group of 6 to 8 participants who stand up, and some number of observers who may sit and watch.

Background

  • Describe a Scrum meeting, so people understand the three questions and the flow of the meeting.
  • Describe common problems and interventions that address them. For example:
    • Implicit impediment: Listen to everything; sometimes someone mentions an impediment but doesn’t identify it as such.
    • Side discussion: ask people to listen when they’re not speaking.
    • Rambling on: ask people to summarize more quickly.
    • Sidetracked meeting: ask people to have a meeting immediately afterwards for people who care about the topic.
    • Observer ("chicken") who speaks: remind them that they’re an observer.
    • Late arrival: Charge them $1 if that’s what your team does; offer to fill them in after the meeting.
    • etc.

Preparation

Prepare a set of cards, each with a secret goal.

Make twenty identical cards that say:

  • Answer the three questions.

The other ten cards should say (one each):

  • Only speak to or look at the ScrumMaster (ignoring everybody else unless you’re asked a direct question).
  • Arrive late.
  • Hidden impediment: Mention an impediment but don’t be obvious about it.
  • Noisy chicken: start by saying, "I’m only an observer" and then report on things the group doesn’t care about.
  • Silent chicken: as an observer, just say "pass" or "I’m just observing" when it’s your turn.
  • Ask a clarifying question on somebody else’s turn.
  • Ramble on until you’re asked to move on.
  • Try to sidetrack the meeting.
  • Try to solve somebody’s problem.
  • Start a side discussion.

Shuffle the cards together.

Mechanics

Have the group stand in a circle. Identify one person as the ScrumMaster.

Tell the group these three things:

  1. "I’d like you to imagine that you’re a member of a team developing an e-commerce site; take a moment to decide what you’ve been working on and how you’ll answer the three questions."
  2. "I’m going to give each of you a secret goal; this card is for you only to see. During the scene, you’ll have this goal as hidden second agenda."
  3. "If the ScrumMaster addresses your behavior, then don’t persist in it."

Have the group do their standup meeting. Then debrief them.

Debrief

"What behaviors did you see?"
This question is for participants and observers.

"How was it for you?"
ScrumMaster
Participant

"What insights do you have for the ScrumMaster?"
This question is for participants and observers.

"What does this tell you about the role of the ScrumMaster?"
This question is for anybody. 

"What will you do in the future?"
This question is for anybody.

Variations

  • You can vary the balance of "simple" vs. dysfunctional cards. (Try the exercise before you make it harder.)
  • You can repeat the article with different people acting as ScrumMaster.
  • You can do a large group "fishbowl" style, with the bulk of the group observing the team.

Source:
[William C. Wake, 2004. Developed for the Scrum Gathering in Denver, October, 2004. Thanks to the participants there for helping to improve it. Name changed 5-16-05; was originally "Second Agenda."]

The Knowledge-Creating Company – Extended Summary

This is a summary and critique of The Knowledge-Creating Company, by Ikujiro Nonaka and Hirotaka Takeuchi. They are the original protagonists of the Scrum approach.

 
The Knowledge-Creating Company, by Ikujiro Nonaka and Hirotaka Takeuchi. Oxford University Press, 1995. ISBN 0-19-509-269-4.

This book is a thoughtful look at how organizations acquire knowledge. I'll describe the main thrusts of their argument, and consider how it relates to software development.

Knowledge

Their thesis is that Japanese and Western societies and companies have developed different understandings of what it means to learn and to innovate, but the way forward is through a synthesis of both styles.

Knowledge is taken as the basis for what an organization does, but it's important to know that creating knowledge can be as important as processing knowledge.

A key idea is that some knowledge is tacit (i.e., internalized) and other is explicit. Western philosophy is described as having struggled to understand whether knowledge is based on what we experience (empiricism) or inherent truths (rationalism), and to have focused more on explicit knowledge. Japanese thought has tended to treat tacit knowledge as more important (but not really addressed epistemology as a philosophical tradition).

The quote Peter Drucker (Post-Capitalist Society, p. 24), saying: a skill "could not be explained in words, whether spoken or written. It could only be demonstrated." And, "the only way to learn a techne [skill, in Greek] was through apprenticeship and experience."

Knowledge creation is the process of making tacit knowledge explicit. The example of the Honda "Tall Boy"  (a new car style that moves away from the sedan look) demonstrates how this can happen. First, metaphors and analogies are used to help provide a framework for thinking about things that can't be easily described. "As such, metaphor is highly effective in fostering direct commitment to the creative process in the early stages of knowledge creation." ("Contradictions inherent in a metaphor are then harmonized by analogy.") Second, knowledge moves from an individual to an organization; teams provide a shared context and multiple points of view. Finally, ambiguity and redundancy are used: ambiguity lets many things fit within a framework; redundancy lets a group explore many aspects of a direction before deciding where to go.

The authors consider the learning theories of Bateson, and Argyris and Schon: single-loop learning and double-loop learning. "From our viewpoint, the creation of knowledge certainly involves interaction between these two kinds of learning, which forms a kind of dynamic spiral." (p. 44) Nonaka and Takeuchi are not as enamored of Senge's system learning; they think it doesn't escape the Cartesian viewpoint.

There are two key dimensions in a theory of organizational knowledge creation: an epistemological dimension–whether the knowledge is tacit or explicit– and an ontological dimension–whether it is known at the individual, group, organization, or inter-organization level. Knowledge is about beliefs and commitment, about action, and about meaning. Western philosophy has the formulation "knowledge is justified, true belief" and has focused on what truth means. The authors view the "justified belief" part as the critical part.

Knowledge conversion (between explicit and tacit) is a crucial part of the social job of sharing knowledge. There are four key modes:

From  \  To

Tacit Knowledge

Explicit Knowledge

Tacit Knowledge 1. Socialization
"Sympathized knowledge": Share experiences to create tacit knowledge. Example: on-the-job training. Example: interacting with customers.
2. Externalization
"Conceptual knowledge":Articulate tacit knowledge explicitly: metaphors, concepts, hypotheses, models, writing.
 
Explicit Knowledge 4. Internalization
"Operational knowledge": Learning by doing, to develop shared mental models and technical know-how.
3. Combination
"Systemic knowledge": Manipulating explicit knowledge by sorting, adding, combining, etc. Example: formal education.

The result of all this is (or can be) a knowledge spiral. It is sustained by using dialog to move from socialization to externalization; by linking explicit knowledge to move from externalization to combination; learning by doing to move from combination to internalization; and field building to move from internalization to socialization. Notice how it moves back and forth between explicit and tacit, and how it can increase its level (individual to group and beyond).

All this doesn't happen accidentally; there are conditions at the organization level that can promote it:

  • Intention: aspiration to goals, fostering commitment by engaging employees in fundamental questions.
  • Autonomy: letting people act independently as far as possible.
  • Fluctuation and creative chaos: to stimulate interaction between the organization and its outside environment; there is a breakdown that must be overcome, followed by reflection on what happened. (Fluctuation without reflection leads to destructive chaos.)
  • Redundancy: "the existence of information that goes beyond the immediate operational requirements of organizational members." (p. 80)
  • Requisite variety: The organization's variety must exceed that of its environment. Requires equal, fast access to information.

With these conditions in place, the organization can move from tacit knowledge by "sharing tacit knowledge, creating concepts, justifying concepts, building an archetype, and cross-leveling knowledge." (p. 84) This moves to explicit knowledge in the market, which can feed back information and help the cycle continue.

Management and Organizational Structure

Top-down (the traditional hierarchical model) and bottom-up management approaches both have limitations. Top-down presumes the answers from from above, but bottom-up can be too independent. Both models ignore the ability of middle management to reconcile the problems.  The middle-up-down approach recognizes that middle managers often create knowledge (the front-line is too busy with today, top management is out of touch). This puts them in a dynamic position, and belies the trend to eliminate middle management.

Middle-up-down acknowledges that top management has dreams, but middle management is the group that makes it happen with a mid-range theory. Middle managers act as catalysts.

A bureaucratic structure is too locked in to repeating its past; task forces or projects are important but need overall structure. The solution is a hypertext organization: combining a business system layer, a project-team layer, and a knowledge base layer. Team members can shift layers, but belong to only one at a time (unlike a matrix approach). Knowledge is combined across layers. Projects tend to be controlled more directly by top management, letting the team focus on short-term needs and speeding up communication.

"These new organizations: (1) tend to be flatter than their hierarchical predecessors; (2) assume a constant dynamic rather than a static structure; (3) support the empowerment of people in building intimacy vis-a-vis customers; (4) emphasize the importance of competencies – unique technologies and skills; and (5) recognize intellect and knowledge as one of the most leverageable assets of a company." (p. 162)

Team Organization

The authors describe three approaches to organizing teams:

  • Relay style
  • Rugby style
  • American football style

Relay style

This is the traditional waterfall approach. It has a long lead time, and slow learning, and won't be discussed further.

Rugby style

Rugby style involves creating a cross-functional team that works together directly. It handles changes in requirements well, and has a short lead time. Long and continuous interaction among project team members clarifies the grand concept (business strategy), mid-range concept, and the product concept.

Benefits of the rugby style:

  • Shorter development cycle.
  • Able to operate with little organizational conflict as everybody is focused on the goal.
  • Involves production, which leads to designs that can be manufactured.
  • Results in short lead time and high quality.

Drawbacks of the rugby style:

  • Danger of groupthink: preserving unity and conformance above change, i.e., moving to the lowest common denominator instead of the best solution.
  • Reliance on socialization can lead to inefficiencies as the project team grows in size.
  • May weight marketing and manufacturing over technological potential (aspiring to less than is possible).
  • May make it hard to set performance targets, as the process changes so much.

American Football Style

To balance the benefits and drawbacks, the authors propose "American football" as a new metaphor, suggesting a way to get short lead time and high performance levels.

  • A small group of project leaders has an intensive dialog at the start of the development, with the goal of clarifying grand concept, mid-range concept, and product concept. (This is like football coaches making a game plan.)
  • Teams form for specialization certain functions.
  • Functional departments move simultaneously (rugby style), to meet cost, performance, and date goals.
    • Large-scale socialization (e.g., visiting markets)
    • Interdepartmental collaboration
    • All teams evaluate prototype to see if product concept is met
  • In this way, a small, cohesive group can make overall plans, and then global teams can apply rugby style mostly independently.

Summary

These are the key ideas:

  • Tacit vs. explicit knowledge.
  • Interaction between tacit and explicit knowledge is by an individual, not an organization.
  • The knowledge spiral happens when team members share their experiences and models.
  • Most organizational knowledge-creation happens at the group level, in an environment supported by the organization.
  • Knowledge creation is non-linear, interactive, and iterative

The authors propose these guidelines for an organization involved in knowledge creation: (page 227)

  1. Create a knowledge vision
  2. Develop a knowledge crew
  3. Build a high-density field of interaction at the front line
  4. Piggyback on the new-product development process
  5. Adopt middle-up-down management
  6. Switch to a hypertext organization
  7. Construct a knowledge network with the outside world.

Finally, their hope for the future is hypertransformation, that is, multiple transformations across multiple dimensions.

Discussion, Relative to Software Development Processes

Nonaka and Takeuchi have focused on an important problem, knowledge creation. It's important in physical manufacturing, and clearly central to software.

Agile teams bring people together to work on problems, in an intensive way. (Collocation, pairing, frequent inspection, and feedback help sustain this intensity.) In working together, the team socializes its values and practices. This lets the team work together efficiently, but makes it harder for outsiders to understand what's going on.

The middle-up-down and hypertext organization approaches propose a mechanism by which middle managers can have an important impact on their teams and the organization as a whole. Too many middle managers spend too much time acting as aggregators of status reports rather than driving and sustaining the creation of organizational knowledge.

The comparison of Rugby (i.e., Scrum) style and American football is especially interesting in the light of efforts to "scale" agile processes. When a job is too big for one team, a small team sets the initial tone and direction, and then rugby-style teams operate in parallel. (Note that this has three parts: big job, cohesive guiding team, and parallel rugby teams.)

Scrum addresses large-project situations with the Scrum of Scrums approach. Grizzly (being developed by Ron Crocker) has practices around team coordination and architecture development (and XP-style practices within teams). XP, to the extent it says anything about scaling, says "conquer and divide": solve the essence in a small team before splitting into parallel teams. All these approaches seem compatible with the football approach.

Other approaches to large teams often seem to miss the essence of the rugby or Scrum approach: they don't value the intense socialization, and may ignore the importance of a cohesive guiding team. In this, they may unwittingly give up opportunities for team learning and performance improvement.

I highly recommend this book: The Knowledge-Creating Company, by Ikujiro Nonaka and Hirotaka Takeuchi.


[Written 2-3-04.]

Bill Wake is the author of the Refactoring Workbook, and Extreme Programming Explored. He works as a software coach and trainer. Contact him at William.Wake@acm.org or 1-804-519-6799.

Reivew – Agile Software Development with Scrum

Agile Software Development with Scrum, Ken Schwaber and Mike Beedle. Prentice Hall, 2001.
Scrum explicitly relies on the emergence of a true team. By focusing on 30-day sprints, daily standup meetings, and management’s ability to remove obstacles, a team has an environment in which to flourish. There’s also the idea of a “Scrum of scrums,” which I think has the best potential for large-scale agile teams. (Reviewed Nov., ’02)