Tag Archives: HCI

Review – Don’t Make Me Think (Krug)

Agile Product Management with Scrum

Don't Make Me Think: A Common Sense Approach to Web Usability, by Steve Krug. New Riders Press, 2005.

This thin book provides guidance on the basics of web usability. It focuses on the importance of doing things in a way that meet users' expectations. In addition to some well-thought out examples of re-design, he spends about a third of the book on a simple and cost-effective approach to usability testing. This would be a good choice for your first book on usability.

Interface Seams: Activity Theory

Activity Theory is a framework that provides some insight into issues around how people use tools. This is a small summary of some of its ideas.


On the agile-usability list, Larry Constantine brought up the notion of Activity Theory as a framework for some aspects of HCI. That prompted me to explore the topic a little. This note is a summary of the terms and key ideas of the paper "Activity Theory," by Tyler Waite. Any misunderstandings are certainly mine.

Key Ideas

Activity – a high-level motivating concept. For example, fixing dinner.
Action – something to accomplish a specific goal. For example, making a salad. The effects of an action are not fully predictable (they may have to deal with exceptional situations). After performing an action, someone will assess the situation to decide if the goal has been achieved.
– behaviors not requiring conscious effort. For example, slicing a carrot. They involve automatic responses to a perceived state.

Activities consist of many actions. Actions consist of many operations.

Key principle: [Over time, and with practice,] activities may become actions (e.g., if they're part of a larger activity), and actions can become operations.

Waite gives the example of learning to write. At first, even drawing a letter is an activity. As the writer improves, it becomes an action, and eventually an operation.

(I'm also reminded of Kano's model: something starts out as a delighter (unexpected but delightful when found), then becomes a satisfier (normal feature, more is better), then a dissatisfier (if you don't have it, you're not even in the game).)

Community – activities have a social component; they don't just affect one individual. A primary role of the community is to create rules.

Rule – inform individual of desired outcome; inform individual about the object that is the focus of the activity; instruct the individual on how to work on an object.

An individual decides which rules to follow, applies them, and revises/adds rules. Rules can be illuminative (provide a way to view the problem) and restrictive (prevent alternative approaches).

Tool – can be physical artifacts, ideas, theoretical frameworks, etc.

Object – can be physical or mental/abstract.

Engestom's Model of Activity Theory: (as shown by Waite)

[Written 1/21/2007.]

Review – Things that Make Us Smart

Things that Make Us Smart, Donald A. Norman, Perseus Books, 1993.

Similar in tone and scope to his other books (that’s an endorsement:), Things that Make Us Smart explores ways in which we use artifacts to improve our cognitive abilities. A lot of it is anecdotal, but anything that makes me want to both re-consider where things are in my kitchen and also re-arrange my software has some power to it. (Reviewed Jan., ’04)

Review – Paper Prototyping

Paper Prototyping, by Carolyn Snyder. Morgan Kaufmann, 2003.

The “Scandinavian design” tradition in human-computer interaction says that people should be involved in creating the systems they will have to use. Electronic tools for rapid screen design may provide a high-fidelity view of how a system will look, but they do so by creating a privileged group (the software people who know the tool). Paper prototypes overcome this by putting everybody on an equal footing. Furthermore, they do so cheaply.

Snyder describes how to create and use paper prototypes of user interfaces. While she describes some systematic ways to use them, this is a lighter approach (and less broad) than something like Constantine and Lockwood’s Usage-Centered Design. That doesn’t diminish the book’s charm. I particularly liked the chapters that showed how to construct pages for panes, scroll-bars, etc. It left me feeling like, “I could have worked my way here, but somebody’s done the work for me.” (Reviewed Dec., ’03)

Interface Seams: Ubiquity

Ubiquitous computing may be worth pursuing.

A material has been developed for storage and transmission of information. It is capable of displaying both images and text. Color support is available at moderately higher cost. While there are a variety of standard display formats, it can be manufactured in almost any size.

The display is solar powered. Manufactured and handled carefully, it can last longer than CD-ROM. It is flexible, and can be attached to many surfaces.

It has some downsides:

  • Duplication is fairly cumbersome.
  • High training costs (on the order of 3-10 years!)
  • High storage costs.

It took centuries to refine the art of paper.

What can be ubiquitous?

  • Pen that knows what it wrote
  • "Voice box" for dictation
  • Tool to show remote state (eg sculpture showing your family)

As hardware prices drop, power management improves, and more systems accommodate remote networked computing, we’ll see less computers but there will be more of them than ever before.

[Written 9-15-1998, published on xp123.com June, 1999.]

Interface Seams: Leaving the Desktop Behind

GUIs aren’t the end.

GUIs (graphical user interfaces) were amazing on a 1MIPS Macintosh. Now that clock speeds are several hundred times that speed, are GUI interfaces several hundred times better?

We need a new interface that’s tantalizing at 500 MHz, great at 5 GHz, and a little boring at 500 GHz. Some possibilities:

  • 3D and VR (virtual reality)
  • Audio
  • Speech (interpreting the spoken word)
  • Ubiquity
  • Appliances
  • Agents
  • Wearable computers
  • Mobile/wireless computers
  • Location-sensitive

We also need to go the other way – what are we missing on the low end?

  • Appliances
  • Focused applications (eg electronic rolodex)
  • Smart badges
  • Text-oriented applications (no fonts etc.)

[Written 9-1-1998.]

Interface Seams: Patterns for Interactive Applications

These patterns describe some of the forces and decisions to be made while implementing an interactive application. [This is a submission to PLoP’98.]

Interactive Applications

Many applications that we think of as "killer applications" on personal computers, such as word processors or spreadsheets, developed before graphical user interfaces were common. While graphical interfaces have brought several benefits to these programs (better usability, scrolling, fancy fonts, and crisper graphics), the core models of these programs haven’t changed: word processors still deal with text, and spreadsheets with matrices of formulas.

Patterns are a literary form for capturing design knowledge. Developed by Christopher Alexander to describe architecture (the physical kind) [1], they have been adopted and adapted to describe software architecture [2] [3]. A pattern guides a decision or class of decisions that must be made, by describing a problem and its solution in context. A group of patterns is sometimes organized into a pattern language, which provides guidance through a series of decisions.

The pattern language described below attempts to generalize from applications such as Emacs [4, 5], WordStar [6], dBASE III [7], and VisiCalc [8], and to capture their essence. These patterns apply fairly early in the design, after task analysis and object-oriented analysis have identified key tasks, objects, and relationships.

In the Portland Pattern Repository [9], Kent Beck has articulated two related patterns: in Story, he notes that while stories are presented in time, the interface must be designed in space. In One Task Per Window, he provides guidance on how to associate windows with tasks. The patterns below are intended to weave into that same design space, and assume Beck’s patterns as a given.

This pattern language addresses three aspects of application design: appearance, behavior, and extensibility.

Design of Screens

Make Position Reflect Relationships

Problem: The user must deal with hundreds, thousands, or more related objects.

Context: Analysis has identified critical objects and relationships.

Forces: The user’s task requires focusing on a small number of abstract, key relationships.

Solution: Give each object a visual representation based on its attributes. Let the object’s placement on the screen reflect the key relationships.

It may be helpful to have a structuring metaphor, a simple guiding principle that lets the user build a mental model of how the system works.


  • Emacs [4, 5]: the objects are characters, the key relationship is their sequence.
  • Spreadsheets [8]: the objects are formulas, arranged in a table.
  • SortTables [10]: the objects are records, arranged in sequence.

Related Patterns: Arrange Zones By Similar Persistence, Make Space Proportional to Importance, and Put Less Important Information on the Periphery discuss how to structure the screen. Beck’s Story and One Window Per Task [9] provide guidance on preserving key relationships between objects and the user’s task.

Related Ideas: Horton [11] discusses ways to arrange online documents to reflect a logical pattern.

Arrange Zones By Similar Persistence

Problem: There are many objects to show, but it is not clear how to share screen space among them.

Context: Screen-based interface.

Forces: Users need a consistent screen structure so they know what to expect.

Solution: Partition the window into zones. Items in a given zone should change at about the same rate. If possible, use dividing lines or other indicators to clearly delineate the zones.

Zones should have inviolate borders – they should not interfere with each other. (Popup menus and dialog boxes may be acceptable exceptions, provided they don’t cover information critical to their use.)

Zones should be arranged in a way that accords with Position Reflects Relationship.

Example: Spreadsheets typically have a strong zone structure: the cells, their labels, and a data entry box.

Counter-Example: Some versions of the Unix editor vi have a status/command line, but it gets covered by the user’s text when a command line is not needed. Text in that line is thus ambiguous.

Related Patterns: One Window Per Task [9]. Beck has alluded to a pattern Split Into a Small Number of Zones that is probably related.

Related Ideas: Publishing uses the notion of a grid in layout design, providing a stable visual format throughout a publication [12].

Make Space Proportional to Importance

Problem: How big should the various zones be?

Context: A window divided into zones.

Forces: Many objects compete for screen space.

Solution: Make the space for a zone proportional to the importance of the objects and relations in that zone.

Example: A spreadsheet devotes most of the screen space to the user’s data.

Counter-Example: At a local copy shop, the word processor is configured to show all indicators: a menu bar, a tool bar, font/size indicators, and other buttons. The screen is small, and the various indicators take up almost half the screen space, leaving little room for editing – the main task.

Related Ideas: Tufte has a rule for graphics: Maximize the data-ink ratio, within reason [13], meaning that ink should be spent providing useful information.

Put Less Important Information on the Periphery

Problem: What goes where on the screen?

Forces: People will tend to focus on the middle.

Solution: Put the biggest and most important zone in the middle. Put less important information surrounding it, on the edges. This information is usually about less important objects, or meta-information about the central objects.

Be aware of reading habits, however. Readers of English expect to look left to right and top to bottom.

Related Patterns: Make Position Reflect Relationships takes precedence over this pattern.

Related Ideas: Horton proposes a similar approach for online documents [11].

Design of Behavior

Computer Adds Value

Problem: The user is working too hard.

Context: A computerized solution is being designed. Forces:

  • The user would like the computer to do the tedious parts of the task.
  • Programming all parts of the task might be expensive or impossible.

Solution: Ensure that the computer adds value to the performance of the user’s task. Look for places where the computer can go beyond merely being a storage device, and can provide summary information, computation, automatic placement, analysis, and arranging.


  • SortTables [10]: a self-sorting box of file cards.
  • Spreadsheet [8]: automatically updating formulas.

Let the User Control Navigation

Problem: The user gets confused when information scrolls past too quickly to read.

Forces: Users like to feel in charge of what they are doing and the pace of their work.

Solution: Let the user control navigation. Users should be able to move the focus of attention at their own pace. This navigation might be controlled by scrollbars and/or navigation keys (cursor keys, top/bottom, page-up/down).

Example: Most programs with graphical interfaces use scrollbars to control navigation.

Counter-Example: Some terminal emulators let text scroll past, without trying to save it.

Engender a Sense of Progress

Problem: The user needs a sense of how much work is done and how much is left to do.

Forces: It’s frustrating for a user not to know where they are.

Solution: Structure the visual layout to provide some indication of pr ogress. This may take the form of progress bars, counters, scroll bars, etc.

Example: The thumb of a scrollbar provides an indication of position as well as being a navigational widget.

Related Ideas: Thimbleby [14] briefly mentions " sense of progress" as a consequence of his equal opportunity design principle.

Support Lapses in Attention

Problem: Users forget what they’re doing, in the middle of doing it.

Forces: Real users are in an environment the computer is unaware of: phones ring, people drop by, and so on.

Solution: Attack the problem on several fronts:

  • Arrange Zones By Similar Persistence lets the user focus on what has changed since they last looked at the screen.
  • Let the User Control Navigation restricts what will happen without user involvement.
  • Modelessness prevents context-dependent situations.
  • If there must be modes, provide visual indicators of the current mode

Counter-Example: The vi editor has a command mode and a text entry mode, but no indication of which is active.

Related Ideas: Thimbleby [14] calls this the "gone for a cup of tea problem," and discusses how it interacts with modelessness.

Design for Extensibility

Two-Layer Architecture

Problem: We need several interfaces to a system (e.g., graphical, audio, application programming interface, etc.)


  • Real systems must operate in many environments.
  • We can’t afford to develop completely independent versions of a system.
  • Users may require different interfaces at different times.

Solution: Use a two-layer architecture: define a core set of commands that do the work, and define bindings that map keystrokes or mouse actions to those commands.

Example: Consider an electronic mail system: we might like a dial-in audio version for use on the road, and a graphical version for use at the desk.

Use an Existing Extension Language

Problem: How to provide everything people want from the command set?


  • Creeping featurism in the command set.
  • Language design shouldn’t be the focus of the application development.

Solution: When a command language (as in Two-Layer Architecture) reaches the point where control structures become necessary, there will be inexorable pressure to grow it into a full programming language. Thus, define the core commands as primitives, and embed them in an existing language such as TCL [15] or Lisp [16].

Example: Emacs [4, 5] has Lisp as an implementation and extension language.

Counter-Example: An early digital video developers’ toolkit provided some simple commands. They tried to grow these into a language, but it had strange restrictions (control structures could be nested only two layers deep, limited number of variables, etc.).


This series of patterns has been designed as a pattern language to help in developing screen-based applications, by identifying tradeoffs and pressures in screen design, behavior, and extensibility. Most of these patterns have strong precedents in the human-computer interaction and documentation fields. With the pattern form, we hope to capture more than just a slogan. We hope that the context and forces expose some design tradeoffs, and that the patterns can be used in a generative way, for a particular class of applications.


  1. Alexander, Christopher, et al. A Pattern Language: Towns, buildings, construction. Oxford University Press (New York), 1977.
  2. Gamma, E., Helm, R., Johnson, R., and Vlissides, J. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley (Reading, MA), 1995.
  3. Coplien, J. O. and Schmidt, D. C., editors. Pattern Languages of Program Design. Addison-Wesley (Reading, MA), 1995.
  4. Stallman, R. EMACS: The Extensible, Customizable Self-Documenting Display Editor. In Proc. SIGPLAN/SIGOA Symposium on Text Manipulation, Portland, OR, ACM, 1981.
  5. Finseth, C. A. The Craft of Text Editing: Emacs for the Modern World. Springer-Verlag (New York), 1991.
  6. Ettlin, W. A. WordStar Made Easy. Osborne/McGraw-Hill, 1981.
  7. Weber Systems Inc., Staff. dBASE III Users’ Handbook. Ballantine Books (New York), 1985.
  8. Beil, D. H. The VisiCalc Book (Atari Edition). Reston Publishing Co. (Reston, VA), 1982.
  9. Beck, K. "User Interface" in the Portland Pattern Repository.
  10. Wake, W. C. and Fox, E. A. SortTables: A Browser for a Digital Library. Conference on Information and Knowledge Management, CIKM-95, ACM, 1995.
  11. Horton, W. Visual Rhetoric for Online Documents. IEEE Transactions on Professional Communication, 33(3), Sept., 1990, pp. 108-114.
  12. Swann, A. How to Understand and Use Design and Layout. North Light Books (Cincinnati, OH), 1987.
  13. Tufte, E. R. The Visual Display of Quantitative Information. Graphics Press (Cheshire, CT), 1983.
  14. Thimbleby, H. User Interface Design. ACM Press (New York), 1990.
  15. Ousterhout, J. K. Tcl and the Tk Toolkit. Addison-Wesley (Reading, MA), 1994.
  16. McCarthy, J. et al. LISP 1.5 Programmers Manual. MIT Press (Cambridge, MA), 1965.


Thanks to Dr. Edward A. Fox of Virginia Tech, for his encouragement and support in exploring this topic.

Copyright 1998, William C. Wake
PLoP ’98 submission, 5-12-98

Interface Seams: Ad-Hoc vs. Planned Collaboration

Groupware is often examined on the dimensions of time and place; ad-hoc vs. planned collaboration introduces a new dimension.

A common framework for examining groupware is time and place: same time – same place, same time – different place, different time – same place, different time – different place. We introduce a new dimension: how planned is the interaction in the groupware?

Some collaboration is formal. For example, when you phone support from a software vendor, you expect an infrastructure: an 800 number (or 900), "trained operators standing by", a way to escalate unresolved problems.

Ad-hoc collaboration can be an effective alternative. Usenet is the prototypical example. There are thousands of newsgroups, such as comp.sys.mac.programmer, where you can post questions and answers. There’s no charge (beyond the cost of getting on the network), but there’s no guarantees about the answers you get.

Ad-hoc collaboration raises several issues:

  • Stake. Without a formal mechanism, nobody has a direct financial stake in helping you. (They may do so out of goodness or a desire for reputation.)
  • Attention. There are thousands of newsgroups. Many newsgroups have hundreds of messages a day. Your question may have been answered already, but it can be hard to find that out. For better or worse, Usenet is a democratic medium, and your question or answer has no particular reason to stand out.
  • Authority and reputation. It can be hard to know who is answering your question, and to know if that answer is right. If you follow a group for a while, you will form opinions about who does or doesn’t know what they’re talking about. Reputation can become important.
  • Anonymity. The web allows anonymity. You may not want people to know who is asking a question. It can be hard to get this quality in formal support, where you may have to identify yourself just to get charged for the help.
  • Buzz/Community. Ad-hoc groups allow a product to generate "buzz" – the sense that the users are a group of real people who are going somewhere great. You can get each other excited, or bring each other down.
  • Serendipity. A side-effect of the volume of newsgroups is that you have chances for serendipity. You’ll often run across something to help you with a problem you know you’ll face but haven’t yet addressed.

Can we combine planned and ad-hoc collaboration? Yes – many companies do this. They either task people, or employees do it out of zeal, to follow the newsgroups and help where they can. This brings several advantages – many people might see an answer in a group, and avoid calling in a problem.


[Written Jan. 20, 1998.]

Interface Seams: Sound and Computers

There are several approaches to computers and sound.

Direct Generation

The computer can be used to directly generate sound. One approach is to put a transistor radio next to the computer. The switching of the transistors causes sharp transitions, which generates radio waves. (This approach was used about 25 years ago; perhaps modern computers have too high a frequency.) Purportedly, you could detect loop patterns by listening.

Another approach was to "program" the line printer. By striking different patterns of letters, you could get different (if tinny) pitches. By judicious printing, you can play tunes.

Computer Control

In the 1980s, many personal computers included "sound chips" that could be programmed. The computer would talk to it as a peripheral, set up parameters for pitch, amplitude, attack, decay, filtering, etc. This was particularly used for sound effects in games.

MIDI control is similar. The computer may act as a mediator between a MIDI keyboard and sound system, perhaps storing patches as well. (MIDI is a standard for synthesizer keyboards.) The computer may act as an intelligent intermediary, modifying what happens as well.

Digital Recording and Playback

You can put together an analog-to-digital converter with the computer storage, feeding a digital-to-analog converter. This technique was used for such things as talking alarm clocks, where a fairly small set of words could serve for everything.

Digital Generation and Manipulation

Once you’ve got the ability to play back a digital signal, you can let the computer generate the sounds. (A very early system, the MUSIC programs of the 1960s, used this approach.) It’s fairly expensive: for a full range of pitches, you must generate up to 40,000 16-bit stereo samples for each second of generated sound. The MUSIC system defined a paradigm of oscillators, filters, etc. hooked up in a virtual patch panel.

Sound manipulation can use this same mechanism. For example, you might get an echo by taking a sound, delaying it a fraction of a second, and mixing in the delayed sound at reduced volume.


Speech generation is not perfect by any stretch. There have been various sound models proposed and tried. Speech output requires addressing more than just "pure sound" – it must take pronunciation, dynamics, intonation, and other factors into account. (Speech recognition is in even worse shape – it’s still very expensive but improving each year.)

Interface Seams: Rethinking the Spreadsheet

Spreadsheets have traditionally been based on a rectangular grid. We’ll look into simplifying formulas, and then at data structures other than a matrix.

Spreadsheets often have columns specified by alphabetic labels, and numbered rows. A specific cell can be identified by its coordinates, e.g., B17. Similarly, rectangles can be specified by their corners, e.g., A2:D14. Formulas are stored in cells in the table. The benefit of an electronic spreadsheet is that formula cells are automatically updated when the values they depend on are changed. A downside of the "formula in cell" approach is that the formulas themselves are mostly invisible.

We’ll examine a number of alternative spreadsheet structures, with an eye toward keeping the formulas explicit and visible.

The simplest "spreadsheet" has no cells. The formula has no value to work with – so in effect it’s a constant.


The next simplest spreadsheet has one data value and one formula (which takes a single argument).

data value

A list of items provides an interesting level of complexity:

formula over list

Commutative operators are particular useful: if
            (a op b) op c = a op (b op c)
then we can drop the parentheses and treat the operator as an n-ary operator. Such operators include addition and multiplication, but also "average". (This is reminiscent of the "reduce" operator in APL: take a list and an operator, and the formula is formed by inserting the operator between each value.) Our formulas can be very compact if they’re just describing one of these expressions. For example, "+" may describe adding all values in a list.

Trees provide a way to structure formulas explicitly. The interior cells provide a place to "hang" the operators. We can treat each node as having a list of children (thus reducing it to the previous "list" example).

For example, we may want a sum of products:

  / | \  
1 2
5 6
3 2

Notice how the leaves of the tree are the data values, and the interior nodes have operators. The value at the interior node is the result of applying the operator to the list of children.

If we’re willing to restrict our formulas somewhat, we can require that each node at a given level have the same operator. (The example tree has this property.) We can have a very compact formula that describes each level. For our example, "+*" describes a sum of products.

What can we bring back to the rectangular spreadsheet? The idea of a visible formula, working across a row or down a column.

  Test Test 2 Score  
Obs 1 1 2 5 10 * 27
Obs 2 3 2 7 12 +
Obs 3 -1 0 6   5 +

(We’ve set it up so the formula cells contain the resulting value, the operator that applies, and an optional label.) By letting operators "stretch" across the cells to which they apply, we’ve removed the need for specifying cell and rectangle coordinates.

In summary, we’ve used a variety of alternative spreadsheets to explore some variations, and brought a new formula style to the rectangular spreadsheet. The new style retains the table structure and dynamic updating that makes spreadsheets so responsive, but also separates the data from the formulas in a way that will make their hidden contents more explicit.

Interface Seams: Paper and Computers

Paper is the most important non-electronic artifact associated with computers.

Paper as Medium

Paper was second only to console lights as the original output medium. At one point, there was talk of the "paperless office". Actually, paper use is up as everybody gets a computer and printer.

Paper Prototypes

Why build a sample interface using code? Code requires a programmer and a lot of work. Instead, let users play with colored paper, scissors, and tape. It’s fun, and it puts users on an equivalent level with developers. [ref Rettig]

Paper as Metaphor

The idea of the computer screen working as a virtual sheet of paper has inspired much of the progress in personal computer user interface design. Terms such as "cut-and-paste" reveal the metaphor. The notion of WYSIWYG ("What You See Is What You Get") has pushed both screen and printer design.

Paper as Book as Metaphor

Books represent a model for efficient use of information. They have inspired hypertext [ref Superbook] and library systems [ref das bookhaus]. The written word is powerful – it can be skimmed or studied as necessary. We use tools such as a table of contents, an index, or footnotes, that can be carried into the computer domain.

Paper as Input

Paper can be used as an input medium. While OCR (optical character recognition) is not perfect, it is effective enough to be used for things such as resume banks and text searching [ref O’Gorman]. [?? CHI 95??] describes a system that keeps track of what it prints, so a scanned document that was printed can be directly recognized.

Paper as Ubiquity

The goal of ubiquitous computing is to build a world of smart devices that provide "embodied virtuality" [ref Kay]. Paper is not smart, but it is ubiquitous. We let it fade into the background, but as I look around my dining room I see books, receipts, art, boxes, address labels, magazines, newspapers, and warning labels. Paper is symbol-oriented, and omni-present.

Why Paper?

Paper has admirable properties:

  • Inexpensive
  • Manipulable
  • Persistent. Compare to the spoken word which has only been captured as sound for a century.
  • Symbol-oriented. Paper lets us record words and pictures. Many artifacts "just are" – their reason for their present form is not obvious. Symbols let us have artifacts that try to explain themselves.
  • Easy-to-use. Learning time is long, but once reading is learned, paper is very effective.
  • Fast. Reading is much faster than speech.

Interface Seams: HCI Resources

There are many resources for people interested in human-computer interaction (HCI). This month, I'll review some of my favorites.

On the Web

HCIBIB. A large bibliography of HCI-related work.

Interactions. ACM's glossy HCI magazine.

ACM SIGCHI. SIGCHI's home page, full of conference proceedings and lots of other good information.


Programming as if People Mattered. Nathaniel Borenstein. Princeton University Press, 1991.
An HCI book that shows awareness of the problems of programming.

Understanding Computers and Cognition. Terry Winograd and Fernando Flores. Ablex, 1986.
This book turned me around on issues of Computer-Supported Cooperative Work (CSCW).

Computers as Theatre. Brenda Laurel. Addison-Wesley, 1991.
This is not a highly prescriptive book, but it does push awareness of computer use as a dramatic activity.

Designing the User Interface: Strategies for Effective Human-Computer Interaction. B. Shneiderman. Addison-Wesley, 1992.
A generic HCI textbook. You can do a lot worse.


Douglas Engelbart. He invented practically everything in collaborative software.

Alan Kay. Inventor of SmallTalk, now pushing ubiquitous computing.

Jakob Nielsen. Usability engineering, hypertext. Publishes Alertbox, which is a definite inspiration for my pages.

Ivan Sutherland. He invented practically everything graphical with computers.


Virginia Tech. (Alma Mater.)

Xerox Parc. Especially their Digital Library work.

Interface Seams: Notes on Guindon – "Designing the Design Process"

Notes on Guindon – "Designing the Design Process"

Reviewer: William C. Wake, 9-16-94. (Published on xp123.com April, 1999.)

"Designing the Design Process: Exploiting Opportunistic Thoughts", by Raymonde Guindon. Human-Computer Interaction, 1990, V5, pp. 305-344.

From Guindon’s abstract:

"This study shows that top-down decomposition is problematic in the early stages of design. Instead, an opportunistic decomposition is better suited to handle the ill-structuredness of design problems. …"


  • Top-down decomposition is problematic…
  • Opportunistic decomposition is better suited…
  • … interleaving decisions at various levels…
  • Verbal protocols…
  • … causes of opportunistic design
  • A top-down decomposition… a special case
  • Two cognitive models
  • Implications

This Study Shows…

… the design process frequently deviates from a top-down approach. But more importantly, it shows that these deviations are not noise or special cases resulting from bad design habits or performance breakdowns. Rather, they are a natural consequence of the ill-structuredness of problems in the early stages of design.

Deviations Occur When…

  • Artifact is new to designer
  • Integration of multiple knowledge sources
  • Subproblems appeared
    • critical,
    • very different, or
    • had an immediately known solution

Early Design

Specification of Requirements

  • Informal
  • Incomplete
  • Ambiguous
==> transforms to ==> High-level design

  • Notation
  • Refined requirements
  • Software subsystems


Design Problems are Ill-Structured

Simon (1973)

  • Incomplete and ambigous specification of goals
  • No pre-determined solution path
  • The need for integration of multiple knowledge domains

Evidence for Prescriptive Design Models

  • Jeffries et al ’81: Two novices, four experts. Showed some deviations form top-down design.
  • Adelson and Soloway, ’84, ’85: Two novices, three experts. Expert designs were systematic and unbalanced. Designs are unbalanced when a mental model exists.
  • Kant and Newell ’84: Two PhD students. Problem-solving and refinement.
  • Parnas and Clements, Mills, Dijkstra, …

Design Decomposition is Opportunistic

  • Data-driven rules (not goal-driven)
  • Opportunistic planning
    • "Blackboard architecture"
    • Take immediate advantage of discoveries
  • Occasional performance breakdowns (e.g., memory limitations)

Categorization Rules

  Unbalanced Balanced
Solution Development
  • Drifting
  • Immediate recognitions of solution in other part of problem
  • Simulating scenarios
  • Immediate solution for inferred requirement
  • Design schema
  • Design method or notation
  • Solution schema
  • Design heuristics
Solution Evaluation If solution unbalanced
  • Test cases
  • Systematic requirements review
  • If solution balanced
Requirements Inferences and additions
  • Systematic strategy
  • Simulating scenarios


The Lift Problem

N elevators for M floors

"Ecologically valid"


  • 8 protocols ==> 3 in depth ==> 2 reported
  • "Prototypical" style
  • Styles guided by
    • Software design method,
    • Past experience, or
    • Programming paradigm


  • Videotape and transcript reviewed by 4 researchers: brainstorming
  • Prompted review session with participant
  • Iterative development of analysis scheme (Templates: activity and level)
  • In-depth analysis

Causes of Opportunistic Design Decomposition

  • Sudden discovery of unbalanced partial solutions
    • Partial solution from rest of problem
    • Simulation bug
    • Low-level solution before decomposition
    • Data-triggered rules
  • Immediate solution development for new requirements (60% of new requirements solved immediately)
  • Drifting
    • Follow train of thought; little cognitive cost
    • Partial solutions may provide critical insights
  • Solution development by problem domain scenarios: Triggered recognition of unbalanced solution or new requirements

Differences between Designers

  • Specialized design schema
  • Designer 2 schema allowed straightforward decomposition
  • Frequent and varied deviations

Different Psychological Models (Caricatured)

  • Anderson: top-down design process with hierarchical goal structures
  • Hayes-Roth: Flexible and easily re-organizable goal structures and online planning.
  • "…very difficult to demonstrate empirically the validity of one psychological model against another."
  • Could embody both, and compare.

Implications for Training, Methods, and Environments

  • "…until the proper design decomposition… the design process should be opportunistic"
  • Don’t force a strict order of activites
  • Allow rapid shifts between tools for objects and their representations
  • Allow easy navigation between objects, but support an agenda

More Implications

  • Representations should have smooth progression from informal to formal
  • Easy editing and reorganization
  • Requirements traceability
  • Representation of interim and partial design objects


  • 4 studies, 15 subjects
  • Strict top-down decomposition is a strawman
  • "Drifting" == end-to-end tracing
  • Hidden agenda for environments (?)
  • Environment sounds like Fischer’s CPS
  • Wrong question?

Interface Seams: Logarithmicons

Icons to represent sets with highly variable sizes can come from an icon family based on powers.

At Virginia Tech, in conjunction with the Envision Project, we are building a large object-oriented database of computer science literature, with a graphical front end. One of our concerns is visually representing sets of retrieved documents. They can vary greatly in size; a query for "computers" might find 100,000 documents, but a search for "logarithmicon" might only find one.

We are considering using icons to represent these sets, with features of the icon suggesting the number of documents. We are creating icon families [Marcus 84], letting area covered by the icon suggest the size of the set.

The key to these icon designs is the mapping of the logarithm of set size to an icon's attribute. This is because:

  • there is great variability in cluster size,
  • most clusters are somewhat small,
  • a linear mapping would make it difficult to distinguish small from medium-sized clusters.

The icon designs (in Figure 1) represent clusters of various sizes. Icons share these features:

  • They have a border (in one case a circle, in the other a triangle) that provides an indication of maximum icon size.
  • They map the logarithm of set size to a linear feature, and use it to increase the area covered. A linear increase in radius or height increases area covered quadratically, which hints at the exponential increase involved.
Figure 1. Two families of logarithmicons.

The "rungs" in the second (triangle-based) family show discrete steps, for a clear visual depiction of the various orders of magnitude. It might be better if there were an indication of the number of steps available, but there is only so much we can show in a 32×32 pixel icon.

In summary, the abstract icons presented here suggest a mapping that lets an icon feature represent widely varying sizes.

Postscript: The name "logarithmicon" is derived from merging the words "logarithmic" and "icon". A quick skim through the dictionary suggests other possibilities: algorithmicons (to do the work), bombasticons (for multimedia demos), iambicons (for Shakespeare), and telepathicons (the ultimate goal).


[Marcus 84] Aaron Marcus. "Corporate Identity for Iconic Interface Design: The Graphic Design Perspective," IEEE Computer Graphics and Applications, pp. 24-32, December, 1984.


I would like to acknowledge discussions with the Envision Project team, particularly Dr. Ed Fox. This work was done in 1992 while I was a grad student at Virginia Tech. I was partially supported by NSF grant 4-27343.

[Written Dec., 1992; published on xp123.com July, 1998.]

Interface Seams: Multimodal Icons

Multimodal icons reflect the attributes of objects.


This paper presents an icon design methodology that emphasizes the attributes of an object. Attributes may be reinforced along several dimensions of a design. As an example, we develop a series of icons for queries in an information retrieval system.

Related Work

Marcus [4] describes the use of a corporate look in developing icons for computer systems. He applies standard graphic design techniques to a set of icons in a graphical computer system. The result is a set of icons that retains a consistent look. Marcus applied his technique to the wide range of icons present on a system; this paper focuses on a closely related set of information-bearing icons.

Henry and Hudson [3] describe multidimensional icons. An icon is considered as a three dimensional cube; various faces of the cube can be exposed to provide information about a file.

Blattner et al. [1] describe how sounds can be attached to icons, and consider the various properties of sound and their consistent meaning in a sonic environment. They propose a sort of grammar of sound attributes, and attach them to actions on various types of objects.

Gaver [2] attached sounds to icons and events in a graphical user interface. He believes that his approach leads to an increase in direct engagement, and provides flexibility in providing information to the user. He suggests, but doesn't develop the idea, that visual and auditory effects can be designed to reinforce each other.

The present research is closest in spirit to Gaver's. Workstations have evolved to the point where designers can exploit a unified approach in visual, auditory, and kinesthetic design. As Gaver suggests, multiple sensory modes, reinforcing each other, should increase the sense of engagement and provide more information to users.


There is a range of variability possible within an icon. Table 1 identifies some of the possibilities. Some attributes are on a linear scale (such as sets); others are just sets of values (e.g., color). Some have obvious representations in current systems (such as shape); others do not (e.g., temperature).


Table 1. Attributes.
Visual Audio Kinesthetic
Size Pitch Weight
Visual Density Timbre Physical Density
Color/Shade Volume Temperature
Shape Rhythm  

A multimodal icon design will map the attributes of our information object onto the attributes of our icons:

  • Identify the attributes of interest in the abstract object.
  • Identify the variable attributes available for icon design.
  • Map the object attributes onto the icon attributes.

Important attributes of the information object should be mapped to multiple icon attributes, to reinforce the infomration provided. Linear information attributes (e.g., object size) should generally be mapped onto linear icon attributes (e.g., density), not non-linear ones (e.g., color).


As an example, we apply the approach outlined above to queries in an information retrieval system. In a typical system, many queries are made, and the user must keep track of the sets of documents returned by the system. To make these sets more directly manipulable, we might wish to provide icons for them.

For this example, we will consider only two attributes of our result sets: their size and the type of query that generated them. We might identify four dimensions of variability for our icons: density, sound, dragging speed, and color. Note that icon density and color are always apparent; the sound and dragging speed are known only when the pointing device is involved.

Since the size of the set and the visual density of the icon are both on a linear scale, we'll map set size to density. The sound and dragging speed will also be mapped from set size. Query types, a discrete set, will be mapped to color.

To complete the example, we must develop the actual icons. For the set sizes typically found in information systems, result sets range from one or two to thousands of documents. This suggests the possibility of using a logarithmic scale, and we will do just that. The visual density of the icon will be proportional to the logarithm of the set size. Triangular figures will be used to support this; the number of lines in the icon will be proportional to the log of the set size. (An earlier version used filled rectangles, which didn't hint at the logarithmic scaling involved.) Thus, we might have this sample of icons:

The sound associated with an icon will be heard when the mouse is clicked on it. Sounds will differ in their timbre: the first icon will have a noise with a lot of reverb and "rattle"; succeeding icons will have less reverb and sound more dense.

The dragging weight will come into play when the icon is moved. A normal mapping might be 1" of mouse movement causing 1" of screen movement. We will vary this from 1:1 for one item up to 2:1 for a million items. This will cause the larger sets to seem heavier.

Finally, we will map query types onto colors, e.g., blue for author queries, yellow for date queries, and green for keyword queries.

Since set size is represented in multiple sensory modes (visual, auditory, and kinesthetic), these attributes should reinforce each other in providing continuous feedback, more direct engagement, and better information (as Gaver proposes).

Future Work

This proposal has not been applied to a real system; this is the natural next step.

The techniques described should be applicable to many areas where icons are (or could be) information-bearing. For example, folders on the Mac could vary their shade depending on how many items are (recursively) in the folder.

This proposal is independent of multi-dimensional icons. Each face of a multi-dimensional icon has all the opportunity for variation possible for simple two-dimensional icons.

Virtual reality systems may have virtual objects corresponding to icons, and would provide an opportunity to map physical dimensions such as "squishyness" and temperature.


This paper has presented a modest methodology for icon design. It provides guidance in mapping information object attributes into the attributes of icons.

An example made this mapping more clear. It applied the mapping to a range of icons representing a single sort of object, rather than an assorted collection of icons. Mapping the same information across multiple sensory modes can provide reinforcement of information.


[1] Blattner, Meera M., Denise A. Sumikawa, and Robert M. Greenbertg. "Earcons and Icons: Their Structure and Common Design Principles," Human-Computer Interaction, v. 4, 1989, pp. 11-44.

[2] Gaver, William W. "The SonicFinder: An Interface That Uses Auditory Icons," Human-Computer Interaction, v. 4, 1989, pp. 67-94.

[3] Henry, Tyson R., and Scott E. Hudson. "Multidimensional Icons," ACM Transactions on Graphics, v. 9, no. 1, January, 1990, pp. 133-137.

[4] Marcus, Aaron. "Corporate Identity for Iconic Interface Design: The Graphic Design Perspective," IEEE Computer Graphics and Applications, December, 1984, pp. 24-32.


I would like to acknowledge discussions with the Envision Project team, particularly Dr. Ed Fox. This work was done in 1992 while I was a grad student at Virginia Tech. I was partially supported by NSF grant 4-27343.

[Originally written 2-20-1992. Modified 5-28-99 to include acknowledgments and to use the figure from "Logarithmicons" rather than re-generating the original figure which had the triangular icons only.]