Fit Spreadsheet – Output

Starting Fit

To use fit, you create a web page that has tables in it; the tables specify tests. (There are other options but that is easiest.) In this case, I’m using Microsoft Word™ and saving the file in HTML format.

The fit FileRunner acts as a filter: given a web page, it copies text outside of tables as is, and runs your program on the table entries. Some table entries represent tests that can pass or fail; fit colors them green or red respectively. The output is another HTML file.

Fit will also put a summary in the file if you put in a table like this:

fit.Summary

counts 0 right, 0 wrong, 0 ignored, 0 exceptions
input file C:P4FirstFitfitFirstFit-in.htm
input update Thu May 01 10:51:42 EDT 2003
output file C:P4FirstFitfitFirstFit-out.htm
run date Thu May 01 10:58:28 EDT 2003
run elapsed time 0:00.05

With this tool, you don’t manipulate screen elements directly. Instead, you work with an abstraction of them. To me, it feels like talking to somebody over the phone, trying to tell them how to use an application. (“In cell cee seventeen, put equals a one; then go to a one and type ‘fish’.”)

This article shows the input to [output from] fit; the result of running it is here [input is here].

Programming and Configuration Notes

Fit is a tool for customers and testers, but programmers will use it as well, and will have to write some of the fixtures the team uses. In this paper, I’ve tried to use the framework mostly straight out of the box.

The CLASSPATH needs to include fit.jar (both in the DOS window and the IDE). The runner command I’m using is:

java fit.FileRunner FirstFit-in.htm FirstFit-out.htm

When I do this on the file I have so far, it creates the output file and writes this to the console:

0 right, 0 wrong, 0 ignored, 0 exceptions

Fixtures

Tables in the input file have the name of a fixture in the first row. A fixture is a class that knows how to process the table. Fit comes with several fixtures built in, and programmers can create others.

One simple fixture is the ColumnFixture. In this fixture, the first row is the fixture name, and the second row has the names of data. If a name ends without parentheses, it is regarded as a field to fill in; with parentheses, it’s treated as a method (function) call. The fixture fills in all the data fields, and then calls the methods to verify that they return the expected results.

Another standard fixture is the ActionFixture. This one consists of a series of commands. These include:

  • start classname: Creates an object of the specified class
  • enter field value: Sets the field to the value
  • press button-name: Calls the method corresponding to the button
  • check method value: Checks that the method returns the expected value

The ActionFixture ignores anything past the first three columns; we’ll use the fourth column for comments.

So, we’re finally ready to start our application.

fit.ActionFixture

start

Spreadsheet

Create a new spreadsheet.

This test doesn’t ask for much, but of course it fails. (There isn’t any code yet!)

            0 right, 0 wrong, 0 ignored, 1 exceptions

Programmer Notes

The exception is thrown because the Spreadsheet object doesn’t exist. To create it as simply as possible, make it extend Fixture:

import fit.Fixture;

public class Spreadsheet extends Fixture {}

This gets us back to

            0 right, 0 wrong, 0 ignored, 0 exceptions

I’ve put together stubs for the fixtures used in this article: Spreadsheet.java, SpreadsheetFormula.java, and Address.java; here’s a zip file containing all three.

A Few Stories

We have several things we want our spreadsheet to do:

  • Track the contents of cells
  • Distinguish data from formulas
  • Provide both data and formula views of cells
  • Support “+” for appending strings, “’” for reversing strings, “()” for grouping, and “>” for string containment.

Cells

The spreadsheet has a number of cells, each of which has an address. Cells contain string data or formulas.

We’ll assume several screen elements:

  • address – the address we’re working with; something like “B19”
  • cell – the cell contents we enter (to the last “address”)
  • formula – the cell contents as entered (for the last “address”)
  • display – the cell contents as seen when the formulas are applied (for the last “address”)

We’ll start with a simple data cell.

fit.ActionFixture

Comments

start

Spreadsheet

enter

a1

abc

check

a1

abc

Text in cell

check

formula

abc

Formula is same. (Looks in last-mentioned cell.)

Now let’s add in a formula cell. (Note that this table omits the “start” line; this means it’s working on the same object as before. This lets us not repeat the setup, but it also makes the tests less independent.)

fit.ActionFixture

Comments

enter

a1

abc

enter

b1

=A1

Simple copying formula

check

formula

=A1

Formula is there

check

a1

abc

Original text in A1

check

b1

abc

Text was copied to B1

The essence of a spreadsheet is the automatic updates. Let’s change A1 and see it happen.

fit.ActionFixture

Comments

enter

a1

abc

enter

b1

=A1

Simple copying formula

check

b1

abc

Copied value

enter

a1

revised

Update A1

check

b1

revised

Automatically updates B1

We already have quite a few elements in use, though we haven’t specified exactly what is valid. Let’s just note the “specification debt” and move on.

  • What can a cell hold? Empty string, other string, formula starts with “=”
  • What’s a valid address? Letter plus digits; ignore leading 0s; case-insensitive.
  • What’s a valid formula? So far, we’ve just used a simple cell reference, but we want operators too.
  • What happens when a cell has an invalid formula?
  • What happens when a cell refers to a cell containing a formula?
  • What happens when formulas form a loop?

We’ll pursue all these, but let’s start with formulas.

Formulas

Formulas can reference formulas.

SpreadsheetFormula

a1

b1

c1

d1

a1()

b1()

c1()

d1()

data

=A1

=B1

=C1

data

data

data

data

Formulas get more interesting when there are operators available. The reverse operator (‘) is probably a good one to start with.

SpreadsheetFormula

a1

b1

b1()

abc

=A1′

cba

abc

=A1””

abc

The most useful string operator is probably append (+):

SpreadsheetFormula

a1

b1

c1

b1()

c1()

abc

=A1+A1

blank

abcabc

blank

abc

def

=A1+B1+B1+A1

def

abcdefdefabc

We have enough features that we can demonstrate an identity: (XY)’=Y’X’. We don’t have parentheses yet, but we can simulate this by putting the parts in separate cells.

SpreadsheetFormula

a1

b1

c1

d1

e1

d1()

e1()

abc

xyz

ignored

=A1+B1

=D1′

abcxyz

zyxcba

abc

xyz

=B1′

=A1′

=C1+D1

cba

zyxcba

Parentheses can be used to group operators. Let’s re-do the previous test, allowing parentheses:

SpreadsheetFormula

a1

b1

c1

c1()

abc

xyz

=(A1+B1)’

zyxcba

abc

xyz

=B1’+A1′

zyxcba

 The operator “>” tells whether one string contains another one. If the first string contains the second, the result is the second. If the first string doesn’t contain the second, the result is an empty string.

SpreadsheetFormula

a1

b1

c1

c1()

banana

ana

=A1>B1

 ana

banana

bab

=A1>B1

 

blank

We haven’t talked about precedence yet. The ‘ and () operators have the highest precedence, then +, then >. A1+B1+C1 is a legal expression, but A1>B1>C1 is not.

SpreadsheetFormula

a1

b1

c1

c1()

abc

xyz

=A1+B1′

abczyx

abc

xyz

=(A1+B1)’

zyxcba

SpreadsheetFormula

a1

b1

c1

d1

e1

e1()

abcdef

ghijkl

e

hgf

=A1+B1>C1+D1′

efgh

Backfill

We have several questions left open:

  • What can a cell hold? Empty string, other string, formula starts with “=”
  • What’s a valid address? Letter plus digits; ignore leading 0s; case-insensitive.
  • What happens when a cell has an invalid formula?
  • What happens when formulas form a loop?

The previous tests made a quick pass through the system. I think of them as generative: they help define the essence of the system. But questions like the above require us to fill in the gaps. I think of tests that do things like check “corner cases,” error cases, and how features interact as elaborative; they fill in what we already have. They might find problems, but they may well work already, depending on how the system was built.

What a cell holds

We already have test cases where a cell holds a string, and where a cell holds a formula, but it would be prudent to check that the operators work correctly on empty strings. If e is the empty string and x is a non-empty string, we expect:

            e’ = e

            e+e=e

            e+x=x

            x+e=x

            e>e=e

            e>x=e

            x>e=e

As I go to write the test, I realize that we never specified what a cell starts with. The answer, of course, is the empty string. So we’ll rely on that: A1 will be empty.

fit.ActionFixture

Comments

start

Spreadsheet

check

a1

Verify that cell starts empty.

Then we can verify those rules about working with the empty string:

SpreadsheetFormula

a1

b1

c1

c1()

Comment

blank

blank

=A1′

blank

e’=e

blank

blank

=A1+A1

blank

e+e=e

blank

blank

=A1>A1

blank

e>e=e

blank

abc

=A1+B1

abc

e+x=x

blank

abc

=B1+A1

abc

x+e=x

blank

abc

=A1>B1

blank

e>x=e

blank

abc

=B1>A1

blank

x>e=e

Valid Addresses

There are two places we use addresses: in the address field and in the cells with formulas. When we get a “real” (graphical) interface, the address will mostly be implicit. But even so, we’ll test it here just to be safe.

Let’s use a ColumnFixture for this: we’ll put address in one column, valid() in another, and standardized() in another. (A programmer will have to write the new fixture for us.)

The rules are: a valid address is a letter (A-Z, a-z) followed by one or more digits (0-9). Case is ignored. Leading 0s are ignored. “0” is not a valid row number.

Address

address

valid()

standardized()

A1

true

A1

a1

true

A1

A9874

true

A9874

Z1

true

Z1

z1

true

Z1

Z3992

true

Z3992

z3992

true

Z3992

AA393

false

zX202

false

é17

false

1

false

~1

false

~D1

false

y&1

false

^

false

X392%

false

H001

true

H1

j00010

true

J10

e000

false

A0

false

z0

false

Let’s make sure that case-insensitivity works in formulas:

SpreadsheetFormula

a1

b1

b1()

abc

=A1+a1

abcabc

Formula Errors

If a formula contains an error, we’d like it to display as “#error.” We’ll put all the invalid names from the previous table into formulas, and verify that formulas behave correctly. Then we’ll try various improper combinations of operators.

fit.ActionFixture

start

Spreadsheet

Create a new spreadsheet.

enter

a1

=AA393

Bad address

check

a1

#error

Marked as error

check

formula

=AA393

Formula as written

enter

a1

=A2

Change to valid address

check

a1

Make sure #error is cleared 

SpreadsheetFormula

a1

a1()

Comment

=zX202

#error

Two letters

=é17

#error

expected


? actual

Non-ASCII

=1

#error

No letters

=~1

#error

No letters

=~D1

#error

Unacceptable character

=y&1

#error

Extra character

=^

#error

No letters/digits

=e000

#error

expected


? actual

Too many digits

=A0

#error

expected


? actual

Invalid row #

=z0

#error

expected


? actual

Invalid row #

=

#error

Missing formula

Then we’ll get to some operators:

SpreadsheetFormula

a1

a1()

Comment

=’A2

#error

 should be postfix

=’A2′

#error

Can’t be before and after

=A2+

#error

Need other term

=A3+A4+

#error

Need other term

=A2++A3

#error

Missing term

=A2+’+A3

#error

‘ isn’t a term

=A2”’+A3

blank

OK to mix things

=A2)

#error

Missing (

=(A2

#error

Missing (

=((((((((((((A2))))))))))))

blank

OK – big expression

=((((((A2+(A3))))+A4)

#error

Unbalanced – too few )

=(((A2>A3

#error

Unbalanced – too few )

=(A2>A3)))

#error

Unbalanced – too many )

=A2>A3>

#error

Can’t trail >

=A2>A3>A4

#error

Can’t repeat >

Loops

If a formula uses itself (directly or indirectly), we don’t want it to loop forever trying to figure it out. Instead, we’d like the display to be “#loop.”

SpreadsheetFormula

a1

b1

c1

d1

e1

a1()

e1()

=A1

blank

blank

blank

blank

#loop

blank

=B1

=C1

=F1+D1

=E1

no-loop

no-loop

no-loop

=B1

=C1

=F1+D1

=E1

=A1

#loop

#loop

Conclusions

This paper has demonstrated a set of tests using the fit acceptance testing framework. Some things to note:

  • The tests here have been written as if a customer specified them, without much demonstration of the programming cycle. But programmers can work with these tests in much the way they would with JUnit.
  • The tests are written without benefit of the feedback of a working system. (I wrote just enough code to make the first test not throw an exception.)
  • The tests look at only part of the system: the core functionality. There are other aspects of a real application that we aren’t testing. (For example, it may be non-trivial to connect a screen to the core code.)
  • Even a small application such as this requires a fairly large set of tests. With more programming work on the fixtures, we might be able to reduce some of the noise. Real applications will organize tests into multiple files, and will have to pay more attention to the challenges of consistency, test independence, and feature interaction.
  • It feels smooth to mix light natural-language specification with formal, executable tests.
  • Fit has a number of features we haven’t used.

I’ve heard that many teams use xUnit for unit testing, but still struggle to get customer tests before or even after stories are implemented. I hope frameworks such as fit can help lower the barriers to doing this crucial task.

fit.Summary

counts 94 right, 4 wrong, 0 ignored, 0 exceptions
input file C:P4FirstFitfitFirstFit-in.htm
input update Thu May 01 10:51:42 EDT 2003
output file C:P4FirstFitfitFirstFit-out.htm
run date Thu May 01 10:58:28 EDT 2003
run elapsed time 0:00.14

Resources and Related Articles

 

[Written April 20, 2003; revised April 26, 2003, to correct mis-stated identity & in response to Ward Cunningham’s great suggestions about improving the fixtures; 2012 – the WordPress version is designed to simulate the original look.]