Willow: Experimental Economics with Python

You are reading the manual for Willow, a Python framework for experimental economics developed at the Center for the Study of Neuroeconomics at George Mason University. This manual presumes that you already have an understanding of experimental economics, of HTML, and of Python programming.

If you are interested in learning about experimental economics, you could do worse than to have a look at the following textbooks:

In order to get started with Python if you already know how to program in some other language, you can try

If you want to learn both the craft of computer programming and the Python programming language at the same time, you are in luck, since Python is an excellent first language. Some free resources include:

Also, in order to use Willow, you need to have basic knowledge of HTML and CSS, the languages used to build web pages. For this I strongly recommend

O’Reilly books are available full-text through an online service called Safari, which many universities subscribe to. In particular, if you are at GMU, you can get Safari at http://proquest.safaribooksonline.com.mutex.gmu.edu/?uicode=viva and you can this book at http://proquest.safaribooksonline.com.mutex.gmu.edu/0596527322 . If you are at some other institution, try your library web site listing of online databases.

Willow is a tool for making the kinds of computer user interfaces typically used in experimental economics. Without Willow, you might design such an interface from scratch, or by modifying earlier from-scratch software written for similar experiments. Using Willow, you can do the same thing, but faster and better.

Willow sets out to accomplish this goal not by providing canned routines for commonly performed experiments. After all, it is precisely because our experiments are different from what has gone before that we bother with them. Rather, Willow is a generic toolkit that makes a lot of the things we do in experimental economics easy.

To use a Willow program, you need only install your program on a single "monitor" computer. The computers used by your research subjects will simply run a web browser, which connects to a web server that runs on the monitor computer. The user interface for the subjects will be displayed inside the browser window.

Willow is a versatile program, and you can use it for many different interfaces; but you might just be curious what a Willow interface can look like. Here are some examples taken from the first study that used Willow at the Center for the Study of Neuroeconomics and the Interdisciplinary Center for Economic Science at George Mason University.

In this manual, I will use fixed-width font to indicate anything that is literally part of some HTML, CSS or Python code. For longer snippets of code, I will use boxes, like so:

Note that there are colors in these boxes, which are merely a typesetting convention to make it easier to read the code. Similarly, you may find that many of the better programmer’s text editors will color your code in a similar (but not identical) way. Again, the colors are not part of the code, and they are not actually stored in your code files, as you can easily verify by opening them up in a less fancy text editor.

Willow relies on HTML, the hypertext markup language, for the purpose of constructing a user interface. A complete description of HTML is outside the scope of this manual, but I’ll give a brief introduction.

An HTML document is a text file with some special markup sequences. To practice with HTML, you can construct HTML documents with a text editor, and load them in a web browser.

Consider this example:

If we type this into a file called example.html and load that file in a web browser, this is what we see:

Even though this is a simple example, it already illustrates some of the most important points about HTML:

Next, we will see a list of common HTML elements.

The last three of these fit together like so:

Some of the more common HTML attributes:

You can learn more about HTML by using either Firefox with the Firebug plugin, or Google Chrome. Right-click anywhere on the page and select "Inspect element." This pulls up an HTML explorer system, which looks like this in Google Chrome (Firefox with the Firebug plugin is similar):

By playing around with this feature of your browser, you can learn a lot of about HTML and CSS.

TBA

You should already know Python before reading this manual, but there are few Python features that you may not be familiar with even if you do have experience with Python and that will come in useful for Willow.

Python has a convenient way of putting together strings with the % operator.

If you run that through Python, you will see

As you can see, the % operator takes a string, called the "format string", on its left hand side, and a tuple on its right hand side. For every "gap" in the format string, indicated by %s, the corresponding element from the tuple will be substituted.

In fact, there are a number of gap markers other than %s (such as %d, %r, etc.) which are described in the "String Formatting Operations" section in the "Built-in Types" chapter of the Python library documentation. In practice, %s is nearly always what you want. If you happen to actually want a percent sign in your string, you should use %%.

Also, if you only have one value to substitute, you don’t need to put it in a one-element tuple; the % operator is nice enough to accept interpret non-tuple values as though they were one-element tuples.

To install Willow, you extract the zip file to a convenient location. You will do this anew for each of your Willow projects. In addition, you will need to have Python installed. Willow has been tested using Python 2.6 on Ubuntu GNU/Linux. You should be able to use it with Python 2.6 on Windows and Mac OSX as well. If you do not have Python 2.6 on your computer, get it from http://www.python.org/download/releases/2.6.3/ and then come back. If you are not sure whether you have Python 2.6 on your computer, just move on to the next section of the manual. If things don’t work, you’ll notice soon enough, and you can go back and install Python. Do not use Python version 3. It will not work.

While programming, you will want to run Willow from within a programmer’s text editor (or, if you care for such things, an Integrated Development Environment). On Mac and Linux, you can use the IDLE editor that ships with Python. If you are running Windows, the version of IDLE that ships with Python on that platform is very buggy, and I recommend strongly that you get a different text editor: in particular, I recommend SciTE, which is a very simple drop-in replacement for IDLE. On Mac or Linux, you can get away with just using IDLE, although you may find that SciTE is a little more polished. Either way, SciTE can be had from http://www.scintilla.org/SciTE.html .

In the Willow folder, you will find a file named lesson.01. You need to run this file: double click on it, or run the command python lesson01.py from the command line. Once the lesson01.py program is running, fire up a web browser and point it to http://localhost:8000/. You should see a page that looks like so:

Now pull up the lesson01.py file in IDLE or SciTE.

In IDLE, this looks like so:

In Scite, it looks like:

See here the code of a very simple but working Willow program. You can run it using the Run > Run Module menu option in IDLE or the Tools > Go menu in SciTE, or by pressing F5 (that shortcut key is the same in both SciTE and IDLE).

Again, you now need to point a web browser to http://localhost:8000.

If you see an error message like "address already in use," you are probably trying to run two instances of Willow simultaneously. You should stop the first one first (just close the window).

Now, let us look at the above code. Most of it is boilerplate, but the good news is that this is all the boilerplate you need. The first line imports the willow library. It assumes that willow.py is in a subdirectory willow of the directory in which lesson01.py is; this is how things are set up for the sample programs, and if you put your own programs in the same place, it will just work. The next two lines define a session function, and the last line instructs Willow to start the web server, using the session function that has just been defined.

Together, these four lines define a web server that you can connect to from a browser. If the browser is running on the same machine as the server, you can use http://localhost:8000 as a URL. If it is running somewhere else, you must find out the IP address of the server. (If you don’t know it, there is a whole chapter further down in the manual about how to figure it out.) If the IP address is, say, 123.123.123.123, then you can reach your Willow server using http://123.123.123.123:8000. The :8000 part is called the port number; you can specify a different port number as an extra argument to run (e.g. run(session, 8001)) if you insist on a port different from 8000.

Once the web server is running, for each client that connects to it, the session function will be called once. It will keep running until it returns. We say that each session function runs in its own "session thread". This means that several "copies" of the session function are running simultaneously, one for each client.

In this case, we want to simply display the text "Hello world." in the web browser of every client that connects, so we have a very simple session function that does nothing but add("<h1>Hello, world.</h1>"). The add() function adds an HTML snippet to the web page being displayed in the client’s browser.

Try connecting to localhost:8000 in multiple browser tabs. You’ll see that each one shows "Hello, world." This is because the session() function gets called once for each client that connects.

In real applications, we usually do not want to do the same for every client that connects. More likely, we’ll want one client to show some sort of experimenter console, one client to show the interface for subject 1, one client the interface for subject 2, etc.

Have a look at lesson02.py:

As you can see, there is still only one session() function, and for each client that connects, a session thread is started that runs the session() function. Each client runs the exact same function, but… the session function takes an argument, which we call me, and for each client, it gets called with a different value for me: 0, 1, 2, and so on. If you reload the page a bunch of times, it looks like this:

In this lesson, we will see some of the Willow functions for manipulating user interface elements. All of these are based on HTML and CSS, and it is crucial that you acquire a basic understanding of these languages. An extensive description of HTML and CSS is outside the scope of this manual.

Have a look at lesson03.py.

This displays a web page like so:

This shows off all of Willow’s UI manipulation functions. Let’s go through it line by line.

First we add a style sheet to the page. Again, CSS is beyond the scope of this manual and you should go out and read up on it, but what this comes down to is that we instruct the browser to typeset in bold all HTML elements that have class "important" turned on.

Note that in Python string literal juxtaposition adds the string literals together, which makes for a convenient way of splitting long strings over several lines.

The <span> element in HTML has no default meaning, and without any style sheets or other manipulations, a piece of HTML enclosed in a <span> is exactly identical to the same piece of HTML not so enclosed. In the above fragment, we have two <span> elementss. Note that because the style sheet instructed the browser to print all elements of class important in bold, the word "division" is printed in bold; or at least, it would be, if we did not manipulate it further.

This shows that you can add text not just to the page as a whole, but also to particular elements. The optional second argument to add is a CSS selector. The most common CSS selectors are:

Note that some selectors can refer to more than one element, and in that case, your HTML snippet gets added to all matching elements.

In this case, what we are asking Willow to do is to find the first HTML element with id=a and add the text " and subdivision" to the end of it it.

The set() function is just like the add() function except that instead of adding HTML to the end of some element, it replaces the entire contents of the element. In particular, set("") clears the entire page!

So far, we have seen the functions add and set that manipulate the contents of HTML elements. But HTML elements have attributes as well as contents. For instance, if you write <a href="foo">…</a>, you have created an <a> element with contents "…" and with an href attribute with value foo. The poke() function serves to manipulate attributes. In this case, we set the href attribute of every single <a> element on the page to be http://ebay.com.

In HTML, a single element can have more than one class. You can write that as <p class=class1 class2>…</p>. The pop() and push() functions are there to remove a class from an element and to add a class to an element, respectively, without changing any other classes that the element might have.

The hide() and show() functions do just that: hide and show elements. Willow is set up so that any element with class hidden set is hidden by default and only shows up when you call show() on it.

So far, we have seen the Willow UI primitives: add, set, poke, push, pop, hide, and show. In combination with HTML and CSS, these primitives make a very powerful UI language.

Now that we have seen how to produce output on the client screens, you may be wondering how we can read input. But before we get to that, we will first discuss another important feature of Willow: its communication system.

Consider lesson04.py.

In this short program, we can see a typical use case for the important Willow functions put() and take(). The first client, for which me==0, acts as a control panel that displays information about other clients coming online. In order to convey that information, each client other than the first puts a dictionary {"tag":"HELLO", "number":me} on the Board. The first client retrieves these dictionaries and uses them to display information about logins.

After the monitor client and 3 other clients have connected, the monitor client display would look like this:

To fully understand this, we must understand the Board. The Board is the one and only data structure that is shared between the various session threads in Willow.

The board contains dictionaries. Dictionaries are sets of key-value pairs. To put my_dictionary on the board, you call put(my_dictionary). For instance, to put the empty dictionary {} on the board, you call put({}), or to put the very abridged Dutch-to-English dictionary {"rood":"red","nederlands":"dutch"} on the board, you call put({"rood":"red","nederlands":"dutch"}).

To retrieve my_dictionary from the board, you call take(my_dictionary). In practice, though, you typically don’t know yet what dictionary you want to retrieve. For that reason, take() allows you to leave out keys you aren’t sure of. In the extreme case, you leave out all keys and call take({}), which means "please retrieve any dictionary whatsoever from the board."

The take() function is blocking. This means that if there is no matching dictionary on the board, it will sit around waiting until one appears. This is what happens in the example code: the first thread (the one with me==0) sits around waiting for dictionaries of the form {"tag":"HELLO", …} to show up on the board. Every time one such dictionary shows up, the take() function returns it, the loop loops, and the take() function is called again, blocking until the next dictionary shows up.

When a dictionary is gotten with take(), it is removed from the board. Therefore, in the above code, you do not need to worry about looping around retrieving the same dictionary over and over again. If you want to retrieve a dictionary and leave it available (maybe for some other thread), you simply put it back on with put().

Another important trick in the use of take() is that you can specify more than one pattern. For instance, take({"tag":"HELLO"}, {"tag":"ERROR"}) blocks until either a dictionary with the key-value pair "tag":"HELLO" or a dictionary with the key-value pair "tag":"ERROR" shows up. take({"tag":"HELLO"}, {"foo":3}) blocks until either a dictionary with the key-value pair "tag":"HELLO" or a dictionary with the key-value pair "foo":3 shows up. And so on.

Finally, I will mention grab(). It is like take(), except that it does not block. If a matching dictionary is available, it will return that, but if there is none, it will simply return None (unlike take(), which will sit there twiddling its thumbs until a matching dictionary shows up).

Now that we have seen how to manipulate what appears on a client screen, we will turn our attention to the issue of input.

There are two basic ways of giving input to a Willow client: you can click things, and you can fill out text fields.

The lesson05.py code demonstrates both.

This should initially display a screen that looks like so:

You can then type something in the text box and click the button labeled "Submit", and the screen will look like:

What’s going on here?

First of all, HTML elements that look like

are buttons.

In HTML, each <input> element that is a button must have two attributes: type must be set to submit, to make sure we are displaying a button and not some other sort of input element, and value must be set to the text that is to be on the button (or, if it is left out, the text defaults to "Submit", or possibly something else if your browser is set up for a language other than English).

In Willow, your button must additionally have an id attribute, which is how we keep track of which buttons get clicked.

When a button gets clicked, a dictionary

is automatically posted onto the board, where

This is why, after we have displayed the button, we can just call take({"tag":"click", "id": "go", "client": me}), which will block until the button is clicked. Note that if you insist on blocking one client thread until a button in some other client thread has been clicked, you can do that, simply by replacing me in the above call to take() by some other integer (but it is not recommended).

To show a text field, we use a piece of HTML that looks like

and then we call peek("#name_of_text_field"), which returns the current content of that text field. You can also use peek() for checkboxes, radioboxes, <textarea> elements, and most other input-type elements that you can read about in HTML books. In the above case, we only start looking for what’s in the field after the user has pressed the button.

Many of the Willow functions, even though by default they operate on the client display associated with the current session thread, can also operate on a different client display if you pass the client=n optional parameter. This is never necessary, but it may be more convenient than passing messages through the board to the other session thread so that it can then manipulate its corresponding client display.

Take for instance the code in lesson 4:

We could rewrite that as follows:

Look at this:

It shows "Hello, " on the screen, and 2.3 seconds later, it adds " world."

That’s all there is to this one. Use the background(f,dt) function to call the function f in the background, after a delay in seconds of dt. Note that you can also use this (typically with dt=0) for any other background tasks, like dealing with a chatbox while the main task deals with the rest of the experiment.

This is how you cause your Willow program to produce a data file.

In this case, a data file was produced in the log subdirectory (i.e. folder) called 2010-02-23-22-00-05.csv. The name is chosen automatically based on the date and time (year-month-day-hour-minute-second).

The file contains:

This is a CSV file, which is a simple text-file based spreadsheet format. It is understood by Excel, OpenOffice, Stata, and other data analysis software. You can also just open it in a text editor.

Each row will automatically get a timestamp added to the front of it, which is a large integer representing the number of seconds that have passed since the "Unix epoch" on January 1, 1970, 0:00 UTC. This may seem like an odd choice of timestamp format, but the advantage of this type of timestamp is that it makes sense to subtract them!

Most of the time, you are going to want input from the mouse or the keyboard. Web browsers know how to talk to mice and keyboards, which makes it easy for Willow to get the input to you. Occasionally, you may have a need for more exotic input devices.

For instance, at GMU we use a device called the "Lumina Response Pad for MRI", manufactured by a company called Cedrus, which is a set of two boxes with two buttons each that you can hand to a subject inside an MRI machine. These boxes are hooked up to something called the "LSC-400 controller," which in turn hooks up to a PC using a serial (RS232) cable.

If you have an LSC-400, or any other device that talks to your PC using a serial cable, you may want to have a look at the file named willow/bb.py. This is a little stand-alone application that will read from a serial port and post each byte that comes in on that serial port onto the board.

To get this to work, start bb.py in a separate terminal, like so:

You can pass three optional arguments to bb.py, like so

The first argument is the port number of the serial port (default 0), the second is the host and port number where willow is running (default "localhost:8000") and the third is the tag to use when bb.py posts things to the board… but here we’re getting ahead of ourselves, so let’s first have a look at some bb.py output.

Suppose we start bb.py and we push a button on our input device that causes some input to be sent in on the serial cable and we see this:

This means that bb.py accurately received a byte from the serial port, to wit "3", but it could not forward it to Willow, because it did not find an instance of Willow running on localhost:8000. Let’s start a Willow program that knows how to take serial input, in a separate window:

The code for this is simple:

Now, we push some more buttons, and we see in the window where bb.py is running:

If we load a web browser and point it to localhost:8000, we now see input coming in from the serial device!

(If you happen to have an LSC-400, by the way, you’ll want to set the "mode" to "ASCII/MEDx" and the "speed" to "9600 baud". If you have some other input device, you may want to play around with Hyperterm on Windows or Cutecom on Linux to see how it works. If you can get it to work with Hyperterm or Cutecom but not with bb.py, you should read the documentation for the pyserial library and use that to hack on bb.py. Let me know!)

The selector argument can take any CSS selector. The CSS selector mini-language is rather rich, but the most typical cases are things like p (to get at all <p> elements), #a, (to get at the first element with id="a"), and .a (to get at all elements with class="a"). For more information, consult any book or web page on CSS. Whenever selector is omitted, the selector "body" is implied, which means you are referring to the entire body of the HTML document. Aside from CSS selectors, jQuery extensions are also allowed. The most useful is of the form selector:eq(n), which selects the n-th element that matches selector.

By specifying an integer, or a sequence of integers, for clients, you can operate directly on clients other than the one associated by default with the current session thread.

Whenever timestamp=True, you are requesting a tuple to be posted of the form {"tag":"timestamp","client":_n_,"time":_t_} that gives you a millisecond timestamp of when the action was executed by the client.

Whenever you specify the delay=dt option, the execution of that action is delayed by dt seconds (you can specify fractions of seconds, too). This is done on the client side with millisecond accuracy.

If you have lots of closely spaced delayed events, at some point the browser may start lagging; but in Google Chrome, I have gotten it up to a set() event every milisecond with only a very slight lag.

Set the content of the HTML element(s) referred to by selector on to be content. Whenever content is a file descriptor that can be read from, the contents of the file are automatically substituted.

Add content to the content of the HTML element(s) referred to by selector. Whenever content is a file descriptor that can be read from, the contents of the file are automatically substituted.

Change the value of attribute attribute to be value in HTML element(s) referred to by selector.

Returns the value of the HTML <input> or <textarea> element(s) referred to by selector.

Add the CSS style style to the HTML element(s) referred to by selector.

Remove the CSS style style from HTML element(s) referred to by selector.

Hide the HTML element(s) referred to by selector.

Show the HTML element(s) referred to by selector, if previously hidden.

Post the dictionary onto the communication board.

Block until a dictionary becomes present on the board that matches at least one of the queries, then remove and return it. A dictionary matches a query if it contains all the key-value pairs in the query (plus, optionally, some others).

If a dictionary is present on the board that matches at least one of the queries remove and return it; otherwise return None. A dictionary matches a query if it contains all the key-value pairs in the query (plus, optionally, some others).

This starts the Willow web server on the specified port (or port 8000 if you leave the second argument out). The web server will create a thread for each client that connects, and each thread will run the session function, passing as an argument the client ID.

Client IDs will be assigned sequentially, unless the URL is of the form http://whatever-host/n, where n is an integer, and n is not taken yet, in which case the client ID will be n.

If a client ID is requested that already exists, an attempt will be made to replay all messages previously sent to that client. This provides a limited amount of recovery from browser crashes. If the browser crashed while a peek or timestamp request was pending, this may not be sufficient. An attempt to fix that is in the works.

The web server will search for requested files in the current directory, among other places, so you can for instance include images in your pages.

Run the function fn in the background (as a separate thread), with an initial delay specified in seconds. Fractions of seconds are allowed.

Wait for delay seconds. Fractions of seconds are allowed.

Write (x,y,…) to the log file as a record, with the number of milliseconds since January 1, 1970 0:00 UTC prepended as the first column. A new log file is created in the log folder whenever the Willow library is loaded, and it bears a name based on the date and time when it was created. For instance, if Willow was invoked at 15:32:42 on February 2 of the year 2010, the file would be called log/2010-02-10-15-32-42.csv. This particular date-and-time format is used so that the files can be sorted by date/time. The log files are in CSV format and can be opened with a text editor or a spreadsheet program.

Some Willow features are accessed by means of CSS classes:

When a button with (whether <input type="submit"> or <button></button>) or any element with class="clickable" is clicked, a dictionary like is inserted into the board automatically behind the scenes like so:

where

When the user presses a key, a dictionary like is inserted into the board automatically behind the scenes like so:

where

This is an extension library. It is liable to change, much faster than Willow itself, and not be be as well documented and tested. But I don’t want to withhold it either. This documentation is generated directly from the branch.py source file.

This function is a replacement for run(). If clients is not specified, it will display buttons on client 0 as clients 1, 2, … log on, which show subsets of the clients that have logged on so far. You can click one of these buttons at any time, and at that point, the function session() will be run once for each client IN THE SUBSET, with its client ID as the first number and the list of all client IDs IN THE SUBSET as its second argument. Clients that have connected but that are not in the subset will get a big X on their screens. The second way of using assemble() is to specify an explicit argument for clients=, some list of integers that indicates which clients you want. Once those client numbers have logged on, the session function is started once for every such client, and again any clients not in the set get a big X. === Examples ===

There is much additional useful code in the example_*.py files, which illustrate various ways to use Willow.

You need the IP address of your monitor computer in order for your client computers to connect to it. It would be convenient if Willow could display the right IP address for you to use, but unfortunately, this is easier said than done, since a computer can have multiple IP addresses at the same time: IP version 4 and IP version 6, on the wireless network, on the ethernet network, on the "loopback" network, and so on.

On Linux on my computer, I use the ifconfig command at the command line. When I enter

the computer responds

There is much information here, but we only need a little. First, notice that this computer has three network interfaces, called eth1, eth2 and lo. lo is the loopback device; it does not correspond to any actual hardware, but it is rather a contraption that the operating system uses so that you can make connections to your own machine at address 127.0.0.1 even when the network is down. The other two network devices are actual network cards that are in the computer; as it happens, eth1 is a wired Ethernet network, and eth2 is a wireless 802.11g network. According to the information above, the computer has IP address 10.143.91.80 on eth2, and it has no IP address on eth1, which makes sense, because the wired connection was not plugged when I executed the command. In the output above you can also see IP v6 addresses (labeled inet6 addr), but those are not commonly used.

Press WINDOWS+R (where WINDOWS is the "Windows Key", the one with the flag) to pull up a "Run Program" dialog, then enter cmd to get a command prompt window. Now enter ipconfig /a. This should give you all the information you need.

In order to prevent subjects from fiddling around with the web browser used to display a user interface to them, you can use a "kiosk mode" web browser. While several web browsers have kiosk modes, I have found that Google Chrome is the most convenient. On Linux, you simply use the command line google-chrome --kiosk instead of just google-chrome, and you get a full-screen browser without any extraneous bits of user interface that subjects might accidentally click on.

The first thing that should strike you when looking at these files is that there’s not very much there. This is on purpose, and took a lot of work. It is my experience that the number of bugs per 100 lines of code is roughly constant, and that the best way of limiting the number of bugs is to limit the amount of code. (In the extreme case, of course that means that the only way to have no bugs is to write no code. This is not as vacuous an observation as it seems: it pays to think twice before deciding to turn something into a computer program in the first place.)