Gamelab Logo


Gamelab is an online game theory laboratory where participants assign mixed strategies to game bimatrices over a series of rounds, accumulating payoffs as the experiment progresses. The mission of gamelab is to allow for laboratory-style interaction without the need for a laboratory.

This is the user's manual for gamelab. (There's also a quickstart guide.) Its purpose is to introduce the system, then describe how to deploy it and how to run experiments.


Gamelab is an online laboratory for conducting game theory experiments. Experiments consist of an experimenter, an experiment configuration, participants, and a set of games. The web interface accomodates for desktop, mobile, and tablet-sized devices. You can even play experiments programatically over a JSON API.

An experimenter begins by configuring—among many other parameters—how rounds in the experiment progress and for how long the experiment will run. Once the experimenter deploys the experiment, participants can log in and play each game once per round. When the round advances, participants earn points by computing the payoff from their submitted strategies against the average mixture submited by participants playing the opponent player role. After the last round, participants' points award lottery tickets, and winners (if winners are part of the configuration) are drawn by selecting lottery tickets.

Gamelab has many configurable parameters, for example, round advancement based on number of participants who have played; computing or not computing lotteries; Amazon Mechanical Turk integration; captive laboratory-based participants who cannot log out or back in; and so on.


Experiment participants are identified by an e-mail address, an opaque identifier in captive configurations, or a worker identifier in Mechanical Turk configurations (this is effectively a captive scenario). An experiments can have as many participants as the server (or the experimenter!) can handle.

Participants are assigned by the experimenter, assign themselves in a captive configuration, or join from Mechanical Turk console. In the first case, participants are e-mailed a password and login credentials when the experimenter begins the experiment. (If no e-mail server has been configured, they are not e-mailed.) These participants can log in and log out over the course of the experiment. In the latter cases, participants are not e-mailed, and are assumed to be captive (i.e., they cannot log out or back in) for the duration of the experiment.

Participants may be added (or add themselves) before an experiment begins, so that participation is static; or be added (or add themselves) both before and after the experiment has begun. In this dynamic case, experimenters can limit the number of simultaneous participants, allowing for a constant flow of participants joining and leaving the system as the experiment progresses. Participants that have logged in but cannot yet play due to the concurrent-participant limit are placed in a lobby until they join the experiment. (The same lobby is also used for the Questionnaire.)

When the experiment begins or when participants otherwise join, participants are randomly assigned a player role (row or column) used for all games throughout the experiment. Experiments always have an equal assignment of row and column player participants unless the total number of participants is odd, in which case the extra participant is assigned a random role. If participants join after the experiment has started, they are assigned to a player role that balances role counts at the time of assignment.


The experimenter may stipulate that participants step through a questionnaire before being allowed to play. In this treatment, participants are initially dropped into a lobby where they must answer questions before beginning. (This lobby is also used to handle an overflow of participants as defined in the Participants section.)

The questionnaire is designed to ensure that participants know how to play with minimum bias toward a given configuration. Results are recorded (duration of answering, number of tries, etc.) so experimenters may correlate performance with outcome.

Experimenters may customise their questionnaire manually when creating their experiment—please contact us if you need guidance in doing so. You'll be able, in doing so, to stipulate any number of free-form questions to ask your participants before they can begin.


A game in gamelab is simply a payoff bi-matrix. Payoffs are rationally-valued numbers. (Note: negative participant tickets will cause the lottery computation to fail.) There is no limit to the number of strategies for this matrix; however, in practise, only so many columns fit neatly on a small screen. Games larger than four or five strategies wide will look funny. Each experiment can have as many games as desired.


Once the participant joins an experiment, participants can submit game-plays, one per game per round. A game-play consists of a strategy mixture: a set of probabilities assigned to each of the player role's actions. The submitted probabilities are non-negative and rationally-valued, and always sum to one. Participants submit these probabilites by clicking on rows in the game matrix (participants appear to play the row player regardless of their player role) and entering in a decimal or rational number corresponding to the action.

Participants can only play one game at a time, with each game played once per round. If they log out and log back in (an action available only to non-captive participants), then the last game to play is displayed, assuming the round has not advanced in the meantime. Game matrices have their rows and columns randomly shuffled between participants, making it slightly more difficult to compare actions. Moreover, the order in which games are displayed is randomised between participants.

At the conclusion of each round, participants who have submitted mixtures for all games have their payoffs computed as points. (Rounds with incomplete games yield zero points.) Points are computed by playing the participant's mixture against the average mixture of participants in the opposing player role. Only opposing role participants who have played all games are considered. The points are then computed from the game payoffs. All of these calculations are over rational numbers and are unaffected by rounding. If not enough participants have played in a given player role, no points are tallied for the round.

After the first round, participants can review the full history of play. This includes the history of their own strategy mixture, the average opponent role mixture, and the participant role's average mixture. Moreover, it includes the participant's accumulation of points, points per round, and the hypothetical points per round of playing pure strategies. Experimenters may also provide a seed history that is prepended to the actual history of play. This allows the results of prior experiments to be incorporated into the current one.


When a participant has exceeded her maximum allowed rounds, she is no longer able to submit plays to the experiment. When all rounds have concluded, the experiment halts. Participants may still log in and review the history, as well as see the total number of points awarded and their lottery ticket allotment. The lottery ticket allotment is computed by rounding up the participant's total payoffs. Note: Mechanical Turk players are not included in the lottery computation, nor are they shown any information pertaining to the lottery.

The experimenter may then trigger the lottery from the console. (Experiments may also run without a lottery.) This consists of seeding a deterministic random number generator with a given value. Each of these values is mapped into the lottery tickets to choose a winner. A participant is only chosen once for winning: if the random number re-selects a participant, a new number is drawn (rejection sampling).

Note: if any final allotments are less than zero, the lottery uses negative-valued semantics as follows. First, all allotments are offset by the minimum (negative) allotment. For example, if the smallest allotment is -11, all allotments are increased by 11. Then lottery is then computed from these non-negative adjustment allotments.

Upon triggering winner selection, winner participants are notified on their login screen, along with the lottery ticket causing the win. This is for transparency of the winner selection. Non-winning participants are also notified.


Gamelab uses a broad set of technologies to work. In this section, I briefly describe the specific technologies used. All of this is visible in the source code, but let this act as a primer. In general, these can be broken down into the user interface, which experimenters and participants use, and the server backend, which is accessed by the user interface.

User Interface

The user interface to gamelab is via a web browser. In general, the user interface involves an HTML5 web-page styled with CSS. Both HTML and CSS have been carefully validated using the W3C Markup Validation Service and CSS Validation Service, respectively. These web-page uses JavaScript to request data from the gamelab server via a series of JSON objects. The JavaScript used by gamelab is fairly recent, so it's unlikely that older browser will work properly. (It has, however, been checked over with JSHint and other related services.) Check that your browser supports ClassList, FormData, and of course XMLHttpRequest (AJAX).

Graphs play a significant role in gamelab. The interface includes a local copy of flotr2 for drawing line and bar graphs. These graphs will display properly on any modern web browser.

The interface is styled with the Font Awesome icon set. The path to the icon configuration may be provided during compilation, and defaults to the on-line repository.

A copy of HumanizeDuration.js is also bundled with the system for displaying readable time durations.

The user interface also makes use of cookies to store your session materials. Gamelab cookies themselves don't store any identifiable material: they store only the server session identifier and a random number to prevent sharing of session identifiers. The system will also operate with session information embedded in URLs, but this is only enabled for Mechanical Turk play.

Server Backend

The backend of gamelab is a BCHS system consisting of two compiled CGI programs: one for the experimenter, one for participants. These are small ISO C applications linking to a number of open source libraries. Consult the Deployment section for how to install these libraries.

Gamelab has been run on OpenBSD with httpd and the historical Apache, Mac OS X with Apache2, and Linux with Apache. System administrators are not encouraged to use Apache2 since it disallows long-running child processes of the CGI scripts. Such processes are used in several circumstances.

Number Format

Internally, gamelab stores all non-integer numbers (strategy mixtures, payoffs, etc.) as rational numbers encoded as strings. These rational numbers are manipulated by GMP routines. The reason behind using rational numbers is precision: while the numbers themselves may grow quite large in size (if they compound), the library ensures that no roundoff occurs when tallying points.

No floating-point numbers are used for internal tallies, although these are distributed to the web frontend for more intuitive representation.


Matrix ordering and player role, among other subsystems, require a non-deterministic random number generator. To accomplish this, gamelab uses the BSD arc4random system interface to provide high-quality randomness. This is emulated with libbsd on GNU/Linux. The lottery computation uses a deterministic random number generator using an experimenter-defined seed: this allows lottery draws to be repeatable.

Note: if you're running on GNU/Linux, you may not benefit from the high quality randomness, as well as the general security and reliability, of BSD systems. We strongly encourage BSD as a platform of choice, especially OpenBSD, which is the main development platform.


Gamelab is designed with security in mind, but inherits most of its security from the operating system. In the recommended OpenBSD deployment configuration, gamelab binaries are statically compiled and run within a file-system jail. You can effect the same conditions on any well-configured web server configuration, however. On any supported system, the library used to communicate with the web server, kcgi, is heavily invested in security.

The logical security of the application itself is proportionate to the data security. Passwords are stored in clear-text because participants cannot actually change the passwords: they are opaque tokens (cookies). The only security-sensitive information on the server is the e-mail server password and the AWS secret key if provided for Mechanical Turk configurations.

To protect against misbehaving linked libraries that can access the network, the gamelab executables do enforce a strict process separation when invoking any e-mail or network-access functions. The rational-number and database libraries are used within the main processing context, however.


Gamelab is a CGI application that runs under any CGI-compatible web server on a modern UNIX system.

In order to use gamelab, you'll need to download, configure, and install it—just like any piece of software. If a system administrator has already installed the server and given you the web address for its laboratory and experimenter interfaces, jump down to the Experimenters section.

If at any time you have problems during an installation, or you encounter confusing language, please e-mail Kristaps with the issue. If you'd rather use a pre-installed environment, please visit


To operate a gamelab server, you'll need a modern UNIX system with a web server accessable to your participants. You are encouraged to use OpenBSD, which is the default testing and development environment. I don't discuss web server deployment in this manual: please consult your operating system documentation on how to do so. Specifically, you'll want to research on how to enable CGI scripts. FastCGI is also possible, but not the default installation.

To date, gamelab has been deployed on Mac OS X, GNU/Linux, and a number of BSD systems. It has been run under the Apache web server, nginx via slowcgi(8), and OpenBSD's httpd(8) via slowcgi(8). The system is designed to run in a jail as is the default with OpenBSD httpd(8). You are encouraged to do the same in any environment for security reasons; however, the system will also run fine in a non-jail context.

As mentioned in Backend, many configurations have been tested: only Apache2 is known to have issues, and these generally are harmless.


With your environment prepared, you'll need to download the gamelab source code as well as its dependencies. These are compile-time dependencies, so if the system compiles, it will run just fine. There are no additional run-time dependencies. The dependencies consist of the following:

  • kcgi (the CGI framework),
  • GMP (rational number library),
  • SQLite (database), and
  • libbsd (Linux compatibility, if applicable),
  • libCURL (e-mailing),
  • libexpat (parsing Mechanical Turk responses), and
  • json-c (testing system).

If you're running without a connection to the Internet, you'll also need to download the Font Awesome icon set and adjust the Compilation flags to point to your local copy.

Each of these systems has its own installation procedure beyond the scope of this document. After you've installed the dependencies, download the source archive and the checksum.

You can download the latest gamelab source code from the snapshots directory, or browse the source on the readonly GitHub repository. This also contains all previous releases.


Move the gamelab.tgz source code into a source directory (we suggest ~/Source or ~/src and unpack. First, however, verify the checksum. These instructions are valid for new installations and for upgrades: the source archive will unpack as gamelab-MAJOR-MINOR-BUILD, so you won't overwrite previous installations on upgrading.

openssl dgst -sha512 gamelab.tgz | cmp - gamelab.tgz.sha512
tar zxf gamelab.tgz && rm -f gamelab.tgz

Once you've unpacked the source code, you need to compile the server. You can override variables in the Makefile by specifying a Makefile.local, which will be automatically sourced.

This is the administrative CGI URI relative to the server root. It will correspond to the mapped location of CGIBIN plus the script name. This is often /cgi-bin/admin.
This is the path used for installing compiled (executable) CGI binaries. This is usually the cgibin directory for your web server. In most web servers, this defaults to /var/www/cgi-bin. It must be accessable (i.e., mapped) by the web server process.
This is the path used for installing read-only data files. This doesn't have a standard location. We recommend using /var/www/data. It must not be accessable by the web server process—only the running CGI processes.
This is a path used for installing read-only static pages accessable by the web server process like CSS, JavaScript, and HTML. This is usually the htdocs directory for your web server. In most web servers, this defaults to /var/www/htdocs. It must be accessable (i.e., mapped) by the web server process.
This is the mapped location of the HTDOCS variable. In most web servers, this defaults to / (the root).
This is the player (laboratory) CGI URI relative to the server root. It will correspond to the mapped location of CGIBIN plus the script name. This is often /cgi-bin/lab.
If you're going to statically link (see STATIC) then you may need to define extra libraries to satisfy your compiler. This really depends on the operating system. For OpenBSD, you'll need to define -lintl -liconv. For non-static compilations, you probably don't need anything here.
Convenience variable for defining the prefix of other variables. This is not used during operation.
This is the path used by the running CGI process to find DATADIR. This is because the running process may be jailed in a file-system (such as /var/www) and the absolute path of DATADIR may not be available.
If you're going to run in a file-system jail and don't want to import dynamic libraries, you can statically compile the gamelab executable by specifying -static.

There are some miscellaneous components that one may tune.

Specify an alternative path for the Fort Awesome icon set. This allows administrators to deploy LAN-only experiments, as otherwise they'd need to contact the given server for font icons.

Once you've defined these variables, compile the software in the usual way by executing make. The software should have no errors or warnings. If it does, please contact Kristaps with details.


Once you've compiled the software and configured its parameters, you can install it by simply running make installcgi. (You'll probably need to do sudo make installcgi, however.)

Warning: this will overwrite any existing database! To update an existing installation (assuming the database hasn't changed—the release notes for each version will tell you), use make updatecgi or sudo make updatecgi. I do not recommend this, as it's easy to miss a small database change. Gamelab does not provide any facility for modifying the database in-flight.

Both of these steps will install the following files.

  • Administrative CGI executables are installed into CGIBIN. These files are identical but for the suffix. These must resolve to ADMINURI.
  • Laboratory CGI executables are installed into CGIBIN. These files are identical but for the suffix. These must resolve to LABURI.
  • Statically-served, read-only, web-server accessible files are installed into HTDOCS.
  • Read-only data files are installed into DATADIR. They are used by the CGI processes to serve content to data requests. For example, there are a number of e-mail templates that are used by gamelab to programmatically send mails.

If you run make installcgi (instead of make updatecgi), the following will also be installed.

  • The writable database file itself, installed into DATADIR. This is initialised to the defaults described in the Experimenters section.

There are a number of file-system requirements to which you must be mindful.

  • The SQLite database system usually needs writable access to a temporary directory. If you're running in a file-system jail (e.g., OpenBSD), you'll need to make sure the temporary directory exists in the root of the jai. On default installations, this consists of a tmp directory in the document root, e.g., /var/www/tmp, with writable permissions.
  • The database directory itself needs file-creation permissions to allow for journalling (it uses write-ahead logging).
  • You will need the /dev/null device in your file-system. The file-system of the device directory must not be nodev in its mount options (see mount(8)).
  • If you're running in a file-system jail, the security measures of kcgi on OpenBSD with systrace(4) (i.e., OpenBSD <5.9) require a systrace device in the dev directory of the document root, e.g., /var/www/dev. Moreover, the file-system of this directory must not be nodev in its mount options (see mount(8)).

Once you've deployed your system, you can test it by going to the experimenters site.


In this section, I describe how to create experiments on the gamelab system. I assume that you've just installed it following the Installation section or that your system administrator has done so.

Gamelab experiments consist of a series of games (normal form bimatrices) played by a set of participants over a finite number of rounds of a certain amount of time. When the experiment is over, you run a lottery that selects a participant as a winner of the experiment. Thus, to run an experiment, you'll need players, games, and a duration of time split into rounds.

You can only run one experiment at a time on each gamelab installation. However, you can back up and wipe the experiment installation database—starting over—at any time to start over.

Creating Experiments

Begin by accessing the experimenter URL with your browser. This will usually look like, but it depends on how your system administrator has configured the system. You will be taken to a page prompting for an e-mail and password. Note: you'll need to configure your browser to accept cookies and run JavaScript for the domain upon which gamelab is running.

The default e-mail address is and the default password is xyzzy. If the system has been properly configured, you'll be logged in. Note: you can log out at any time by clicking the navigation bars in the upper left-hand corner, then selecting Logout.

The first thing you should do upon logging in is change the experimenter e-mail address and password using the Change Password and Change E-mail sections of the experimenter console. Set the experimenter e-mail to be your own e-mail address and the password to something fairly complicated. You'll be logged out—and must log back in—after each step. Make sure that no participants will be able to guess (or know) your credentials!

The next step is optional e-mail configuration in the E-mail Server section. You can run some experiments (e.g., captive-only) without e-mail support, but you won't be able to e-mail yourself the database. So it's best to set this. You may need your system administrator's help, but you should be able to enter the same information as when you configure your e-mail reader: the from address the e-mails will use (usually your own so that they can respond to you); the outgoing (SMTP) e-mail server in the form smtp://; and the user and password for accessing that server. Gamelab won't work for non-secure (i.e., non-TLS) e-mail access. When you're finished entering this information, test the configuration. You should receive an e-mail in a few minutes from the gamelab server.

Next, the experiment itself. Start with a set of normal form bimatrix games in the Games section. These can have any number of strategies and either positive integer or rational payoffs. Enter these games into the Games section. Bimatrix games are laid out top-left to bottom-right. So let's say you have the following game shown first as a symmetric game then with the implied payoffs to the opponent player.

1 0 -1
0 -1 1
-1 1 0
1, -1 0, 0 -1, 1
0, 0 -1, 1 1, -1
-1, 1 1, -1 0, 0

These would decompose into 1 -1 0 0 -1 1 0 0 -1 1 1 -1 -1 1 1 -1 0 0, where opponent payoffs are coloured.

Next, specify a set of participants identified by e-mail address in the Participants section. Identifiers may be separated by white-space (newlines, tabs, spaces). You'll see valid identifiers noted after submitting the form. If you specify Allow captive participants, participants can register themselves by accessing the playerautoadd.html URL from the moment you submit the form, so you may see participants appear as they add themselves. They are captive in the sense that their logout button is disabled and that they are never e-mailed (their identifiers are free-form, not e-mail addresses), nor are they retained if the database is wiped. Captive mode is disabled when you start an experiment: this is a security measure so that participants can't join after you've started. You can opt to allow participants to continually join by specifying that captive mode should be preserved when the experiment begins.

Once you've added your games and your participants, you're ready for the experiment configuration as assigned in Experiment Parameters. Start with the simple options (don't click on Show Advanced Options yet.) Begin with the time and date when the experiment begins, the number of rounds, duration of each round, and a fraction of participants that will cause the round to advance before the maximum time has elapsed. (You can set this to zero to force the maximum time to elapse before the round advances.) For example, if set to 50%, then 50% or more participants playing all games in both roles will cause the round to advance regardless the amount of time left. If there are zero participants in either player role, then it's as if nobody plays, making the round run until the maximum time elapses. The minimum time per round is one moment; the minimum number of rounds is one round.

The advanced options allow for finer control—be careful in using these! See the Mechanical Turk section for adding Mechanical Turk workers to your experiment.

The login URL is used when sending non-captive participants the welcome e-mail. Their query string credentials will be appended to the given URL. You don't want to change this unless, for example, the laboratory CGI script is sitting in another URL path.

You can customise the instructions shown to participants by choosing from a set of pre-made instructions or providing your own. If you provide your own, it's up to you to make sure that the instructions are a well-formatted HTML5 fragment. Review the existing instructions on how to customise.

You can customise round advancement in several ways. The first is limiting the total number of rounds per participant. Thus, if one specifies 10 total rounds and 5 rounds per participant, each participant will play at most five rounds. This is useful for running with captive participants that can join and participant over an extended period of play time. The minimum-time (in minutes) establishes a grace time for play. This is especially useful when <100% of participants play before the round automatically advances to prevent fast participants from having an unfair advantage. Lastly, the maximum limit per round bounds the number of simultaneous participants.

There are a number of option toggles in the last subsection. You can disable the lottery computation, which is useful for Mechanical Turk scenarios, so that participants are only shown their payoffs and not ranked and assigned lottery numbers. The questionnaire is a more significant option: if specified, participants are required to step through a series of questions before playing. Note: if you specify a questionnaire, participants will not automatically be joined to the experiment when it begins. In other words, until they finish the experiment, they will not be included in the count of current participants when looking at the fraction of completed game-plays per round. They join the round following the current one upon finishing the questionnaire. Lastly, you may specify that round advancement will trigger an e-mail to the participants.

Running Experiments

Once you've deployed your experiment, you must now administer it. You can enable and disable participants, reset participant passwords, and reconfigure your experimenter credentials. You can also review some statistics on the current experiment status.

If a participant forgets her password, you can reset it by clicking on the participant's email address and selecting the Reset Password option. (Captive participants are not affected.) You can reset all non-captive participants' passwords with the Reset All Passwords option. If a participant is misbehaving, you can disable her login by toggling the button next to her e-mail address. Once toggled, she'll no longer be able to log in. If she's already logged in, she'll be logged out; no plays are accepted.

If you misconfigured the e-mail server and no e-mails were sent out, you can retry the failed e-mail attempts by re-configuring (correctly!) the server and selecting Resend Error Mails.

When an experiment has finished and you have provided a lottery amount, you can compute its lottery by specifying a seed for the random number generator and a number of lottery winners.

Once the winner has been chosen and notified, and the game wound down, you can wipe and start afresh by selecting the Wipe Experiment option. This will first back up the database (e-mailing it to you in the process), then wipe all by the experimenter credentials, listed games, and participant identities (though not their play, of course). Note that captive participants are not retained if the database is wiped.

Mechanical Turk

Mechanical Turk is a system for allowing human workers to participant in Internet-based tasks. It is part of the Amazon Web Services product constellation. This section assumes a bit of familiarity with the Mechanical Turk system overall.

Experiment Setup

Setting up a Mechanical Turk experiment is very easy. First, have your AWS access key identifier and secret key handy. Fill these in when configuring your experiment, then set the number of workers (corresponding to the max assignments in the API reference), the reward amount (reward in the API), and currency conversion (used when computing bonuses). You can use either the Developer Sandbox or production mode—I recommend starting with the first. You can also enter a name and a description, which will be shown to potential workers, as well as the keywords which are used when searching for HITs. Lastly, you can set some worker requirements. You can specify an assignment approval percent, number of accepted HITs, and locale. By default, there are no requirements.

You'll probably also want to use the Mechanical Turk instructions, and you should enable the questionnaire to control for workers' understanding of game-play.

When you start your experiment, the request will be sent to either the sandbox or production server. We strongly suggest that you use the sandbox server a few times to make sure that your HIT posts as you wish it to. If there are errors, you'll be notified of them.

Note: when Mechanical Turk workers join the system, their session information is embedded in the query string. This is because most browsers disallow third-party cookies by default. So be warned that this is not a secure method of operation!

Granting Bonuses

You can optionally assign bonuses to Mechanical Turk players in your experiment after the experiment has concluded. To do so, you'll need to click the Assign bonus button that appears beneath the high-scorer table on the experimenter page. This button is only shown for Mechanical Turk experiments. It's ok to click this twice: the Mechanical Turk system will keep track of who has been awarded bonuses and not award the same bonus twice.

Bonuses are only awarded to Mechanical Turk players who have indicated completion. You can see which players have done so by looking whether they have checkmarks next to their identifiers in the player listing.

The bonus assigned is the number of tickets times the conversion rate.

Analysing Experiments

At this time, gamelab doesn't have any special functionality for analysing played (or in-flight) experiments. You can, however, directly access the SQLite game database by backing up on the experimenter console. This will e-mail a redacted database to the experimenter. The redacted bits consist of user and experimenter e-mail passwords. The database is in WAL mode.

Database Schema

The database schema is fairly straightforward. It consists of the following tables. In all of these, note that Unique Identifier fields are non-zero SQLite rowid fields. Rational numbers are stored as text, such as 1/2 or just 2, starting with 0/1. You can also see a detailed map of the schema.

Experimenter (a.k.a. administrator) credentials. There is always one (and only one) row that exists, initialised to default values.
The experimenter e-mail address. This is used as a predefined template value for the instructions. It is also used as the destination for backups of the database. Initialises to
The experimenter password. Note: this is stored as cleartext, so it is not really a hash. Initialises to xyzzy.
Unique identifier.
Bit-field on whether the email and hash have been set by the user. Contains 0x01 if the e-mail has been set, 0x02 for the password.
A strategy mixture created when a player plays a round in a game.
When this was created (epoch).
The game.
Unique identifier.
The participant.
The round number (starting at zero).
The session.
A text (space-separated) list of rational numbers of the strategy mixture ordered from the top if a row-playing role (i.e., player.role) or left if a column-player.
Number of entries in choice.strats. This obviously equals the number of strategies available to the participant in that game, given the player role.
Most of the time, gamelab users will want to use the pre-bundled questionnaire at the start of a game. However, sometimes they'll want their own questions. Provide a space for that here.
The answer to the question. This is checked by a strict string comparison for equality.
Unique identifier.
The free-form question. This is in regular text.
The rank of the question.
This describes an experiment configured by the experimenter (see admin). There is always one (and only one) row in this table.
The experiment is currently accepting (or did accept) auto-added participants.
Preserve the experiment.autoadd state during and after the experiment start. This is useful for rolling experiments so as not to have a break between starting the experiment and re-enabling captive mode.
If not an empty string, a Mechanical Turk access key. This and experiment.awssecretkey must be set in order to enable a Mechanical Turk experiment.
How to convert a Mechanical Turk participant's final score (tickets) into real currency, if applicable.
Description of this Mechanical Turk experiment, if applicable.
If not an empty string, set to an error from a preceeding Mechanical Turk “start new HIT” operation that failed.
Keywords (comma-separated) of this Mechanical Turk experiment, if applicable.
If non-empty and applicable, the locale to request (country code) for Mechanical Turk participants.
What to call this Mechanical Turk experiment, if applicable.
The base reward to give to Mechanical Turk participants, if applicable.
Whether to use the Mechanical Turk sandbox in all enquiries, if applicable.
If not an empty string, a Mechanical Turk secret key. This and experiment.awsaccesskey must be set in order to enable a Mechanical Turk experiment.
If <=0 and applicable, the number of HITs for each Mechanical Turk participant.
How many Mechanical Turk workers to request, if applicable.
If <=0 and applicable, the percent (out of 100) of approved HITs for each Mechanical Turk participant.
Bit-field. Contains 0x01 if the history is not to be transmitted to participants, 0x02 if games and rows shouldn't be randomised prior to display, and 0x04 to show participants the relative count of rounds—relative to when they started to play.
A JSON file that provides a “fake” history prepended to the actual experiment history. This is only shown to users through the browser interface.
HIT (Mechanical Turk) identifier given by the Mechanical Turk server for an mturk experiment. Otherwise, an empty string.
Unique identifier.
The instructions shown to participants. This must contain valid HTML5. It may contain @@gamelab-admin-email@@, which is filled in with the experimenter's configured e-mail address; @@gamelab-games@@ for the number of games; @@gamelab-rounds@@ for the number of rounds; and @@gamelab-round-time@@, a decimal number of the number of hours per round.
Set when the experiment begins (i.e., when experiment.state <0), this is the URL given to players in their initial e-mail for when they log in. This is suffixed by “?ident=EMAIL&password=PASSWORD”.
Human-readable amount awarded in lottery, e.g., 10 000 SEK. If set to the empty string, there's no lottery.
The number of minutes per experiment.round. Minimum of one.
The maximum number of participants per player role at any given time. If zero, the number of participants is unbound.
The number of rounds playable by each participant. If zero, participants will play until the end of the experiment. If <0, participants will not be allowed to play more than the given number of rounds within the experiment.
If non-zero, new participants are not directly assigned a join round and must use the lobby facility. There, they will be asked questions and cannot proceed until all questions have been answered. See questionnaire.
The current round of the experiment, or -1 if the rounds have not begun incrementing. This will be set to experiment.rounds when the experiment concludes.
If round is non-negative, then the epoch when the round was incremented. Otherwise, this is zero.
If roundpct is non-zero, this is the “grace time” (in minutes) before which the round will automatically advance giving the percentage.
If <0, this represents the fraction of participants per player role who play all games and determine that the round automatically advances. In other words, if set to 0.5 (50%), then from both player roles, if 50% or more participants have played all games, the round advances in advance of the set round termination time.
If specified when configuring the experiment, a daemon will be started that periodically checks to see if the round has advanced. Its process identifier is stored in this field. Otherwise, this is zero.
The number of rounds that will be played. Minimum of one.
Epoch when the experiment when the first round of the experiment begins.
The running state of the experiment, being either 0, for new (still in the configuration stage); 1, for started (participants can log in, though the experiment itself may not be accepting plays yet); 2, where the experiment has expired, but the winner has not been chosen by the experimenter; or 3, where the experiment has expired and the winner has been chosen.
When the experiment finishes (i.e., when experiment.state is set to 2), this is filled in with the total number of lottery tickets (the player.finalscore) awarded to all participants.
A game is a payoff bimatrix configured by the admin. There may be any non-zero number of games in a running experiment.
Unique identifier.
The name of the experiment. This is shown only to the admin.
Number of strategies for row player.
Number of strategies for column player.
A list of space-separated payoffs from the top-left to the bottom-right of the payoff matrix, ordered row-player payoff, column-player payoff.
During a given round, this records a player's status in terms of number of choice rows (plays) made.
The number of games this participant has played, i.e., the count of choice rows.
Unique identifier.
Participant identifier.
The round number (starting at zero).
A lottery is created for an individual player when the corresponding participant has been granted payoff for all game rows in a round. This doesn't really have anything to do with the “lottery” concept of reward: the name is just an historic holdover.
The player's aggregate payoff (as a rational number) computing by adding the previous round's aggregate payoff to the current lottery.curpayoff.
The value of lottery.aggrpayoff represented as a natural number of tickets.
The participant's current payoff (as a rational number) computing by accumulating her payoff.payoff for all games in the experiment. This is set to 0/1 if the participant has not played all games.
Unique identifier.
The participant owning the lottery.
The round number (starting at zero) for which this lottery was computed.
Records the average strategy mixture of a given player role for a given round and game when a round has concluded.
For the row player, the average strategy mixture of the current round. Strategy mixtures are only considered for choice rows where the player played all games for the round, so this will be a set of zeroes if no participant completed all games. This is recorded as a space-separated sequence of rational numbers, one per strategy.
Like past.currentsp1 but for the column player role.
Game being referenced.
Unique identifier.
The total number of plays that were submitted (over all games) in this round if and only if the participant submitted for all games in that round.
The round starting at zero.
The accumulated count of rounds not skipping, i.e., the count of zero-valued skip rounds.
If zero row or column-player participants played all games, this is set to 1, else it is 0 (sufficient players played).
When the given round has completed, this consists of the payoff of the participant's choice strategy mix for a given game when played against the average strategy of the opposing player role.
Game identifier.
Unique identifier.
A rational number of the payoff.
Participant identifier.
The round number (starting at zero).
The central player table consists of all participants. We use the term “participant” instead of player in the literature to disambiguate a player role and an experiment subject. The player.hash variable is redacted when the experiment is wiped.
If the questionnaire facility has been enabled (via experiment.questionnaire), this is non-zero if the questionnaire has been answered.
If the participant joined from Mechanical Turk, the assignment identifier. Otherwise set to the empty string.
Player was auto-added (i.e., auto-added herself). These participants are never e-mailed.
Player e-mail address or identifier, in the event of a captive participant. This is unique in the set of all participants.
Whether a participant is allowed to login during an experiment.
The minimum of the slot of the participant's tickets among all participants' tickets. For example, given 100 participants with roughly 10 tickets each, this might be 543 to indicate that slot 543 to 543 plus player.finalscore are this participant's slots in the lottery. Mechanical Turk players are not included in this computation, so this value is not meaningful for them.
Set to the accumulated payoffs from lottery.aggrpayoff rounded up to the nearest integer. Mechanical Turk players are not included in this computation, so this value is not meaningful for them.
The participant's password set when the experiment is started (i.e., when experiment.state is <0) or when reset, or when a captive participant registeres. Note: this is stored in the clear: it is not a hash!
If the participant joined from Mechanical Turk, the HIT identifier. Otherwise set to the empty string.
Unique identifier.
Whether the participant should be shown instructions when she logs in (versus being taken directly to the game-play tab).
The round when the participant joined (i.e., started playing) the experiment. This is set to zero when participants join at the outset. If -1, the participant exists, but has not yet started participating in the experiment.
Whether the Mechanical Turk participant, if applicable, has indicated that they've finished the experiment. Note that this is unprotected and can be set at any time: it is only used to display a message to report to the Mechanical Turk server.
The player role (0 for row player, 1 for column player) set when experiment.state is <0 or when the participant joins.
A cryptographically random number <0 given to the participant when created. This is used for many purposes, one of which being the bonus identifier for Mechanical Turk participants.
The state of a participant can be 0, meaning the participant is newly-added and has no password; 1 when the participant has been mailed her password; 2, the participant has logged in; and 3, an error occured when the password e-mail was attempted.
A number from zero that indicates the number of time the player columns have been updated. This is used to make the object cachable.
The questionnaire is an optional set of questions given to each participant. These participants must submit a set number of correct answers before joining the game. Only a participant's first answer counts.
The epoch time when the row is created.
Unique identifier.
The question number starting at zero.
The number of tries, correct or not. This is zero only in the race after initially creating the row and noting an attempt. It should not be used and will be removed.
A browser (or otherwise www) session. Sessions are the usual browser session used when participants (or the experimenter) are interacting with the system.
Magic random cookie for each session.
Epoch when session was created.
Unique identifier.
Participant identifier, if not NULL. Otherwise, this is the admin logged in.
Browser-reported user agent.
This consists of the SMTP server information used in sending e-mails. There is always only one row set, which defaults to empty values (see smtp.isset).
The e-mail address used as the “From” address in all communication from the server to participants in the experimenter. It is usually the same as the experimenter email set in admin, but may be set as a standard “No-Reply”. (This is discouraged, as your participants should be able to reply to you if things go wrong.)
Unique identifier.
Whether these entries have been set.
The SMTP username's password (used for logging in). Note that this is stored in cleartext, so make sure that your password isn't used elsewhere.
The SMTP server in smtp://server:port format.
The SMTP server username (used for logging in).
The winner table consists of rows corresponding to a player and her winner status. This table does not exist until the experiment.state field is set to 3. This table is deleted when the database is wiped.
Unique identifier.
The random number modulo the total number of tickts used for this winning draw. In other words, this is the winning lottery ticket.
Boolean value as to whether the player is a winner. If this is false, then the winner.winrank and winner.rnum columns are undefined.
If the player is a winner, the rank (first, second, third draw...) of their winning.

Programmable API

As described in the Technicalities section, gamelab is at heart a web application making JSON requests of a server. This JSON interface is simple and easy to use to create a programmatic interface. If you wish to create a programmatic interface (a bot), and your experimenter consents, it's a fairly easy process. In fact, a tool already exists within the gamelab source distributions to stress test systems using this interface.

To do this, you'll need a programming language that has support for JSON and making HTTP requests. In C, you can use json-c and libCURL, respectively.

Note to developers: if you're writing your own bot, do not flood requests. Be courteous to other users: there is no competitive advantage to be gained by polling the system constantly.

Note to experimenters: in order to run a programmable experiment, you cannot use a Questionnaire, as questions are free-form. In theory, you could enable the questionnaire and have players play through it, then turn on their bots, but in practise participants knowledgeable enough to deploy bots won't need the questionnaire!

General Sequence

The general sequence is as follows:

  1. Get your username and password, or optionally, have your utility register with doautoadd.json as a captive player and get a randomly-generated password.
  2. Wait for the experiment to accept players by polling dologin.json in order to login.
  3. Log in to dologin.json with your username and password, accepting your session identifier as cookies.
  4. Wait for the experiment to begin by polling doloadexpr.json. Use your session identifier as an identifier cookie.
  5. Accept the experiment data from doloadexpr.json. Use your session identifier as an identifier cookie.
  6. Using the experiment data from doloadexpr.json, construct and submit a series of game plays to doplay.json. Use your session identifier as an identifier cookie.
  7. Wait for the next round, or end of game, by polling polling doloadexpr.json. Use your session identifier as an identifier cookie.

Do not flood the system with requests: be considerate! Unless specifically stipulated by the experimenter, you are not rewarded for fast play. Avoid re-connecting with anything less than 30–60 seconds between attempts, unless specifically allowed by an experimenter, or your bot may be banned from the experiment.


This section describes the resources (URLs) that may be accessed for programmatic play. Each of these resources is the immediate path following the laboratory CGI script name, for example, if you're running as localhost/lab.cgi, these would be mapped to localhost/lab.cgi/doautoadd.json.

doautoadd.json (POST)

Post your e-mail address using the ident field name. Returns HTTP error code 400 (invalid or missing e-mail field), 403 (player by that e-mail already exists), 404 (captive players disallowed), or 200 (success). Upon success, the return body is a JSON object consisting of the ident and password string fields.

dologin.json (POST)

Post your e-mail address (or identifier) and password using the ident and password field names. Returns HTTP error code 400 (invalid or missing input fields), 409 (experiment not started yet), 200 (success). If you receive HTTP error code 409, wait and continue polling: the experiment is not yet accepting player logins. Upon success, the return body is empty, but the cookie field consists of sessid, your session identifiers (a number); and sesscookie, a magic number attached to your identifier.

doloadexpr.json (GET)

Get a doloadexpr object describing the full experiment. You will need to pass the sessid and sesscookie cookie fields as described in dologin.json. Returns HTTP error code 304 (not changed since last request), 409 (experiment not started yet), 429 (player accepted, but not yet joined for play), 200 (success). If you receive HTTP error code 409 or 429, wait and continue polling: the experiment is not yet started or has started, but you are not allowed to join yet. Upon success, returns the JSON experiment object.

doplay.json (POST)

Post a probability mixture for a game's actions. You will need to pass the sessid and sesscookie cookie fields as described in dologin.json. The game identifier and round must be encoded in the gid and round fields, respectively. Each action with a requested probability must be posted as field indexNNN, where N is the action index (starting at zero), e.g., index0, index1, and so on. Probabilities may be decimal or fractional strings. Missing actions are assigned zero probability. Actions greater than the maximum action are ignored. Returns HTTP error code 400 (invalid game, round, or probability, or probability sum doesn't equal one), 409 (round has passed or is being played again, experiment hasn't started or has finished, or player hasn't joined), or 200 (success). Note: the error codes for this resource are likely to become more specific in future releases.


When using the Programmable API via the web Resources, you'll need to examine some JSON objects to ascertain the experiment status. The following defines the objects available to you by resource. The type of each field is listed alongside the field name: object for dictionaries (i.e., things that contain other things), strings being UTF-8 strings, integers being 64-bit signed integers, doubles being double-precision floating points, and times being seconds since UNIX epoch.

There are many undocumented fields that are transmitted while playing: these are to be considered unstable and will not be included in subsequent releases.


Obtained from a successful doloadexpr.json request, i.e., with an HTTP error code 200 and a parsed JSON object. The resulting dictionary consists of the following top-level fields.

expr (object)

Object describing the experiment status. This object is always non-null. This contains more or less all the information you'll need.

history (object)
Additional history. See the history object.
instr (string)
Instructions for participating in the experiment. This is an HTML5 fragment describing how to use the experiment. It is generally formatted in the web interface, so the styling may make the data non-obvious.
minutes (integer)
The maximum number of minutes per round (all games). This is the maximum: in reality, rounds may range from roundmin to this value.
prounds (integer)
The total number of rounds played by the logged-in participant. The joined round plus this is the participation span for any participant.
round (integer)
The current round number starting at zero. (If the round is less than zero, the experiment has not yet begun.)
roundbegan (time)
When the current round began.
roundmin (integer)
The minimum amount of time, in minutes, per round. This is also called the grace time. If this is non-zero, players are given this much time to play freely until the round will advance based upon the player fraction (see roundpct).
rounds (integer)
Total number of rounds in the experiment. Note: you may not actually play this number of rounds. See prounds.
start (time)
Time when the experiment started (or will start).
player (object)

Information about the current participant. This object is always non-null.

joined (integer)
The round when the current participant may first play. If this is less than the current round, the participant has been accepted into the system, but is not yet playing.
role (integer)
Whether the participant is playing the row (zero) or column (non-zero) role in the game bimatrices.
history (array)

The game history and information about individual games. This is an array of objects with the following fields.

id (integer)
The identifier of this game.
p1 (integer)
Number of actions for the row player.
p2 (integer)
Number of actions for the column player.
payoffs (array)
Payoff matrix. This is an array (of length p1 of arrays (each of length p2) of strings. Each string is a rational number formatted as a fraction (e.g., 1/2, -1/2, 0, 1, etc.). Clients should use the GMP library to work with these rational numbers.
roundups (array)

An array of prior play for this game, indexed by round. If this is set to an empty-length array, the experimenter might be prohibiting the display of history, or there may not yet be a history to display given the current round number.

skip (integer)
Non-zero if there weren't enough plays in the given round to compute payoffs. Zero otherwise.
navgp1 (array)
An array of doubles representing the average weight of this action for the row player role. The array if of length p1.
navgp2 (array)
An array of doubles representing the average weight of this action for the column player role. The array if of length p2.
navgs (array)
An array of arrays consisting of the multiples of the average fraction for the given row and column player cell.