Gamelab is an online game theory laboratory where participants assign mixed strategies to game bimatrices over a series of rounds, accumulating payoffs as the experiment progresses. The mission of gamelab is to allow for laboratory-style interaction without the need for a laboratory.
This is the user's manual for gamelab. (There's also a quickstart guide.) Its purpose is to introduce the system, then describe how to deploy it and how to run experiments.
Gamelab is an online laboratory for conducting game theory experiments. Experiments consist of an experimenter, an experiment configuration, participants, and a set of games. The web interface accomodates for desktop, mobile, and tablet-sized devices. You can even play experiments programatically over a JSON API.
An experimenter begins by configuring—among many other parameters—how rounds in the experiment progress and for how long the experiment will run. Once the experimenter deploys the experiment, participants can log in and play each game once per round. When the round advances, participants earn points by computing the payoff from their submitted strategies against the average mixture submited by participants playing the opponent player role. After the last round, participants' points award lottery tickets, and winners (if winners are part of the configuration) are drawn by selecting lottery tickets.
Gamelab has many configurable parameters, for example, round advancement based
on number of participants who have played; computing or not computing lotteries; Amazon Mechanical Turk integration; captive
laboratory-based participants who cannot log out or back in; and so on.
Experiment participants are identified by an e-mail address, an opaque identifier in
captive configurations, or a worker identifier in Mechanical Turk configurations (this is effectively a captive scenario).
An experiments can have as many participants as the server (or the experimenter!) can handle.
Participants are assigned by the experimenter, assign themselves in a
configuration, or join from Mechanical Turk console.
In the first case, participants are e-mailed a password and login credentials when the
experimenter begins the experiment.
(If no e-mail server has been configured, they are not e-mailed.)
These participants can log in and log out over the course of the experiment.
In the latter cases, participants are not e-mailed, and are assumed to be
they cannot log out or back in) for the duration of the experiment.
Participants may be added (or add themselves) before an experiment begins, so that participation
is static; or be added (or add themselves) both before and after the experiment has begun.
In this dynamic case, experimenters can limit the number of simultaneous participants, allowing
for a constant flow of participants joining and leaving the system as the experiment progresses.
Participants that have logged in but cannot yet play due to the concurrent-participant limit are
placed in a
lobby until they join the experiment.
(The same lobby is also used for the Questionnaire.)
When the experiment begins or when participants otherwise join, participants are randomly assigned a player role (row or column) used for all games throughout the experiment. Experiments always have an equal assignment of row and column player participants unless the total number of participants is odd, in which case the extra participant is assigned a random role. If participants join after the experiment has started, they are assigned to a player role that balances role counts at the time of assignment.
The experimenter may stipulate that participants step through a questionnaire before being
allowed to play.
In this treatment, participants are initially dropped into a
lobby where they must answer
questions before beginning.
(This lobby is also used to handle an overflow of participants as defined in the Participants section.)
The questionnaire is designed to ensure that participants know how to play with minimum bias toward a given configuration. Results are recorded (duration of answering, number of tries, etc.) so experimenters may correlate performance with outcome.
Experimenters may customise their questionnaire manually when creating their experiment—please contact us if you need guidance in doing so. You'll be able, in doing so, to stipulate any number of free-form questions to ask your participants before they can begin.
A game in gamelab is simply a payoff bi-matrix. Payoffs are rationally-valued numbers. (Note: negative participant tickets will cause the lottery computation to fail.) There is no limit to the number of strategies for this matrix; however, in practise, only so many columns fit neatly on a small screen. Games larger than four or five strategies wide will look funny. Each experiment can have as many games as desired.
Once the participant joins an experiment, participants can submit game-plays, one per game per round. A game-play consists of a strategy mixture: a set of probabilities assigned to each of the player role's actions. The submitted probabilities are non-negative and rationally-valued, and always sum to one. Participants submit these probabilites by clicking on rows in the game matrix (participants appear to play the row player regardless of their player role) and entering in a decimal or rational number corresponding to the action.
Participants can only play one game at a time, with each game played once per round. If they log out and log back in (an action available only to non-captive participants), then the last game to play is displayed, assuming the round has not advanced in the meantime. Game matrices have their rows and columns randomly shuffled between participants, making it slightly more difficult to compare actions. Moreover, the order in which games are displayed is randomised between participants.
At the conclusion of each round, participants who have submitted mixtures for all games have their payoffs computed as points. (Rounds with incomplete games yield zero points.) Points are computed by playing the participant's mixture against the average mixture of participants in the opposing player role. Only opposing role participants who have played all games are considered. The points are then computed from the game payoffs. All of these calculations are over rational numbers and are unaffected by rounding. If not enough participants have played in a given player role, no points are tallied for the round.
After the first round, participants can review the full history of play.
This includes the history of their own strategy mixture, the average opponent role mixture, and
the participant role's average mixture.
Moreover, it includes the participant's accumulation of points, points per round, and the
hypothetical points per round of playing pure strategies.
Experimenters may also provide a
seed history that is prepended to the actual history of
This allows the results of prior experiments to be incorporated into the current one.
When a participant has exceeded her maximum allowed rounds, she is no longer able to submit plays to the experiment. When all rounds have concluded, the experiment halts. Participants may still log in and review the history, as well as see the total number of points awarded and their lottery ticket allotment. The lottery ticket allotment is computed by rounding up the participant's total payoffs. Note: Mechanical Turk players are not included in the lottery computation, nor are they shown any information pertaining to the lottery.
The experimenter may then trigger the lottery from the console. (Experiments may also run without a lottery.) This consists of seeding a deterministic random number generator with a given value. Each of these values is mapped into the lottery tickets to choose a winner. A participant is only chosen once for winning: if the random number re-selects a participant, a new number is drawn (rejection sampling).
Note: if any final allotments are less than zero, the lottery uses negative-valued semantics as follows. First, all allotments are offset by the minimum (negative) allotment. For example, if the smallest allotment is -11, all allotments are increased by 11. Then lottery is then computed from these non-negative adjustment allotments.
Upon triggering winner selection, winner participants are notified on their login screen, along with the lottery ticket causing the win. This is for transparency of the winner selection. Non-winning participants are also notified.
Gamelab uses a broad set of technologies to work. In this section, I briefly describe the specific technologies used. All of this is visible in the source code, but let this act as a primer. In general, these can be broken down into the user interface, which experimenters and participants use, and the server backend, which is accessed by the user interface.
user interface to gamelab is via a web browser.
In general, the user interface involves an HTML5 web-page styled with CSS.
Both HTML and CSS have been carefully validated using the
W3C Markup Validation Service and
CSS Validation Service, respectively.
via a series of JSON objects.
older browser will work properly.
(It has, however, been checked over with JSHint and other
Check that your browser supports
FormData, and of course
Graphs play a significant role in gamelab. The interface includes a local copy of flotr2 for drawing line and bar graphs. These graphs will display properly on any modern web browser.
The interface is styled with the Font Awesome icon set. The path to the icon configuration may be provided during compilation, and defaults to the on-line repository.
A copy of HumanizeDuration.js is also bundled with the system for displaying readable time durations.
The backend of gamelab is a BCHS system consisting of two compiled CGI programs: one for the experimenter, one for participants. These are small ISO C applications linking to a number of open source libraries. Consult the Deployment section for how to install these libraries.
Gamelab has been run on OpenBSD with httpd and the historical Apache, Mac OS X with Apache2, and Linux with Apache. System administrators are not encouraged to use Apache2 since it disallows long-running child processes of the CGI scripts. Such processes are used in several circumstances.
Internally, gamelab stores all non-integer numbers (strategy mixtures, payoffs, etc.) as rational numbers encoded as strings. These rational numbers are manipulated by GMP routines. The reason behind using rational numbers is precision: while the numbers themselves may grow quite large in size (if they compound), the library ensures that no roundoff occurs when tallying points.
No floating-point numbers are used for internal tallies, although these are distributed to the web frontend for more intuitive representation.
Matrix ordering and player role, among other subsystems, require a non-deterministic random number generator. To accomplish this, gamelab uses the BSD arc4random system interface to provide high-quality randomness. This is emulated with libbsd on GNU/Linux. The lottery computation uses a deterministic random number generator using an experimenter-defined seed: this allows lottery draws to be repeatable.
Note: if you're running on GNU/Linux, you may not benefit from the high quality randomness, as well as the general security and reliability, of BSD systems. We strongly encourage BSD as a platform of choice, especially OpenBSD, which is the main development platform.
Gamelab is designed with security in mind, but inherits most of its security from the operating system. In the recommended OpenBSD deployment configuration, gamelab binaries are statically compiled and run within a file-system jail. You can effect the same conditions on any well-configured web server configuration, however. On any supported system, the library used to communicate with the web server, kcgi, is heavily invested in security.
logical security of the application itself is proportionate to the data security.
Passwords are stored in clear-text because participants cannot actually change the passwords:
they are opaque tokens (
The only security-sensitive information on the server is the e-mail server password and the AWS
secret key if provided for Mechanical Turk configurations.
To protect against misbehaving linked libraries that can access the network, the gamelab executables do enforce a strict process separation when invoking any e-mail or network-access functions. The rational-number and database libraries are used within the main processing context, however.
Gamelab is a CGI application that runs under any CGI-compatible web server on a modern UNIX system.
In order to use gamelab, you'll need to download, configure, and install it—just like any piece of software. If a system administrator has already installed the server and given you the web address for its laboratory and experimenter interfaces, jump down to the Experimenters section.
If at any time you have problems during an installation, or you encounter confusing language, please e-mail Kristaps with the issue. If you'd rather use a pre-installed environment, please visit gametheorylab.org.
To operate a gamelab server, you'll need a modern UNIX system with a web server accessable to your participants. You are encouraged to use OpenBSD, which is the default testing and development environment. I don't discuss web server deployment in this manual: please consult your operating system documentation on how to do so. Specifically, you'll want to research on how to enable CGI scripts. FastCGI is also possible, but not the default installation.
To date, gamelab has been deployed on Mac OS X, GNU/Linux, and a number
of BSD systems.
It has been run under the Apache web server,
nginx via slowcgi(8),
and OpenBSD's httpd(8) via
The system is designed to run in a
jail as is the default with OpenBSD httpd(8).
You are encouraged to do the same in any environment for security reasons; however, the system
will also run fine in a non-jail context.
As mentioned in Backend, many configurations have been tested: only Apache2 is known to have issues, and these generally are harmless.
With your environment prepared, you'll need to download the gamelab source code as well as its dependencies. These are compile-time dependencies, so if the system compiles, it will run just fine. There are no additional run-time dependencies. The dependencies consist of the following:
- kcgi (the CGI framework),
- GMP (rational number library),
- SQLite (database), and
- libbsd (Linux compatibility, if applicable),
- libCURL (e-mailing),
- libexpat (parsing Mechanical Turk responses), and
- json-c (testing system).
If you're running without a connection to the Internet, you'll also need to download the Font Awesome icon set and adjust the Compilation flags to point to your local copy.
Each of these systems has its own installation procedure beyond the scope of this document. After you've installed the dependencies, download the source archive and the checksum.
You can download the latest gamelab source code from the snapshots directory, or browse the source on the readonly GitHub repository. This also contains all previous releases.
Move the gamelab.tgz source code into a source directory (we suggest ~/Source or ~/src and unpack. First, however, verify the checksum. These instructions are valid for new installations and for upgrades: the source archive will unpack as gamelab-MAJOR-MINOR-BUILD, so you won't overwrite previous installations on upgrading.
openssl dgst -sha512 gamelab.tgz | cmp - gamelab.tgz.sha512
tar zxf gamelab.tgz && rm -f gamelab.tgz
Once you've unpacked the source code, you need to compile the server. You can override variables in the Makefile by specifying a Makefile.local, which will be automatically sourced.
This is the administrative CGI URI relative to the server root.
It will correspond to the mapped location of
CGIBINplus the script name. This is often /cgi-bin/admin.
This is the path used for installing compiled (executable) CGI binaries.
This is usually the
cgibindirectory for your web server. In most web servers, this defaults to /var/www/cgi-bin. It must be accessable (i.e., mapped) by the web server process.
- This is the path used for installing read-only data files. This doesn't have a standard location. We recommend using /var/www/data. It must not be accessable by the web server process—only the running CGI processes.
This is a path used for installing read-only
htdocsdirectory for your web server. In most web servers, this defaults to /var/www/htdocs. It must be accessable (i.e., mapped) by the web server process.
This is the mapped location of the
HTDOCSvariable. In most web servers, this defaults to / (the root).
This is the player (laboratory) CGI URI relative to the server root.
It will correspond to the mapped location of
CGIBINplus the script name. This is often /cgi-bin/lab.
If you're going to statically link (see
STATIC) then you may need to define extra libraries to satisfy your compiler. This really depends on the operating system. For OpenBSD, you'll need to define
-lintl -liconv. For non-static compilations, you probably don't need anything here.
- Convenience variable for defining the prefix of other variables. This is not used during operation.
This is the path used by the running CGI process to find
DATADIR. This is because the running process may be jailed in a file-system (such as /var/www) and the absolute path of
DATADIRmay not be available.
If you're going to run in a file-system jail and don't want to import dynamic libraries,
you can statically compile the gamelab executable by specifying
There are some miscellaneous components that one may tune.
- Specify an alternative path for the Fort Awesome icon set. This allows administrators to deploy LAN-only experiments, as otherwise they'd need to contact the given server for font icons.
Once you've defined these variables, compile the software in the usual way by executing make. The software should have no errors or warnings. If it does, please contact Kristaps with details.
Once you've compiled the software and configured its parameters, you can install it by simply running make installcgi. (You'll probably need to do sudo make installcgi, however.)
Warning: this will overwrite any existing database! To update an existing installation (assuming the database hasn't changed—the release notes for each version will tell you), use make updatecgi or sudo make updatecgi. I do not recommend this, as it's easy to miss a small database change. Gamelab does not provide any facility for modifying the database in-flight.
Both of these steps will install the following files.
Administrative CGI executables are installed into
CGIBIN. These files are identical but for the suffix. These must resolve to
Laboratory CGI executables are installed into
CGIBIN. These files are identical but for the suffix. These must resolve to
Statically-served, read-only, web-server accessible files are installed into
Read-only data files are installed into
DATADIR. They are used by the CGI processes to serve content to data requests. For example, there are a number of e-mail templates that are used by gamelab to programmatically send mails.
If you run make installcgi (instead of make updatecgi), the following will also be installed.
The writable database file itself, installed into
DATADIR. This is initialised to the defaults described in the Experimenters section.
There are a number of file-system requirements to which you must be mindful.
- The SQLite database system usually needs writable access to a temporary directory. If you're running in a file-system jail (e.g., OpenBSD), you'll need to make sure the temporary directory exists in the root of the jai. On default installations, this consists of a tmp directory in the document root, e.g., /var/www/tmp, with writable permissions.
- The database directory itself needs file-creation permissions to allow for journalling (it uses write-ahead logging).
You will need the /dev/null device in your file-system. The
file-system of the device directory must not be
nodevin its mount options (see mount(8)).
If you're running in a file-system jail, the security measures of kcgi on OpenBSD with systrace(4)
(i.e., OpenBSD <5.9) require a systrace device in the dev
directory of the document root, e.g., /var/www/dev. Moreover,
the file-system of this directory must not be
nodevin its mount options (see mount(8)).
Once you've deployed your system, you can test it by going to the experimenters site.
In this section, I describe how to create experiments on the gamelab system. I assume that you've just installed it following the Installation section or that your system administrator has done so.
Gamelab experiments consist of a series of games (normal form bimatrices) played by a set of participants over a finite number of rounds of a certain amount of time. When the experiment is over, you run a lottery that selects a participant as a winner of the experiment. Thus, to run an experiment, you'll need players, games, and a duration of time split into rounds.
You can only run one experiment at a time on each gamelab installation. However, you can back up and wipe the experiment installation database—starting over—at any time to start over.
The default e-mail address is firstname.lastname@example.org and the default password is
If the system has been properly configured, you'll be logged in.
Note: you can log out at any time by clicking the navigation bars in the upper
left-hand corner, then selecting
The first thing you should do upon logging in is change the experimenter e-mail address and
password using the
Change Password and
Change E-mail sections of the
Set the experimenter e-mail to be your own e-mail address and the password to something fairly
You'll be logged out—and must log back in—after each step.
Make sure that no participants will be able to guess (or know) your credentials!
The next step is optional e-mail configuration in the
E-mail Server section.
You can run some experiments (e.g., captive-only) without e-mail support, but you won't be able
to e-mail yourself the database.
So it's best to set this.
You may need your system administrator's help, but you should be able to enter the same information as
when you configure your e-mail reader:
from address the e-mails will use (usually your own so that they can respond to you);
the outgoing (SMTP) e-mail server in the form smtp://foo.bar:587;
and the user and password for accessing that server.
Gamelab won't work for non-secure (i.e., non-TLS) e-mail access.
When you're finished entering this information, test the configuration.
You should receive an e-mail in a few minutes from the gamelab server.
Next, the experiment itself.
Start with a set of normal form bimatrix games in the
These can have any number of strategies and either positive integer or rational payoffs.
Enter these games into the
Bimatrix games are laid out top-left to bottom-right.
So let's say you have the following game shown first as a symmetric game then with the
implied payoffs to the opponent player.
|1, -1||0, 0||-1, 1|
|0, 0||-1, 1||1, -1|
|-1, 1||1, -1||0, 0|
These would decompose into 1 -1 0 0 -1 1 0 0 -1 1 1 -1 -1 1 1 -1 0 0, where opponent payoffs are coloured.
Next, specify a set of participants identified by e-mail address in the
Identifiers may be separated by white-space (newlines, tabs, spaces).
You'll see valid identifiers noted after submitting the form.
If you specify
Allow captive participants, participants can register themselves by
accessing the playerautoadd.html URL from the moment
you submit the form, so you may see participants appear as they add
They are captive in the sense that their logout button is disabled and that they are
never e-mailed (their identifiers are free-form, not e-mail addresses), nor are they
retained if the database is wiped.
Captive mode is disabled when you start an experiment:
this is a security measure so that participants can't join after you've started.
You can opt to allow participants to continually join by specifying that captive mode should
be preserved when the experiment begins.
Once you've added your games and your participants, you're ready for the experiment configuration as
Start with the simple options (don't click on
Show Advanced Options yet.)
Begin with the time and date when the experiment begins, the
number of rounds, duration of each round, and a fraction of participants that will cause the round to
advance before the maximum time has elapsed.
(You can set this to zero to force the maximum time to elapse before the round advances.)
For example, if set to 50%, then 50% or more participants playing all games in both roles will cause
the round to advance regardless the amount of time left.
If there are zero participants in either player role, then it's as if nobody plays, making the
round run until the maximum time elapses.
The minimum time per round is one moment; the minimum number of rounds is one round.
The advanced options allow for finer control—be careful in using these! See the Mechanical Turk section for adding Mechanical Turk workers to your experiment.
The login URL is used when sending non-captive participants the welcome e-mail. Their query string credentials will be appended to the given URL. You don't want to change this unless, for example, the laboratory CGI script is sitting in another URL path.
You can customise the instructions shown to participants by choosing from a set of pre-made instructions or providing your own. If you provide your own, it's up to you to make sure that the instructions are a well-formatted HTML5 fragment. Review the existing instructions on how to customise.
You can customise round advancement in several ways.
The first is limiting the total number of rounds per participant.
Thus, if one specifies 10 total rounds and 5 rounds per participant, each participant will play
at most five rounds.
This is useful for running with captive participants that can join and participant over an extended
period of play time.
The minimum-time (in minutes) establishes a
grace time for play.
This is especially useful when <100% of participants play before the round automatically
advances to prevent
fast participants from having an unfair advantage.
Lastly, the maximum limit per round bounds the number of simultaneous participants.
There are a number of option toggles in the last subsection.
You can disable the lottery computation, which is useful for Mechanical Turk scenarios, so that participants are only
shown their payoffs and not ranked and assigned lottery numbers.
The questionnaire is a more significant option: if specified, participants are required to step
through a series of questions before playing.
Note: if you specify a questionnaire, participants will not automatically be
joined to the experiment when it begins.
In other words, until they finish the experiment, they will not be included in the count of
current participants when looking at the fraction of completed game-plays per round.
They join the round following the current one upon finishing the questionnaire.
Lastly, you may specify that round advancement will trigger an e-mail to the participants.
Once you've deployed your experiment, you must now administer it. You can enable and disable participants, reset participant passwords, and reconfigure your experimenter credentials. You can also review some statistics on the current experiment status.
If a participant forgets her password, you can reset it by clicking on the participant's email
address and selecting the
Reset Password option.
(Captive participants are not affected.)
You can reset all non-captive participants' passwords with the
Reset All Passwords option.
If a participant is misbehaving, you can disable her login by toggling the button next to her
Once toggled, she'll no longer be able to log in.
If she's already logged in, she'll be logged out; no plays are accepted.
If you misconfigured the e-mail server and no e-mails were sent out, you can retry the failed
e-mail attempts by re-configuring (correctly!) the server and selecting
When an experiment has finished and you have provided a lottery amount, you can compute its lottery by specifying a seed for the random number generator and a number of lottery winners.
Once the winner has been chosen and notified, and the game wound down, you can wipe and start
afresh by selecting the
Wipe Experiment option.
This will first back up the database (e-mailing it to you in the process), then wipe all by the
experimenter credentials, listed games, and participant identities (though not their play, of
Note that captive participants are not retained if the database is wiped.
Mechanical Turk is a system for allowing human workers to participant in Internet-based tasks. It is part of the Amazon Web Services product constellation. This section assumes a bit of familiarity with the Mechanical Turk system overall.
Setting up a Mechanical Turk experiment is very easy.
First, have your AWS access key identifier and secret key handy.
Fill these in when configuring your experiment, then set the number of workers
(corresponding to the
max assignments in the API
the reward amount (
reward in the API), and
currency conversion (used when computing bonuses).
You can use either the Developer
Sandbox or production mode—I recommend starting with the first.
You can also enter a name and a description, which will be shown to potential workers, as well
as the keywords which are used when searching for HITs.
Lastly, you can set some worker requirements.
You can specify an assignment approval percent, number of accepted HITs, and locale.
By default, there are no requirements.
You'll probably also want to use the Mechanical Turk instructions, and you should enable the questionnaire to control for workers' understanding of game-play.
When you start your experiment, the request will be sent to either the sandbox or production server. We strongly suggest that you use the sandbox server a few times to make sure that your HIT posts as you wish it to. If there are errors, you'll be notified of them.
Note: when Mechanical Turk workers join the system, their session information is embedded in the query string. This is because most browsers disallow third-party cookies by default. So be warned that this is not a secure method of operation!
You can optionally assign bonuses to Mechanical Turk players in your experiment after the experiment has concluded.
To do so, you'll need to click the
Assign bonus button that appears beneath the high-scorer table on the experimenter page.
This button is only shown for Mechanical Turk experiments.
It's ok to click this twice: the Mechanical Turk system will keep track of who has been awarded
bonuses and not award the same bonus twice.
Bonuses are only awarded to Mechanical Turk players who have indicated completion. You can see which players have done so by looking whether they have checkmarks next to their identifiers in the player listing.
The bonus assigned is the number of tickets times the conversion rate.
At this time, gamelab doesn't have any special functionality for analysing played (or in-flight) experiments. You can, however, directly access the SQLite game database by backing up on the experimenter console. This will e-mail a redacted database to the experimenter. The redacted bits consist of user and experimenter e-mail passwords. The database is in WAL mode.
The database schema is fairly straightforward.
It consists of the following tables.
In all of these, note that
Unique Identifier fields are non-zero SQLite
Rational numbers are stored as text, such as 1/2 or just 2, starting with
You can also see a detailed map of the schema.
As described in the Technicalities section, gamelab is at heart a web application making JSON requests of a server.
This JSON interface is simple and easy to use to create a programmatic interface.
If you wish to create a programmatic interface (a
bot), and your experimenter consents, it's a
fairly easy process.
In fact, a tool already exists within the gamelab source distributions to stress
test systems using this interface.
To do this, you'll need a programming language that has support for JSON and making HTTP requests. In C, you can use json-c and libCURL, respectively.
Note to developers: if you're writing your own bot, do not flood requests. Be courteous to other users: there is no competitive advantage to be gained by polling the system constantly.
Note to experimenters: in order to run a programmable experiment, you cannot use a Questionnaire, as questions are free-form. In theory, you could enable the questionnaire and have players play through it, then turn on their bots, but in practise participants knowledgeable enough to deploy bots won't need the questionnaire!
The general sequence is as follows:
- Get your username and password, or optionally, have your utility register with doautoadd.json as a captive player and get a randomly-generated password.
- Wait for the experiment to accept players by polling dologin.json in order to login.
- Log in to dologin.json with your username and password, accepting your session identifier as cookies.
- Wait for the experiment to begin by polling doloadexpr.json. Use your session identifier as an identifier cookie.
- Accept the experiment data from doloadexpr.json. Use your session identifier as an identifier cookie.
- Using the experiment data from doloadexpr.json, construct and submit a series of game plays to doplay.json. Use your session identifier as an identifier cookie.
- Wait for the next round, or end of game, by polling polling doloadexpr.json. Use your session identifier as an identifier cookie.
Do not flood the system with requests: be considerate! Unless specifically stipulated by the experimenter, you are not rewarded for fast play. Avoid re-connecting with anything less than 30–60 seconds between attempts, unless specifically allowed by an experimenter, or your bot may be banned from the experiment.
This section describes the resources (URLs) that may be accessed for programmatic play. Each of these resources is the immediate path following the laboratory CGI script name, for example, if you're running as localhost/lab.cgi, these would be mapped to localhost/lab.cgi/doautoadd.json.
- doautoadd.json (POST)
Post your e-mail address using the
identfield name. Returns HTTP error code 400 (invalid or missing e-mail field), 403 (player by that e-mail already exists), 404 (captive players disallowed), or 200 (success). Upon success, the return body is a JSON object consisting of the
- dologin.json (POST)
Post your e-mail address (or identifier) and password using the
passwordfield names. Returns HTTP error code 400 (invalid or missing input fields), 409 (experiment not started yet), 200 (success). If you receive HTTP error code 409, wait and continue polling: the experiment is not yet accepting player logins. Upon success, the return body is empty, but the cookie field consists of
sessid, your session identifiers (a number); and
sesscookie, a magic number attached to your identifier.
- doloadexpr.json (GET)
Get a doloadexpr object describing the full experiment. You will need to pass the
sesscookiecookie fields as described in dologin.json. Returns HTTP error code 304 (not changed since last request), 409 (experiment not started yet), 429 (player accepted, but not yet joined for play), 200 (success). If you receive HTTP error code 409 or 429, wait and continue polling: the experiment is not yet started or has started, but you are not allowed to join yet. Upon success, returns the JSON experiment object.
- doplay.json (POST)
Post a probability mixture for a game's actions. You will need to pass the
sesscookiecookie fields as described in dologin.json. The game identifier and round must be encoded in the
roundfields, respectively. Each action with a requested probability must be posted as field
Nis the action index (starting at zero), e.g.,
index1, and so on. Probabilities may be decimal or fractional strings. Missing actions are assigned zero probability. Actions greater than the maximum action are ignored. Returns HTTP error code 400 (invalid game, round, or probability, or probability sum doesn't equal one), 409 (round has passed or is being played again, experiment hasn't started or has finished, or player hasn't joined), or 200 (success). Note: the error codes for this resource are likely to become more specific in future releases.
When using the Programmable API via the web Resources, you'll need to examine some JSON objects to ascertain the experiment status. The following defines the objects available to you by resource. The type of each field is listed alongside the field name: object for dictionaries (i.e., things that contain other things), strings being UTF-8 strings, integers being 64-bit signed integers, doubles being double-precision floating points, and times being seconds since UNIX epoch.
There are many undocumented fields that are transmitted while playing: these are to be considered unstable and will not be included in subsequent releases.
Obtained from a successful doloadexpr.json request, i.e., with an HTTP error code 200 and a parsed JSON object. The resulting dictionary consists of the following top-level fields.
- expr (object)
Object describing the experiment status. This object is always non-null. This contains more or less all the information you'll need.
- history (object)
- Additional history. See the history object.
- instr (string)
- Instructions for participating in the experiment. This is an HTML5 fragment describing how to use the experiment. It is generally formatted in the web interface, so the styling may make the data non-obvious.
- minutes (integer)
- The maximum number of minutes per round (all games). This is the maximum: in reality, rounds may range from roundmin to this value.
- prounds (integer)
- The total number of rounds played by the logged-in participant. The joined round plus this is the participation span for any participant.
- round (integer)
- The current round number starting at zero. (If the round is less than zero, the experiment has not yet begun.)
- roundbegan (time)
- When the current round began.
- roundmin (integer)
The minimum amount of time, in minutes, per round.
This is also called the
grace time. If this is non-zero, players are given this much time to play freely until the round will advance based upon the player fraction (see roundpct).
- rounds (integer)
- Total number of rounds in the experiment. Note: you may not actually play this number of rounds. See prounds.
- start (time)
- Time when the experiment started (or will start).
- player (object)
Information about the current participant. This object is always non-null.
- joined (integer)
- The round when the current participant may first play. If this is less than the current round, the participant has been accepted into the system, but is not yet playing.
- role (integer)
- Whether the participant is playing the row (zero) or column (non-zero) role in the game bimatrices.
- history (array)
The game history and information about individual games. This is an array of objects with the following fields.
- id (integer)
- The identifier of this game.
- p1 (integer)
- Number of actions for the row player.
- p2 (integer)
- Number of actions for the column player.
- payoffs (array)
- Payoff matrix. This is an array (of length p1 of arrays (each of length p2) of strings. Each string is a rational number formatted as a fraction (e.g., 1/2, -1/2, 0, 1, etc.). Clients should use the GMP library to work with these rational numbers.
- roundups (array)
An array of prior play for this game, indexed by round. If this is set to an empty-length array, the experimenter might be prohibiting the display of history, or there may not yet be a history to display given the current round number.
- skip (integer)
- Non-zero if there weren't enough plays in the given round to compute payoffs. Zero otherwise.
- navgp1 (array)
- An array of doubles representing the average weight of this action for the row player role. The array if of length p1.
- navgp2 (array)
- An array of doubles representing the average weight of this action for the column player role. The array if of length p2.
- navgs (array)
- An array of arrays consisting of the multiples of the average fraction for the given row and column player cell.