AGI agent simulator.

Koen Holtman, August 2019, March 2020.

This software is Open Source, licensed under the Apache License version 2.0.

The paper in this archive can be redistributed under the Creative
Commons CC BY-ND 4.0 license.  Please contact the author if you would
like to obtain broader rights.

** Introduction

This AGI agent simulator was written together with the paper
'Corrigibility with Utility Preservation', also stored in this
repository.

See the paper for more details on the agent and simulation model.

In the standard configuration, the simulator (the version tagged v1.2)
will run a set of simulations to produce the figures in the paper.

** Running the simulator

The simulator is implemented as a set of AWK scripts.  AWK is a
scripting language that comes pre-installed on most Unix systems.

The shell script file file 'sim' contains the AWL invocation needed to
run the simulator.  To run the simulator, copy all files to a
directory, and run sim, or an equivalent AWK invocation for your
system.  When run, the simulator will show output on stdout, and will
also write several .tex files containing figures for the paper.

While the sim file calls 'gawk', the GNU implementation of AWK, the
simulator can also be run using other modern implementations of the
AWK language.

To experiment with different parameter combinations for simulation
runs. edit the params.awk file and run the simulator again.

** Overview of the code

The agisim.awk file has the main simulation code.  The condeutil.awk
file has extra simulation code used in appendix A of the paper. The
params.awk file has code that invokes the simulator with various
parameters.  Not all invocations are enabled by default, edit the
dotestruns() function in params.awk to enable a different set.

See the DETAILS file for more detailed remarks about the code.

** Questions, comments, and contributions

Questions, comments, and contributions can be sent to the author at
[email protected], or submitted to GitHub, e.g. with the pull
request function.

If there is interest, the author might do some refactoring to clean up
the code, or write additional documentation.

** Version history

The version tagged v1.2 is the first version, used for the simulations
in the August 2019 paper 'Corrigibility with Utility Preservation'.

The v2 version adds some bug fixes, none which materially change the
simulation results in the V1 version of the paper above.  As one fix,
a floating point inaccuracy workaround was added, that arguably made
the v1.2 version sometimes do the button press one time step too late.
Also in V2, several additional simulation modes and related simulation
parameter settings in params.awk were added, with some documentation
inline in the source files.