------------------------------------------------------------------------------
 INTRODUCTION
------------------------------------------------------------------------------

  This directory contains the scripts and test cases for developing both the
Aeneas compiler and associated libraries. While Aeneas and supporting libraries
are written in pure Virgil code and are thus self hosted, the testing
infrastructure needs to run on platforms that do not yet have a fully functioning
Virgil port. Otherwise the entire testing infrastructure would itself be written
in Virgil (!). As such, it is mostly written in bash with some supporting utilities
that need to be written in C.

------------------------------------------------------------------------------
 CONFIGURING THE TEST PLATFORM
------------------------------------------------------------------------------

  A script is included in this repository,

    ./test/configure

that tries to determine the host platform and sets up symlinks in the
{test/config/} directory so that the rest of the tests can just invoke
executables and scripts linked there. This script is invoked automatically
if needed by the top-level test script.

------------------------------------------------------------------------------
 RUNNING TESTS
------------------------------------------------------------------------------

  The main top-level script,

    ./test/all.bash

or invoking the command {aeneas test}, which simply executes this script, is
the main way to invoke the testcases for the compiler. It senses the host
platform and configures a number of settings, and then invokes test scripts
that are in several subdirectories below. By default, this script will always
build Aeneas from the source in this repo using the stable compiler. This is
usually what you want. In this directory,

   ./test/codegen.bash  - runs a smaller set of codegen tests
   ./test/asm.bash      - runs a smaller set of assembler tests
   ./test/diagnose.bash - helps streamline debugging of a single test
   ./test/common.bash   - contains common configuration code, no need to run

It doesn't matter from which directory you run these tests. They figure out
their own directory automatically and can find the repository root from there.
No need for environment variables, or even to have ./bin in your path.

Below the main directory, sub directories are dedicated to testing particular
parts of the language, compiler, runtime system, or libraries. You can
generally run these standalone if you are in the right directory.

   (cd ./test/parser && ./test.bash)
   (cd ./test/seman && ./test.bash)
   (cd ./test/execute && ./test.bash)

------------------------------------------------------------------------------
 ENVIRONMENT VARIABLES
------------------------------------------------------------------------------

  By default, the scripts in the test directory try to figure out where they
are, what platform they are being run on, which version of the compiler to
test, where and if there is a test cache, etc. However, these can be
overridden to narrow the tests down.

  TEST_HOST=  	     Override the assumed host platform
  TEST_TARGETS=	     Override the set of targets to test
  TEST_CACHE=	     Override the place where test binaries are cached
  V3C_STABLE=	     Override the stable compiler version used to start up
  V3C_OPTS=	     Add additional compiler options (e.g. optimizations)
  AENEAS_TEST=	     Change the compiler under test

You may not want to wait the entire < 1 second for the stable compiler to
bootstrap the current sources. In that case, you can set the AENEAS_TEST
environment variable to point to ./bin/dev/v3c-dev and the tests will then
run the current compiler in the stable compiler's interpreter. Or you might
have already bootstrapped or otherwise have a binary that you can use.

The TEST_CACHE allows you to compile tests for a target platform which you
cannot run, but compare their binaries against the cache. This allows, e.g.
testing that the current compiler generates the same binaries as the stable
compiler, ensuring these tests would pass if it were possible to run them.

------------------------------------------------------------------------------
 NARROWING THE TESTS RUN
------------------------------------------------------------------------------

  You may want to use the environment variables to narrow the testcases that
are run by one of the above scripts, or for these scripts:

  ./test/all.sh [directories]

  ./test/codegen.sh [target [directories]] 
  
To limit the targets or subdirectories run. Additionally when using
AENEAS_TEST= with the {all.sh} script, it will not bootstrap from source.
This allows you to achieve a very short (< 1 second) turn around time when
developing a particular feature, optimization, target, or debugging.

  Generally, most subdirectory tests also support specifiying the tests to
run as a list of {.v3} files on the command line.

  (cd parser && ./test.bash a*.v3)

------------------------------------------------------------------------------
 SUPPORTING UTILITIES
------------------------------------------------------------------------------

  A few supporting utilities are needed to get off the ground:

  progress	   	     Processes a standard text format from tests and can
    (C)		   	     format the result for pretty organized output

  execute-${target}-test     Forks/execs generated native tests and checks
    (C)			     their output against expectations

  ./bin/dev/sense_host	     Tries to determine the host platform from UNIX
    (bash)		     commands

  btime			     A benchmarking utility to get precise execution times
    (C)

  To bring up Virgil on a new platform, these utilities must first be ported, so
that tests themselves can be run. For example, to port Virgil to $newcpu-linux,
we can generally reuse the C programs written against POSIX. The bash scripts
should be portable, so we simply need a JVM to use "jar" as the host platform.