Create simplest, running, Azure Batch kick-off script

[natemcintosh]

Goal

Create a single python script to configure and run the model in Azure Batch.

Context

This will eventually be the primary script used for production runs. For now though, it should just get something super basic up and running in Azure Batch.

Required features

• Use a certain pre-made pool in Azure
• Run a few models in parallel, e.g. multiple states, multiple diseases
• Run a post production task, that is dependent on all the model runs completing.
• Make sure all output is saved to blob storage
• Be able to kick it off as either a poetry script or a make command. Want to be able to run this from the command line locally, as well as run it in an automated fashion from CI (at a later point in time).

Specifications

• Have good command line help and documentation
• Read in a single primary configuration file
• Start a single main job
• For each model to run, kick off a task, giving it a properly formed:
  • Docker command, including bind mounts
  • Config file to run on
• Create a post production task that depends on all model runs finishing. Should have properly formed:
  • Docker command, including bind mounts
  • Config file to run on

Related documents

• None that I can think of