To test:
cd test
pytest -v
To run using yaml files (preferred way):
python run_hydra.py --config_file
To run locally with json file:
python run.py --config_file {path to json}
1- Setup AWS CLI and make sure you're authenticated:
>aws2fa 123456
>Done!
2- Run job: Make sure the experiment name contains only characters, numbers and dashes (AWS requirement)
python run_aws.py --config-path conf/file_name.yaml
To connect to tensorboard from your local machine / or notebook
F_CPP_MIN_LOG_LEVEL=3 tensorboard --logdir s3:https://location_of_experiment_root
1- setup environment variable for azure subscription id in .bash_profile by adding the following line (replace with actual subscription id)
export AZ_SUBS_ID=abdabdabdabd
2- run code:
python run_az.py --config_file {path to json}
# You can change vm type or disable GPU:
python run_az.py --config_file {path to json} --vm_size "STANDARD_NC6" --disable_gpu
3- to run multiple experiments.json files in a specific directory:
submit_multiple.sh run_azure.py path_to_directory_with_multiple_json_files
{
"root_path": "/Users/jkhouja/workspace/experiments/arabic_media/", #This path will be loaded to azure storage and will be the parent of the experiment
"experiment_name": "claim_verification", # This will be the experiment name in azure with suffix _azure. Will be saved under root. All runs will be directories with dates/time under this experiemnt
}