Soaring weather forecast web distribution
Bundles two Google Cloud Functions, triggered by HTTP calls :
- launch_instance : Spins up a Google Compute Engine instance, and starts a specific docker container on it ;
- delete_instance : Stops and deletes a specific instance.
$ gcloud functions deploy launch_instance --runtime python37 --trigger-topic launch-instance
$ gcloud functions deploy delete_instance --runtime python37 --trigger-topic delete-instance
$ gcloud beta scheduler jobs create pubsub compute-launch \
--schedule '0 5 * * *' \
--topic launch-instance \
--message-body '{"image": "raspit-compute", "zone": "us-east1-b", "instance_type": "n1-highcpu-8", "env": {"GCS_BUCKET": "raspit-compute", "PUBSUB_TOPIC": "run-finished"}}' \
--time-zone 'Europe/Paris'Docker image used to post-process raw WRF files. It fetches WRF files from a Google Cloud Storage bucket, generates one JSON document per WRF file, containing variables we want to extract, and push them to Google Cloud Datastore.
$ gcloud builds submit -t gcr.io/<gcp_project_id>/raspit-forecast-service .
$ gcloud beta run deploy --image gcr.io/raspit-248118/raspit-forecast-service --platform managed --region europe-west1 --update-env-vars GCS_BUCKET=raspit-compute --memory=1GiYou have to generate a service account JSON key from the IAM GCP console first.
$ docker run -it -e GOOGLE_APPLICATION_CREDENTIALS="/key.json" -v $(pwd):/src/ -v <path_to_json_key>:/key.json --rm gcr.io/<gcp_project_id>/raspit-forecast-service:latest bash
# python wrf_to_json.py --bucket-name <bucket_name> --prefix <prefix>WRF-ARW Weather forecast compute image for SW France. Include 2 docker images :
- Docker-rasp-wrfv3/ : Generic wrfv3 rasp dockerfile (for development purpose) ;
- Docker-raspit-compute/ : "Production" image, derived from the above. Does a run for the next 3 days upon startup, transfer the data generated to Cloud Storage and kills the host (via a raspit-cloud-driver call) when done.
Uses two different domains :
- PYR2/ : 1-stage 0.5deg GFS initialized 6km grid, for days n+2 and n+3 ;
- PYR3/ : 1-stage 0.25deg GFS initialized 3km grid, for day n+1.
$ cd raspit-compute/Docker-rasp-wrfv3
$ wget http:https://rasp-uk.uk/SOFTWARE/WRFV3.x/raspGM.tgz
$ wget http:https://rasp-uk.uk/SOFTWARE/WRFV3.x/raspGM-bin.tgz
$ wget http:https://rasp-uk.uk/SOFTWARE/WRFV3.x/rangs.tgz
$ wget https://github.com/WRF-CMake/WPS/releases/download/WPS-CMake-4.0.2/wps-cmake-4.0.2-serial-basic-release-linux.tar.xz
$ docker build -t rasp-wrfv3 .
$ cd ../Docker-raspit-compute
$ docker build -t gcr.io/<gcp_project_id>/raspit-compute .
$ docker push gcr.io/<gcp_project_id>/raspit-compute$ cd raspit-compute/Docker-raspit-compute
$ docker run --rm -it \
-e "DEV_ENV=y" \
-v $(pwd):/root/rasp/Raspit-compute-image \
rasp-wrfv3 bash
Google App Engine flask-based webserver. Also includes an App Engine cron job, that calls the /compute-launch (see below) route daily.
- / : User endpoint ;
- /forecast : Returns JSON forecast data fetch from Google Cloud Datastore ;
- /compute-launch : Triggers a forecast run, via a launch_instance Cloud Function call.
$ cd raspit-web
$ gcloud app deploy
$ gcloud app deploy cron.yaml
- Dr Jack : RASP, a WRF-ARW forecast distribution ;
- V. Mayamsin : Rasp docker scripts