The Astro CLI is the modern command-line interface for data orchestration. It is the easiest way to run Apache Airflow on your local machine. With the Astro CLI, you can both develop and test DAGs locally as well as interact with our Astro and Astronomer Software offerings.
astro is a command line interface for Airflow working within the Astronomer Cloud or Software
Usage:
astro [command]
Available Commands:
completion generate the autocompletion script for the specified shell
config Manage project configuration
context Manage Astro & Software contexts
deploy Deploy your project to a Deployment on Astro
deployment Manage your Deployments running on Astronomer
dev Run your Astro project locally
help Help about any command
login Log in to Astronomer
logout Log out of Astronomer
version astro version
workspace Manage Astronomer Workspaces
Flags:
-h, --help help for astro
Use "astro [command] --help" for more information about a command.
To install and use the Astro CLI on Mac, you must have:
- Docker Desktop (v18.09 or higher)
To install and use the Astro CLI on Linux, you must have:
- Docker Engine (v0.13.1 or higher).
To install and use the Astro CLI on Windows, you must have:
- Windows 10
- Docker Desktop WSL 2 backend (v0.13.1 or higher).
brew install astro
curl -sSL install.astronomer.io | sudo bash -s
To install a specific version Astro CLI use @major.minor.patch. For example, to install v1.0.0 run:
brew install astronomer/tap/[email protected]
To install a previous version of Astronomer, you can add the tag at the end of your curl
command via the following syntax:
curl -sSL https://install.astronomer.io | sudo bash -s -- [TAGNAME]
ie:
curl -sSL https://install.astronomer.io | sudo bash -s -- v1.0.0
Note: If you get mkdir error during installation please download and run godownloader script locally.
$ cat godownloader.sh | bash -s -- -b /usr/local/bin
-
Download the latest release of the Astro CLI from this page into your project
-
Copy
astro.exe
somewhere in your%PATH%
-
Open cmd or PowerShell console and run the
astro version
command. Your output should look something like this:C:\Windows\system32>astro version Astro CLI Version: x.y.z
- Make sure you go through instruction to install Docker on windows properly https://docs.docker.com/docker-for-windows/install/
- Install Windows WSL2 and Docker Desktop for WSL 2 Backend which is the recommend way to run the Astro CLI on Windows.
- Alternativly, you can run the Astro CLI natively by enabling Hyper-V, which is required for Docker and Linux Containers. Review this document https://docs.docker.com/docker-for-windows/troubleshoot/ if you have issues.
- Create a project:
$ mkdir hello-astro && cd hello-astro
$ astro dev init
- Install the binary and Confirm the install worked:
$ ./astro
This will generate a skeleton project directory:
.
├── dags
│ ├── example_dag_advanced.py
│ ├── example_dag_basic.py
├── tests
│ ├── dags
│ ├── test_dag_integrity.py
├── Dockerfile
├── include
├── packages.txt
├── plugins
└── requirements.txt
DAGs can go in the dags
folder, custom Airflow plugins in plugins
, python packages needed can go in requirements.txt
, and OS level packages can go in packages.txt
.
- Start airflow
Run astro dev start
to start a local version of airflow on your machine. This will spin up a few locally running docker containers - one for the airflow scheduler, one for the webserver, and one for postgres.
(Run astro dev ps
to verify)
The CLI includes a help command, descriptions, as well as usage info for subcommands.
To see the help overview:
$ astro help
Or for subcommands:
$ astro dev --help
$ astro deploy --help
How to get started as a developer:
-
Install
go
1.18+ - https://go.dev/doc/install -
Install
golangci-lint
to run linter locallybrew install golangci-lint
golangci-lint run .
-
Install
pre-commit
to run lint on every commitbrew install pre-commit
pre-commit install
Run lint locally:
pre-commit run --all-files
-
Clone and Build:
$ cd $GOPATH/src/github.com/astronomer/astro-cli
$ git clone [email protected]:astronomer/astro-cli.git
$ cd astro-cli
$ make build
The Astro CLI is a single command-line interface that allows users to interact with both Astro and Astronomer Software.
In order to test cloud locally you will need to update your global or local config to point to right platform type and local astro endpoint. Ex.
local:
platform: cloud
astro: https://localhost:8871/v1
Similarly in order to test software locally you will need to update to right platform type and local houston endpoint:
local:
platform: software
houston: https://localhost:8871/v1
To run unit-tests you can run:
Note: Make sure you have running locally houston or Astro on https://localhost:8871/v1 it is required for running some tests
make test
We currently use mockery to generate mocks for interfaces. Installation guide for mockery: https://github.com/vektra/mockery#installation
Steps to regenerate already existing interface mocks:
- Run
make mock
.
Steps to generate mocks for new interface:
- Run
mockery --filename=<file_name_where_interface_is_present> --output=<output_dir_to store_mocks> --dir=<directory_where_to_search_for_interface_file> --outpkg=<mock_package_name> --name <name_of_the_interface>
to generate mock for an interface. - Add the above command in appropriate target under
mock
rule inMakefile
.
The Astronomer platform is under very active development. Because of this we cannot make backwards compatibility guarantees between versions.
The astro-cli is following a semantic versioning scheme
{MAJOR_RELEASE}.{MINOR_RELEASE}.{PATCH_RELEASE}
with all releases up until 1.0.0 considered beta.
astro-cli
tightly tracks the platform release versioning, this means that compatibility is only guaranteed between matching minor versions of the platform and the astro-cli. ie. astro-cli v0.9.0
is guaranteed to be compatible with houston-api v0.9.x
but not houston-api v0.10.x
astro-cli
has a debug flag that allows to see queries and different helpful logs that are done internally. You can enable this by passing --verbosity=debug
to your commands or you can edit your ~/.astro/config.yaml
and add the following to it. This will turn on debug for all requests that are done until it is changed to info
or removed from config.
verbosity: debug
If you need support, start with the Astronomer documentation.
If you still have questions, you can post on the Astronomer web forum or if you are a customer, you can open a support ticket.
Apache 2.0 with Commons Clause