A command-line interface (CLI) for working with fiboa.
In order to make working with fiboa easier we have developed command-line interface (CLI) tools such as inspection, validation and file format conversions.
You will need to have Python 3.9 or any later version installed.
Run pip install fiboa-cli
in the CLI to install the validator.
Optional: To install additional dependencies for specific converters,
you can for example run: pip install fiboa-cli[xyz]
with xyz being the converter name.
Note on versions:
- fiboa CLI >= 0.3.0 works with fiboa version > 0.2.0
- fiboa CLI < 0.3.0 works with fiboa version = 0.1.0
After the installation you should be able to run the following command: fiboa
You should see usage instructions and available commands for the CLI.
fiboa CLI supports various commands to work with the files:
To validate a fiboa GeoParquet or GeoJSON file, you can for example run:
- GeoJSON:
fiboa validate example.json --collection collection.json
- GeoParquet:
fiboa validate example.parquet --data
Check fiboa validate --help
for more details.
The validator also supports remote files.
https://
orhttps://
: no further configuration is needed.s3:https://
:s3fs
needs to be installed (runpip install .[s3]
) and you may need to set environment variables. Refer here for how to define credentials.gs:https://
:gcsfs
needs to be installed (runpip install .[gcs]
). By default,gcsfs
will attempt to use your default gcloud credentials or, attempt to get credentials from the google metadata service, or fall back to anonymous access.
To create a fiboa-compliant GeoParquet for a fiboa-compliant set of GeoJSON files containing Features or FeatureCollections, you can for example run:
fiboa create-geoparquet geojson/example.json -o example.parquet -c geojson/collection.json
Check fiboa create-geoparquet --help
for more details.
To create one or multiple fiboa-compliant GeoJSON file(s) for a fiboa-compliant GeoParquet file, you can for example run:
- GeoJSON FeatureCollection:
fiboa create-geojson example.parquet -o dest-folder
- GeoJSON Features (with indentation and max. 100 features):
fiboa create-geojson example.parquet -o dest-folder -n 100 -i 2 -f
Check fiboa create-geoparquet --help
for more details.
To look into a fiboa GeoParquet file to get a rough understanding of the content, the following can be executed:
fiboa describe example.parquet
Check fiboa describe --help
for more details.
To create a JSON Schema for a fiboa Schema YAML file, you can for example run:
fiboa jsonschema example.json --id=https://fiboa.github.io/specification/v0.1.0/geojson/schema.json -o schema.json
Check fiboa jsonschema --help
for more details.
To validate a fiboa Schema YAML file, you can for example run:
fiboa validate-schema schema/schema.yaml
Check fiboa validate-schema --help
for more details.
Once you've created and git cloned a new extension, you can use the CLI to update all template placeholders with proper names.
For example, if your extension is meant to have
- the title "Timestamps Extension",
- the prefix
ts
(e.g. fieldts:created
orts:updated
), - is hosted at
https://github.io/fiboa/timestamps-extension
(organization:fiboa
, repositorytimestamps-extension
), - and you run fiboa in the folder of the extension.
Then the following command could be used:
fiboa rename-extension . -t Timestamps -p ts -s timestamps-extension -o fiboa
Check fiboa rename-extension --help
for more details.
The CLI ships various converters for existing datasets.
To get a list of available converters/datasets with title, license, etc. run:
fiboa converters
Use any of the IDs from the list to convert an existing dataset to fiboa:
fiboa convert de_nrw
See Implement a converter for details about how to
To install in development mode run pip install -e .
in this folder.
For the tests first run pip install -r requirements-dev.txt
to install pytest.
Then you can run pytest
to execute the tests.
The following high-level description gives an idea how to implement a converter in fiboa CLI:
- Create a new file in
fiboa_cli/datasets
based on thetemplate.py
- Implement the
convert()
function / test it / run it - Add missing dependencies into a separate dependency group in
setup.py
- Add the converter to the list above
- Create a PR to submit your converter for review
An in-depth guide how to create a cloud-native fiboa dataset using fiboa CLI is available at: https://github.com/fiboa/data/blob/main/HOWTO.md