Skip to content

Latest commit

 

History

History
327 lines (235 loc) · 10.8 KB

INSTALL.md

File metadata and controls

327 lines (235 loc) · 10.8 KB
Raw

Getting started

This document describes how to manually configure your system for running CyclOSM.

Requirements

You will need, to use this style:

  • PostgreSQL
  • PostGIS
  • osm2pgsql

On Ubuntu/Debian, these can be installed with

sudo apt-get install postgresql-9.6 postgis osm2pgsql

OpenStreetMap data

You need OpenStreetMap data loaded into a PostGIS database. These stylesheets expect a database generated with osm2pgsql using the pgsql backend (table names of planet_osm_point, etc), the default database name (osm).

Start by creating the database

sudo -u postgres createuser -s $USER
createdb osm

Enable PostGIS and hstore extensions with

psql -d osm -c 'CREATE EXTENSION postgis; CREATE EXTENSION hstore;'

then grab some OSM data. It's probably easiest to grab an PBF of OSM data from Geofabrik. Once you've done that, import with osm2pgsql:

osm2pgsql -c -G --hstore -d osm ~/path/to/data.osm.pbf

Shapefiles

This style requires some preprocessed shapefiles for some features. They are listed in the project.mml file. You can either download them manually or rely on Kosmtik to handle this for you for a developement setup (see below).

Fonts

The stylesheet uses Noto, an openly licensed font family from Google with support for multiple scripts. The stylesheet uses Noto's "Sans" style where available. If not available, this stylesheet uses another appropriate style of the Noto family. The "UI" version is used where available, with its vertical metrics which fit better with Latin text.

DejaVu Sans is used as an optional fallback font for systems without Noto Sans. If all the Noto fonts are installed, it should never be used. Noto Naskh Arabic UI is used an an optional fallback font for systems without Noto Sans Arabic.

Hanazono is used a fallback for seldom used CJK characters that are not covered by Noto.

Unifont is used as a last resort fallback, with it's excellent coverage, common presence on machines, and ugly look.

Installation on Ubuntu/Debian

On Ubuntu 16.04 or Debian Testing you can download and install most of the required fonts

sudo apt-get install fonts-noto-cjk fonts-noto-hinted fonts-noto-unhinted fonts-hanazono ttf-unifont

Noto Emoji Regular can be downloaded from the Noto Emoji repository.

It might be useful to have a more recent version of the fonts for rare non-latin scripts. The current upstream font release has also some more scripts and style variants than in the Ubuntu package. It can be installed from source.

DejaVu is packaged as fonts-dejavu-core.

Installation on other operation systems

The fonts can be downloaded here:

After the download, you have to install the font files in the usual way of your operation system.

Non-latin scripts

For proper rendering of non-latin scripts, particularly those with complicated diacritics and tone marks the requirements are

  • FreeType 2.6.2 or later for CJK characters

  • A recent enough version of Noto with coverage for the scripts needed.

Elevation data

This part of the guide is heavily based on the very good documentation from OpenTopoMap. This part will guide you through generating the required data and import elevation lines into a PostgreSQL database for the render.

First, you should install gdal and geotiff tools. On Ubuntu/Debian, this can be done using

sudo apt-get install gdal-bin python-gdal geotiff-bin

We will also need phyghtmap to generate the elevation lines (contours). On Ubuntu/Debian, you can download the Debian package on their download page and run

sudo dpkg -i phyghtmap*.deb
sudo apt-get --fix-broken install  # To ensure missing dependencies are installed

We will be running everything from now on in the dem folder.

Download some elevation data

NASA provides elevation data from the Shuttle Radar Topography Mission (SRTM). The easiest way to download the data is probably to use viewfinderpanoramas.org. Download all the required tiles for the area you want covered and put them in the dem folder at the root of this repository.

Create hillshades

First, let us fill the voids in the HGT data and convert to TIFF files:

cd dem
# Unzip downloaded files
for zipfile in *.zip; do unzip $zipfile; done
rm *.zip
# Convert all HGT files to TIF
for hgtfile in */*.hgt; do gdal_fillnodata.py $hgtfile $hgtfile.tif && rm $hgtfile; done

Then, let us merge .tifs into one large .tif.

gdal_merge.py -n 32767 -co BIGTIFF=YES -co TILED=YES -co COMPRESS=LZW -co PREDICTOR=2 -o raw.tif */*.hgt.tif

The raw.tif is the full resolution DEM. This data will be passed through gdal to create the contours and hillshades. We should now reproject it into Mercator projection, interpolate and shrink:

gdalwarp -co BIGTIFF=YES -co TILED=YES -co COMPRESS=LZW -co PREDICTOR=2 -t_srs "+proj=merc +ellps=sphere +R=6378137 +a=6378137 +units=m" -r bilinear -tr 5000 5000 raw.tif warp-5000.tif
gdalwarp -co BIGTIFF=YES -co TILED=YES -co COMPRESS=LZW -co PREDICTOR=2 -t_srs "+proj=merc +ellps=sphere +R=6378137 +a=6378137 +units=m" -r bilinear -tr 1000 1000 raw.tif warp-1000.tif
gdalwarp -co BIGTIFF=YES -co TILED=YES -co COMPRESS=LZW -co PREDICTOR=2 -t_srs "+proj=merc +ellps=sphere +R=6378137 +a=6378137 +units=m" -r bilinear -tr 500 500 raw.tif warp-500.tif
gdalwarp -co BIGTIFF=YES -co TILED=YES -co COMPRESS=LZW -co PREDICTOR=2 -t_srs "+proj=merc +ellps=sphere +R=6378137 +a=6378137 +units=m" -r bilinear -tr 90 90 raw.tif warp-90.tif
# raw.tif is no longer needed
rm raw.tif

Note the gdalwarp arguments:

  • -co BIGTIFF=YES: if output > 4 GB
  • -co TILED=YES: intern tiles
  • -co COMPRESS=LZW -co PREDICTOR=2: lossless compression with prediction
  • -t_srs "+proj=merc +ellps=sphere +R=6378137 +a=6378137 +units=m": convert into Mercator
  • -r cubicspline: interpolation for tr < 90 m, bilinear for tr > 90 m
  • -tr 30 30: desired resolution in meters

We can now create hillshades for different zoom levels:

gdaldem hillshade -z 7 -co compress=lzw -co predictor=2 -co bigtiff=yes -compute_edges -combined warp-5000.tif hillshade-5000.tif
gdaldem hillshade -z 7 -co compress=lzw -co predictor=2 -co bigtiff=yes -compute_edges -combined warp-1000.tif hillshade-1000.tif
gdaldem hillshade -z 4 -co compress=lzw -co predictor=2 -co bigtiff=yes -compute_edges -combined warp-500.tif hillshade-500.tif
gdaldem hillshade -z 4 -co compress=lzw -co predictor=2 -co bigtiff=yes -compute_edges -combined warp-90.tif hillshade-90.tif

Note: gdaldem and gdalwarp have problems compressing huge files while generation. You can compress those afterwards by using gdal_translate -co compress=…

Finally, we can clean the now useless files:

rm warp-*.tif
# Delete all the extracted folders, no longer needed
find . ! -path . -type d | xargs rm -r

Create the contour lines database

Run phyghtmap and generate contours (this step can take quite a few tens of minutes):

wget http:https://download.geofabrik.de/europe/france.poly
phyghtmap --polygon=france.poly -j 2 -s 10 -0 --source=view3 --max-nodes-per-tile=0 --max-nodes-per-way=0 --pbf
# Remove now useless files
rm -r hgt
rm france.poly

Note: You can use any other polygon from Geofabrik. The previous command will generate elevation lines for the whole France area.

The output of this will be in a OpenStreetMap Protocolbuffer Binary Format called something like lon-6.30_9.90lat41.26_51.33_view3.osm.pbf.

We will load the contours into a database called contours. If a contours database exists already, you will need to drop it and recreate it first.

sudo -u postgres createdb contours
sudo -u postgres psql contours -c 'CREATE EXTENSION postgis;'

Load the data into the contours database:

sudo -u postgres osm2pgsql --slim -d contours --cache 5000 --style ./contours.style ./*.osm.pbf
rm ./*.osm.pbf

Kosmtik

You can use Kosmtik to develop on this style. You can use the kosmtik-fetch-remote plugin to handle downloading remote preprocessed shapefiles automatically. Information on setting up Kosmtik is available in the CONTRIBUTING.md file.

You can also use a localconfig.json file put in the root of this directory to overload parts of the project.mml file. For instance, if you want to use a database named gis with credentials, you can use

[
  {
    "where": "Layer",
    "if": {
      "Datasource.type": "postgis"
    },
    "then": {
      "Datasource.password": "gis",
      "Datasource.user": "gis",
      "Datasource.host": "localhost"
    }
  },
  {
    "where": "Layer",
    "if": {
      "Datasource.dbname": "osm"
    },
    "then": {
      "Datasource.dbname": "gis"
    }
  }
]

You can also generate a Mapnik XML configuration file using Kosmtik and the localconfig.json file with:

kosmtik export path/to/cyclosm-cartocss-style/project.mml --output path/to/cyclosm-cartocss-style/mapnik.xml

which you can then use in Tirex, mod_tile or any other tile queue manager.

Useful tweaks for production setup

Here are a few useful tweaks for running CyclOSM in a production setup, such as on https://cyclosm.org/.

Database indexes

A few databases indexes can help speed up the queries to the PostgreSQL database. On https://cyclosm.org/, we are currently using:

create index planet_osm_bicycle_routes on planet_osm_line using gist(way) where route='bicycle' OR route='mtb';
  • an additional index for the ferry lines:
create index planet_osm_line_ferry on planet_osm_line using gist (way) where route = 'ferry';

Notes

This guide is based on the openstreetmap-carto installation guide.