[ g e o m o d s ]

Geomods is a set of Python modules and console scripts for utilizing elevation data and Digital Elevation Models.

==============================================
Python Dependencies:
==============================================

 - GDAL *
 - numpy *
 - scipy
 - lxml *
 - requests *
 - matplotlib

==============================================
External Dependencies for DEM generation and data processing:
==============================================

 - GDAL *
 - GMT
 - MBSystem
 - LASTools
 - VDatum 4.x

* required

==============================================
Install via git and pip:
==============================================

 % git clone https:/github.com/ciresdem/geomods.git
 % cd geomods
 % pip install --user --upgrade ./

==============================================
Install via git and setup.py:
==============================================

 % git clone https://github.com/ciresdem/geomods.git
 % cd geomods
 % python setup.py build
 % sudo python setup.py install --record geomods.install

==============================================
Install via zip and setup.py:
==============================================

 % wget https://github.com/ciresdem/geomods/archive/master.zip
 % unzip master.zip
 % cd master
 % python setup.py build
 % sudo python setup.py install --record geomods.install

==============================================
Uninstall from setup.py:
==============================================
 
 % sudo python setup.py clean
 % sudo xargs rm -r < geomods.install

==============================================
Update with setup.py:
==============================================

 % git pull
 % sudo python setup.py clean
 % sudo xargs rm -r < geomods.install
 % python setup.py build
 % sudo python setup.py install --record geomods.install

==============================================
Python Modules:
==============================================

 - fetches - Data fetching
 - waffles - DEM generation and processing
 - datalists - datalist processing
 - regions - region processing
 - xyzfun - xyz data functions
 - gdalfun - gdal functions
 - gmtfun - gmt functions
 - mbsfun - mb-system functions
 - vdatumfun - vdatum functions
 - utils - general utilities and processing

==============================================
Console Scripts:
==============================================
 
 - waffles - Generate a Digital Elevation Model and derivatives.
 - fetches - Fetch elevation data from various sources.
 - datalists - process and analyze datalists.
 - gdal_chunk.py - Split a gdal grid into chunks.
 - gdal_crop.py - Crop a gdal grid by it's nodata value.
 - gdal_clip.py - Clip a gdal grid to an ogr vector.
 - gdal_null.py - Generate a null gdal grid.
 - gdal_mask.py - Apply a raster mask to a gdal grid.
 - gdal_split.py - Split a gdal grid based on z value.
 - gdal_query.py - Query the z value of a gdal grid with x/y coordinates.
 - vdatum_cmd.py - Use NOAA's VDatum via command-line.
 - hillshade.sh - Generate an ETOPO1 style hillshade image.

==============================================
Examples:
==============================================

 - fetch and process lidar data from NOAA's Digital Coast for all the regions found in tiles.shp 
   % fetches dc -R tiles.shp -p --where 'Datatype LIKE "lidar%"'

 - list all the NOS hydrographic data urls in the given region
   % fetches nos -R -90/-89/29/30 -l > nos.urls

 - generate a DEM for each region found in tiles.shp using GMT surface and the data found in input_data.datalist
   % waffles -M surface -R tiles.shp -E .00000925925 input_data.datalist --verbose

 - generate spatial-metadata from the datalist input_data.datalist
   % waffles -R tiles.shp -E .00000925925 input_data.datalist --verbose -M spat-meta

 - generate a vertical datum conversion grid:
   % waffles -R -90/-89/29/30 -E 0.00083333 -M vdatum:ivert=navd88:o_vert=mhw

 - generate a 'bathymetry surface' DEM:
   % waffles -R region.shp -E 1s:.3333333s -X6 -M surface:upper_limit=0 -C coast_ply.shp:invert=True

 - generate a nearest-neighbor DEM using all the .tif grid files in the current directory:
   % waffles -R region.shp -E1s -Mnearneighbor:radius=3s *.tif

 - split grid file tmp.tif into chunks of 1000x1000 cells:
   % gdal_chunk.py tmp.tif -c 1000

 - transform the vertical datum of an xyz data file:
   % vdatum_cmd.py elev_data.xyz -i lmsl:ft:sounding -o navd88:ft:height

==============================================
Waffles CLI:
==============================================

waffles [OPTIONS] <datalist/entry>

Generate DEMs and derivatives and process datalists.

General Options:
  -R, --region		Specifies the desired REGION;
			This can either be a GMT-style region ( -R xmin/xmax/ymin/ymax )
			or an OGR-compatible vector file with regional polygons. 
			If a vector file is supplied it will search each region found therein.
			If omitted, use the region gathered from the data in DATALIST.
  -E, --increment	Gridding CELL-SIZE in native units or GMT-style increments.
			append :<inc> to resample the output to the given <inc>: -E.3333333s:.1111111s
  -F, --format		Output grid FORMAT. [GTiff]
  -M, --module		Desired DEM MODULE and options. (see available Modules below)
			syntax is -M module:mod_opt=mod_val:mod_opt1=mod_val1:...
  -O, --output-name	BASENAME for all outputs.
  -P, --epsg		Horizontal projection of data as EPSG code [4326]
  -X, --extend		Number of cells with which to EXTEND the REGION.
			append :<num> to extend the processing region: -X6:12
  -T, --filter		FILTER the output DEM using one or multiple filters. <fltr:fltr_val:split_value=z>
			Available filters:
			1: perform a Gaussian filter at -T1:<factor>.
			2: use a Cosine Arch Filter at -T2:<dist(km)> search distance.
			3: Spike Filter at -T3:<stand-dev. threshhold>.
			The -T switch may be set multiple times to perform multiple filters.
			Append :split_value=<num> to only filter values below z-value <num>.
			e.g. -T1:10:split_value=0 to smooth bathymetry using Gaussian filter
  -Z, --z-region		Restrict data processing to records that fall within the z-region
			Use '-' to indicate no bounding range; e.g. -Z-/0 will restrict processing to data
			records whose z value is below zero.
  -C, --clip		CLIP the output to the clip polygon -C<clip_ply.shp:invert=False>
  -K, --chunk		Process the region in CHUNKs. -K<chunk-level>
  -W, --w-region	Restrict data processing to records that fall within the w-region (weight).
			Use '-' to indicate no bounding range; e.g. -W1/- will restrict processing to data
			records whose weight value is at least 1.
  -G, --wg-config	A waffles config JSON file. If supplied, will overwrite all other options.
			generate a waffles_config JSON file using the --config flag.

  -p, --prefix		Set BASENAME to PREFIX (append inc/region/year info to output BASENAME).
  -r, --grid-node	Use grid-node registration, default is pixel-node
  -w, --weights		Use weights provided in the datalist to weight overlapping data.

  -a, --archive		Archive the datalist to the given region.
  -m, --mask		Generate a data mask raster.
  -u, --uncert		Generate an associated uncertainty grid.
  -c, --continue	Don't clobber existing files.
  -q, --quiet		Lower verbosity to a quiet. (overrides --verbose)

  --help		Print the usage text
  --config		Save the waffles config JSON and major datalist
  --modules		Disply the module descriptions and usage
  --version		Print the version information
  --verbose		Increase the verbosity

Datalists and data formats:
  A datalist is a file that contains a number of datalist entries, while an entry is a space-delineated line:
  `/path/to/data format weight data,meta,data`

Supported datalist formats: 
  -1	['datalist', 'mb-1']
  -2	['zip', 'gz']
  -4	['nos', 'dc', 'gmrt', 'srtm_cgiar', 'srtm_plus', 'mar_grav', 'charts', 'mb', 'tnm', 'emodnet', 'chs', 'hrdem', 'cudem']
  168	['xyz', 'csv', 'dat', 'ascii']
  200	['tif', 'img', 'grd', 'nc', 'vrt', 'bag']
  300	['las', 'laz']

Modules (see waffles --modules <module-name> for more info):
  surface, triangulate, cudem, nearest, num, vdatum, mbgrid, invdst, average, linear, spat-meta, coastline, uncertainty

==============================================
Fetches CLI:
==============================================

fetches [OPTIONS] <module[:parameter=value]* ...>

Fetch geographic elevation data.

General Options:
  -R, --region		Specifies the desired region to search;
			This can either be a GMT-style region ( -R xmin/xmax/ymin/ymax )
			or an OGR-compatible vector file with regional polygons. 
			If a vector file is supplied it will search each region found therein.
  -W, --where		restricted_where: Attribute query (like SQL WHERE)
  -E, --increment	Blockmedian/mean CELL-SIZE in native units or GMT-style increments.
  -Z, --z-region		Restrict data processing to records that fall within the z-region
			Use '-' to indicate no bounding range; e.g. -Z-/0 will restrict processing to data
			records whose z value is below zero.
  -F, --fg-config	A fetches config JSON file. If supplied, will overwrite all other options.
			generate a fetches_config JSON file using the --config flag.

  -l, --list		Return a list of fetch URLs in the given region.
  -d, --dump		Dump the XYZ elevation data in WGS84 to stdout.
  -p, --process		Process fetched elevation data to ASCII XYZ format in WGS84.
			If -E or -Z are set, processing will use those switches in data processing.
  -u, --update		Update the Fetches Remote Elevation Datalist.
  -i, --index		Print the fetch FRED results in the given region.

  --help		Print the usage text
  --config		Save the fetches config JSON
  --version		Print the version information

Modules (see fetches --modules <module-name> for more info):
  dc nos charts ncei_thredds usace tnm gmrt mb mar_grav srtm_plus emodnet ngs chs hrdem

Examples:
 % fetches -R -90.75/-88.1/28.7/31.25 nos --where "Date > 2000"
 % fetches -R region.shp -p dc nos:datatype=bag charts:datatype=enc
 % fetches -R region.shp dc:datatype=lidar -l > dc_lidar.urls
 % fetches -R -89.75/-89.5/30.25/30.5 tnm:ds=4:formats=IMG gmrt:res=max:fmt=geotiff