GeoPicker
A super fast rest interface to pick Geospatial datasets in NodeJs.
(UPDATED: September 2023)
It’s not only another simple elevation service but is designed to easily distribute small packets of geospatial data picked from large geospatial datasets, is part of a new generation of Fastify based microservices that are building a more modern and performant back-end world.
This is an open source project https://github.com/opengeo-tech/geopicker
Preamble
Thr original project start at 2015 previously named Geotiff-picker, it was designed to support only raster files in Geotiff format wih a trivial api rest interface.
Old version archived here:
https://github.com/opengeo-tech/geopicker/tree/old-version
written in python/gdal and then in PHP via single child process through gdal utility gdallocationinfo.
simple but functional project was the basis of a larger framework KeplerJs which use Geotiff-picker to calculate the altitude and aspect to the ground of the new places created in KeplerJs, the different datasets were distributed in different php endpoints to ease the load on the server.
Project Update
The old project has been rewritten in 2022 with more modern technologies, using the fastest web frawork Fastify and a new non-blocking asynchronous bindings library for GDAL named node-gdal-async by Momtchil Momtchev
all based on NodeJs technology.
This takes the form of a microservice that is packaged in Docker using a minimal nodejs image node:18-buster-slim published in Dockerhub here: stefcud/geopicker
The codebase is a nodejs module that exposes some methods, the service consists of a Fastify server that uses this module and a very detailed configuration file in Yaml that allows you to organize different geospatial datasources set alias, limits and other options.
Features
- Large Rest API: many endpoints suitable for each data request context
- JSON Schema: validation of routes and parameters, can be deactivated
- Customization: friendly configs and to help devs in many deploy contexts
- Formats: support for different geospatial input and output formats
- Open source: a clear structure and documented source code to help contributors
API Rest
Before defining the rest api endpoints, I carefully analyzed all the apis currently available in various pre-existing opensource tools, including:
the result was to obtain an API as complete and detailed as possible which resembles all those already used.
Geopicker has been specially designed to offer the widest range of formats and methods of data requests that is possible, to adapt to any context of use by the client. Each endpoint and the parameters it accepts have been designed on the basis of the functioning of already existing services, gathering a complete and coherent collection of APIs.
In below table a description of all endpoints in the current version and the Swagger Documentation online: https://opengeo.tech/geopicker/docs/
includes additional utility features:
- Densify add more points in a sequence of coordinates, this improves the display on an elevation graph, adding intermediate positions at a minimum fixed distance.
- Simplify unlike densify it removes points that are too close together from a geometry.
- Meta additional informations calculated for a certain geometry, can be for example the direction of a path.
Configuration
All the configurations you need is found in the handy Yaml file server/config.yml which can be easily extended and customized with environment variables, by this integrated library:
this below is a sample configuration part that deals with the settings of the Fastify instance, Cross-Origin Resource config and the definitions of the datasets (default, test, elevation, altitude, aspect) to be exposed:
fastifyConf:
maxParamLength: 1024
logger:
level: 'info'
transport:
target: 'pino-pretty'
cors:
origin: '*'
datasets:
default: elevation
test:
path: test_4611_dem.tif
band: 1
epsg: 4326
elevation:
path: trentino-altoadige_dem_90m.tif
band: 1
epsg: 4326
altitude: elevation
# alias of elevation dataset
aspect:
path: italy_aspect_30m.tif
band: 2
epsg: 4326Formats
This service allows a very wide conversion of formats for each format there is a collection of drivers for reading or writing which is progressively expanded. Some others convenient common interchange formats are simple JSON in Array of coordinates or Object. In additional the compressed Google Polyline format could not be missing or CSV and Shapefile. Refer to the table above for the support status of the formats.
Testing
This is just two simple example of using GeoPicker Api sending stringified coordinates whose altitude you want to know, the result in meters.
curl "http://localhost:9090/elevation/11.1/46.1"
[195]curl "http://localhost:9090/elevation/11.1,46.1|11.2,46.2|11.3,46.3"
[195,1149,1051]Post anyone GeoJSON geometry, the same input geometry is always returned which has a third dimension
$ curl -X POST -H 'Content-Type: application/json' \
-d '{"type":"LineString","coordinates":[[11.1,46.1],[11.2,46.2],[11.3,46.3]]}' \
"http://localhost:9090/elevation/geometry"
{"type":"LineString","coordinates":[[11.1,46.1,195],[11.2,46.2,1149],[11.3,46.3,1051]]}Post a very long LineString in body saving bytes:
$ curl -X POST -H 'Content-Type: application/json' \
-d '[[10.9998,46.0064],[10.9998,46.0065],[10.9999,46.0066],[11.0000,46.0067]]' \
"http://localhost:9090/elevation/locations"
[[10.9998,46.0064,900],[10.9998,46.0065,898],[10.9999,46.0066,898],[11.0000,46.0067,900]]the project contains an advanced map client that implements requests for all endpoints, useful tool for testing and studies. This page is accessible to the url http://localhost:9090/ after the option demo_page: true has been enabled in the configuration file or set environment var DEMO_PAGE=true
Official online Demo https://opengeo.tech/geopicker
as you can see the interactions are very fast and can be used to trace in real time entire mountain altimetric profiles.
Performance
A set of scripts for tracking benchmarks has been included using nodejs lib AutoCannon by Matteo Collina
Some results, testing a dataset of 10KB geotiff, 244k requests in about 10 seconds and 54 MB read
┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────┤
│ Latency │ 0 ms │ 0 ms │ 0 ms │ 1 ms │ 0.02 ms │ 0.16 ms │ 6 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬──────────┬─────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼─────────┼─────────┼──────────┼─────────┼─────────┤
│ Req/Sec │ 18111 │ 18111 │ 22783 │ 23471 │ 22175.28 │ 1473.21 │ 18099 │
├───────────┼─────────┼─────────┼─────────┼─────────┼──────────┼─────────┼─────────┤
│ Bytes/Sec │ 4.02 MB │ 4.02 MB │ 5.05 MB │ 5.21 MB │ 4.92 MB │ 327 kB │ 4.01 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴──────────┴─────────┴─────────┘License and Contributions
GeoPicker project is released under open source BSD 2-Clause license and awaits many of your Pull Requets.
The Roadmap is waiting to be completed and there are a lot of simple to code improvements that you can practice contributing to.
Please share this project if you think it’s interesting.
