Contains modified Copernicus Sentinel data
Contents
Context
The mission platform needed a whole country's terrain, imagery, roads and urban data on site, with no network to fetch anything later. This pipeline builds all of it.
Constraints
- No network at the destination. Tiles, terrain, place search and routing all ship with the system.
- National scale for every layer.
- Large datasets at every step.
- A 3D engine downstream that has to hold frame rate on what the pipeline produces.
Architecture
Components and flows
Components
- Sources
- Terrain
- Imagery
- Roads and urban data
- Processing
- Nationwide ETL (large datasets as Parquet)
- Tile download and packaging (tiles for a defined area)
- Offline serving
- PostGIS
- Tile and terrain sources
- Place search and routing
- Delivery
- Delivery package
- Mission platform
Flows
- Terrain to Nationwide ETL
- Imagery to Nationwide ETL
- Roads and urban data to Nationwide ETL
- Nationwide ETL to PostGIS
- Nationwide ETL to Tile and terrain sources
- Tile download and packaging to Delivery package (map tiles)
- PostGIS to Place search and routing
- Tile and terrain sources to Delivery package
- Place search and routing to Delivery package
- Delivery package to Mission platform
- Sources: terrain, imagery, roads and urban data, nationwide.
- Processing: large GIS datasets handled as Parquet.
- Tile download and packaging: a system I built that fetches the map tiles for a defined area and packs them into the delivery package.
- Serving: PostGIS and offline tile and terrain sources.
- Place search and route navigation that work with no network.
- Consumer: the mission platform's 2D map and 3D engine.
Terrain OFFLINE DATA BUILD
- CONTOURS
- MAP TILES
- ROADS
- URBAN DATA
- PLACE SEARCH
- DELIVERY AREA
Illustrative concept, not a product screenshot. Terrain: public Terrarium elevation tiles over the Karakoram, chosen for relief.
Key decisions
-
The sites are air-gapped, so nothing can be fetched on request. Map tiles for each area, search and routing are built before delivery and shipped inside the delivery package.
My role
I was the architect and engineering lead: I owned the pipeline's requirements, architecture and technology decisions and oversaw the engineering. I built the tile-download and packaging system myself.
Outcome
The mission platform searches places and plans routes on nationwide data with no network, from sources this pipeline builds and packs into each delivery.
Stack
Each links to its entry on the stack page
Ask about this
How does the mission platform search places and plan routes with no network?
Answers come from the content on this site.
The chat needs JavaScript. Without it, ask through the contact page.