My challenge: show Dutch rail tracks on a map. This challenge is composed of several steps:
- find a dataset that contains geographic data on Dutch railways
- convert that data to a format I can process – ideally GeoJSON as I know how to work with GeoJSON in a D3,js web application
- create a base map of The Netherlands (from a GeoJSON file) and add the rail tracks from the GeoJSON objects produced by converting the original data format
It turns out that rail track data is indeed available from various sources in different formats – including ArcGIS JSON (from ProRail ArcGIS service) and OpenGIS GML (from PDOK – The Open GeoData Portal by the Dutch Kadaster). Both formats can be converted to GeoJSON format.
First iteration – Low Resolution Rail Track data
Download from PDOK Viewer (https://app.pdok.nl/lv/bgt/download-viewer/ –) – in GML Light format .
Install QGIS (Open Source Desktop GIS – a professional GIS application that is built on top of and proud to be itself Free and Open Source Software (FOSS)). Note: QGIS has a 1.2 GB installer download (on Windows).
Open GML file in QGIS: start project. Click the layer menu and add a vector layer.
Browse to the location of your GML file and select the GML file type.
Add the layer to your project.
Note: PDOK downloads in GML have EPSG:28992 as the default CRS (Coordinate Reference System). This is not automatically discovered by QGIS, so it needs to be explicitly defined for the layer.
To create the GeoJSON file: select the layer in the layer browser and right click on the layer.
Click on Export and Save Features As.
Select GeoJSON from the available formats and specify as CRS one of type Geographic Coordinate Systems (GCS),for example WGS 84: These use latitude and longitude to define locations on the earth’s surface. Values are in degrees.
Specify the name and location of the GeoJSON file to be created by QGIS. Then click on OK. The resulting GeoJSON file can now be used in the D3.js application.
Try to run web application.
It will take a fairly long time before the map shows up. Over 50MB of GeoJSON data is loaded and close to 60K features are generated into SVG shapes.
The data is fairly high resolution – which becomes clear when I start to zoom in:
and further yet – with all trams, light rail, train and metro/subway tracks in Rotterdam:
In order to work smoothly with all this data, I need a trick to only use a subset (reduce resolution) when the scale factor is small and only start adding the locally relevant fine grained data when the scale factor is increased. Perhaps I should create multiple data sets (a low res one covering all of The Netherlands and high res files per area). When the user zooms in on a specific area, I should switch to/bring in a high res data set.
Additionally, the style definition use a stroke width of 0.45px which is fine at lower resolutions but which is too large when the scale factor increases.
Here with a scale factor of 140K to see how much detail the data set contains. The map as shown in upper right hand corner is screen shot of the PDOK download viewer from which I downloaded all data.
Alternative (and more detailed) source of Dutch Railway data – ArcGIS Server
There are other data sets with railway data. A detailed set is available from Vervoersnetwerken: Spoorwegen – Transport Networks: Rail (INSPIRE geharmoniseerd) https://www.nationaalgeoregister.nl/geonetwork/srv/dut/catalog.search#/metadata/3a7dd0a6-d130-4c4c-b0ba-24365cf036e2 and from ProRail: Spoorgeometrie Permanent link https://data.overheid.nl/en/dataset/prorail-spoorgeometrie
The ProRail Base Map can be inspected in the ArcGIS Online Map Viewer. Data can be downloaded from MapService: https://mapservices.prorail.nl/arcgis/rest/services/ProRail_basiskaart_001/FeatureServer/query .
Easier still: to visualize data from the ProRail Data Set in QGIS (and from there export it as GeoJSON data) do the following:
Click on the node ArcGIS REST Servers. Right mouse menu, New Connection. In the window that popups for defining the connection details, enter https://mapservices.prorail.nl/arcgis/rest/services/ProRail_basiskaart_001/FeatureServer as the URL. Press OK.
When you now expand the newly created node, you will find a number of layers to inspect.
These layers can be added to the project and will be visualized in the viewer.
Just for fun – a quiz: what is this next figure?
(Railway stations in The Netherlands)
Resources
A GitHub Repo for this article. Live demo of web application (note: it takes a while – 30-90 seconds) for the visualization to show up because of the large data set).
Visualize railway data on GitHub GeoJSON viewer.
Free QGIS The Leading Open Source Desktop GIS – QGIS is a professional GIS application that is built on top of and proud to be itself Free and Open Source Software (FOSS). – https://qgis.org/
Comparing QGIS with ArcGIS – https://rutwikroutu.medium.com/arcgis-pro-vs-qgis-everything-you-need-to-know-8c9fe0aba19c
PDOK Download Viewer – https://app.pdok.nl/lv/bgt/download-viewer/ – Public Data On Map provides a wealth of geo data on the public space – in 33 categories that relate to roads, vegetation, water, buildings and yes: railways. Data can be downloaded in various formats: stufgeo , gmllight and citygml.
The StUF-Geo format is described here: https://www.geonovum.nl/geo-standaarden/bgt-imgeo/berichtencatalogus-stuf-geo-imgeo-versie-12 .
GML Light , also known as GML Simple Features Profile, is a simplified subset of the Geography Markup Language (GML) designed by the Open Geospatial Consortium (OGC). GML itself is a complex XML-based format for describing geographic features and spatial data, and it can be quite verbose and intricate for some applications. GML-Light or the GML Simple Features Profile addresses this by providing a streamlined yet fully compatible version of GML, focusing on the most commonly used features and reducing complexity. GML data can be converted to GeoJSON data. There are online tools available that can convert GML to GeoJSON (MyGeoData Converter , Aspose Geo Data Converter, GroupDocs Geo Data Converter). However, be cautious about data privacy when using online tools for sensitive data. Some GIS software like QGIS allows you to import GML and export GeoJSON directly.
Medium Article HK Spot Height Data: From GML to GeoJSON (in Python) – https://medium.com/@devlog/hk-spot-height-data-from-gml-to-geojson-b7c348d58908
Python tool to Convert ArcGIS JSON to GeoJSON – https://pypi.org/project/arcgis2geojson/
My earlier articles on World Map data visualization:
1. Create Interactive World Map to Visualize Country Data – https://technology.amis.nl/frontend/web-html5-css3/create-interactive-world-map-to-visualize-country-data/ – introducing World Map Data Visualization with d3, SVG and JavaScript.
2. Presenting the World in Data using World Map Visualization – https://technology.amis.nl/frontend/presenting-the-world-in-data-using-world-map-visualization/ – bring together a straightforward approach to data visualization (from article one) and a rich data set from Kaggle that provides many interesting details for all countries in the world, related to education, health, economy, demography, climate and more.
3. Interactive Data Visualization in World Map–Zoom, Translate, Legend – https://technology.amis.nl/tech/interactive-data-visualization-in-world-map-zoom-translate-legend/ – a closer look at adding some interactivity to the world map. In particular: changing the rotation of the map, select countries (by clicking on them), show country details popup window and show legend – color scale (mapping heatmap colors to numerical values)
4, World Map Data Visualization with d3.js, GeoJSON and SVG–zooming and panning and dragging – https://technology.amis.nl/frontend/world-map-data-visualization-with-d3-js-geojson-and-svg-zooming-and-panning-and-dragging/ – brining zooming and panning to the world map, both user initiated and programmatically.
Appendix
Tech Guidance from Gov.UK: https://use-land-property-data.service.gov.uk/datasets/inspire/tech-guidance
GML format
GML is an open data format that is widely used across Europe and complies with the EU INSPIRE Directive. HM Land Registry supports the use of open standards. We use the GML format to supply a standard data structure to users of all GIS/CAD systems.
How to use GML data in other tools or systems
The GML data format can be converted to other geographic information system (GIS) formats.
Depending on your system, you may need to convert GML to suit the data format of your GIS. Contact your GIS provider for details of how to convert the format within your system.
To use GML data in different tools you may have to convert the data into a format used by that tool and also ensure the data is in an appropriate projection.
For example, if you want to use GML data in Google Maps you’ll have to convert the GML file into keyhole markup language (KML) format.
Quantum geographic information system (QGIS) is a free open source system that can be used to view GML data and convert it to other data formats.
How to open GML files
- Click the layer menu and add a vector layer.
- Browse to the location of your GML file and select the GML file type.
- Add the layer to your project.
How to convert GML data to different formats
- Add the GML layer to your project.
- Select the layer in the layer browser and right click on the layer.
- Click on save as and choose from the available formats such as:
- ESRI shapefile
- Mapinfo MIF or TAB
- AutoCad DXF
- GeoJson
- KML
