SRTM- The Hard Way

Background

This page contains information previously on UsingSRTM Terrain Data. It still needs to be edited quite a bit to get rid of the duplicated bits…

This page builds on the section Using DEM data in Rail3D 2kD in the Rail3D User Guide.

My general verdict when I wrote that page was along the lines of “it can be done, but it probably shouldn’t.” Since that time, the situation has changed a bit, and it is starting to look as though DEM data could make a very valuable contribution to layout modelling for Rail3D. In particular:

• Rail3D now has built-in distant terrain simplification, so terrain points distant from the camera don’t have the killer effect on framerate that they used to have, and we have a need to populate large areas with “coarse terrain”, especially on mountain layouts.
• Previously, it was very difficult to get high-resolution dems for areas outside the usa. With the availability of 3-arc-second data from the nasa srtm project we have a free source of data covering most of the world at a resolution more than adequate for Rail3D.

1 srtm Data

The srtm (Shuttle Radar Topography Mission)¹ used a radar interferometry technique to measure the surface of the earth from space. This has its limitations:

• the radar has trouble with dense clouds
• the vertical resolution is only about 15m
• it doesn’t work at high latitudes, so it won’t be much help to fans of the Kiruna iron ore trains or the White Pass & Yukon.

Fortunately, it is possible to fill in the holes and tidy up the coastlines by interpolation techniques, and some of the data now available on the internet have been “cleaned up” this way.

• Note that the coastline-clipping process is not very accurate, at least for uk data — you may find that the edge of the dem area has shifted a few hundred metres compared to the coastline on the map.

However, we should be realistic: the srtm data are great for distant mountains, or for places where you simply don’t have topographic maps, but they may need quite a bit of hand tweaking near to the track - 15m is rather a big bump for a train!

1.1 Getting the data

You can download srtm data from the usgs Seamless data distribution site http://seamless.usgs.gov/. However, I wouldn’t recommend this: their server responds very slowly, and you’ll need a lot of patience to use the overcomplicated interface.

If you want raw, unfiltered data (clip your own coastlines…), try ftp://e0mss21u.ecs.nasa.gov/srtm/

A better option in most cases is the Centre for Tropical Agriculture CGIAR-CSI. (Please note that this is an academic server intended for researchers — don’t abuse their hospitality by downloading big chunks of data you don’t need!)

They simply have a map where you can select the 5×5 degree chunk of the earth’s surface you want, and then you just have to choose between GeoTiff² and ArcInfo ascii ³ formats. Both of these formats can be read byt the Rail3D coordinate Tool. GeoTiff is best for most purposes, because it is easy to manipulate in MicroDEM.

For example, tile (36,02) covers Wales and most of England. Go to the ftp download site ftp://srtm.csi.cgiar.org/ and grab srtm_36_02.zip from the Data_GeoTiff directory. The files are about 40MB each, and take about 30 to 45 minutes to download on my broadband connection. (If you want to try it out but don’t have a fast connection, look for a tile with a lot of sea on it — they are smaller!.)

1.2 MicroDEM

To get started with manipulating dem data, you will need Professor Peter Guth’s freeware program MicroDEM⁴. This is a collection of utilities for manipulating and visualising dems and vector maps of many different kinds. It looks horrifically complicated, but fortunately we won’t need to use many of the mysterious buttons (it’s fun to have a play though, especially with the visualisation tools). Try them out with the “Hanging Rock Canyon” data file that comes with the MicroDEM download.

You could download and install MicroDEM while your srtm data are downloading.

2 Extracting the data

When you’ve downloaded the srtm zip file, unzip it into the mapdata\dems folder that was created when you installed MicroDEM.

2.1 Viewing the file in MicroDEM

Start MicroDEM, and use the Open dem command to open the tiff file that you downloaded. MicroDEM will display it as an image, visualising the terrain by adding shading. You can adjust the colours and shading types to get different effects, or even set up a 3d “fly-through”. You’ll notice that the latitude and longitude of the mouse cursor are shown in the status bar. If you open another dem or map covering the same area, you’ll see a second cursor that moves around the other image in parallel to yours…

This Geotiff image is 6000×6000 pixels in size, i.e. it contains 36 million data points. That’s a bit much for us to deal with in one go.

Part of the srtm data for Scotland visualised in MicroDEM
The map area is about 40×40km, and includes two railway stations

For the moment, what we want to do is select a manageable chunk of the image, covering roughly the area we want to cover with terrain in Rail3D. If there are conspicuous mountains and coastlines in your region, you can probably do this by eye — otherwise, check on a map to estimate roughly the latitude and longitude bounding the area of interest.

2.2 Resolution

A nautical mile, the distance subtended by an arc of one minute of latitude at the earth’s surface is roughly 1800m long

The data points in the srtm file are spaced at angular intervals of 3 arc-seconds. This means that in the north-south direction they are about 90m apart. In the East-West direction they are also 90m apart at the equator, but get nearer together as you approach the poles. In Europe the distance is 50–60m.

You can see the resolution of the data in your file in MicroDEM by clicking on the Map Information button. You’ll get something like this (for an unmodified srtm file):

 Screen Pixels: 471 x 774
 Screen pixel size: 711.6 m
 1.0% of points on map
 dem=S_37_02.tif
  size:   6000×6000
  z range: −185 to 448 m
 2-byte int  ,  Elev units:  meters
   space: 3×3″
    (about 56.45×92.73 m)
 True north at top of map
 Display Datum: world geodetic system, 1984
 dem native datum: nad83
 Magnetic declination: 358.5°
 utm Grid declination:  −0.4°

The interesting part is the data about the number of points and their spacing. Our file contains 6000×6000 points (36 million in all), they are 3 seconds apart, which corresponds to a total geographical coverage of 5×5 degrees.

2.3 Reducing the data

The resolution of the srtm data means that when using unmodified data in Rail3D, you will get about 200 points per square km. This is a lot. Even a little branchline layout of 20×5 km would have 20 000 points; a layout covering the rather flat and boring East Coast Main Line from London to York (about 100×300 km), would have about 6 million.

In practice, Rail3D will ignore most of these points when displaying the layout, as they will never come within detailed terrain range (usually 2 or 3 km from the camera, depending on your settings). Above this distance, Rail3D displays the terrain in a simplified form, taking only the minimum number of points required to generate a mesh bearing in mind the maximum triangulation distance of 1000m between points. This needs about 2 points per sq km.

So, when putting down coarse background terrain, you need to aim for a point spacing of about 700m.

3 Converting dem’s to Rail3D format — the hard way

Use the Subset and zoom tool in MicroDEM to select the bit of the image you need. Keep it small for the time being — you can always come back for more later. Note that the tool only changes the part of the dem displayed — it doesn’t change the original dem file. You have to save the subset under a new name using the Save subset as dem command.

If you need to export a very large area, you can use the Thin or re-interpolate commands on the Save menu to reduce the resolution.

• Thin in MicroDEM reduces the amount of data by keeping only every Nth point in each axis, where N is an integer. If you thin by a factor of 5, for example, you will get a file with data points at a spacing of 15×15 seconds, or 450×250m.
• re-interpolate allows you to create a set of data points at any arbitrary spacing. The elevation of each new point is calculated by interpolating between the elevations of the points closest to it in the original file.

Open the new file you have created. Check that it is a sensible size, then re-save it to ascii .xyz format. Be sure to specify Lat/Long rather than utm for the coordinate system.

3.1 Processing the ascii file for Rail3D

The data in the ascii file is just a list of three columns of numbers — latitude, longitude, and elevation.

  57.202081   −6.166250     16
  57.202914   −6.166250     12
  57.203747   −6.166250     11
  57.204581   −6.166250      9
  57.205414   −6.166250      9
  57.206247   −6.166250     13
  57.207081   −6.166250      1
  57.207914   −6.166250     17
  57.208747   −6.166250     45
  57.209581   −6.166250    150
  57.210414   −6.166250    208
  57.211247   −6.166250    263
  [etc.]

Notice that longitude is negative in the Western hemisphere.

It is straightforward, but tedious, to convert this data to Rail3D format in a spreadsheet. Most Rail3D layouts are based on maps using a grid system like the British National Grid, but with coordinates offset from the real grid values to fit neatly into the Rail3D CoordinateRail3D Coordinate space.

Essentially, what you have to do is:

1. Transform the latitude and longitude from the wgs84 datum to the datum of the particular grid system⁵
2. Convert from latitude and longitude to the Mercator projection of the grid system
3. Shift the coordinates to take into account the “false origin” you are using in Rail3D
4. Put the data in Rail3D’s terrain X Y Z format and paste them into Rail3D.

4 Rail3D Coordinate Tool — the easy way…

If you’re happy with manipulating complex trig formulae, it can be quite fun to set this up in Open Office Calc or ms Excel, but I found I was making too many mistakes, so I knocked up a little program in Visual Basic to do this in one go⁶.

The tool currently supports maps based on the following grid systems:

• Great Britain (Ordnance Survey)
• Switzerland
• Netherlands
• Germany (Gauß-Krüger)
• Belgium
• utm (global)⁷

If you need another grid system, let me know.

4.1 Using the Tool

• Open a map session saved with the Digitiser
• Calibrate the map to real-world coordinates by putting in the position of a reference point in Rail3D and grid coordinates so that the program can work out the offset
• Open a dem file. The Tool can read GeoTiff, ArcInfo .asc, MicroDEM’s .dem format, and ascii xyz triplets. The input file has to be in Lat/Long format (if someone has a good reason to use input data in utm format, let me know).
• specifying the parts of the file you want to convert by marking a rectangle on the map⁸.
• Specify whether you want to use the full 90m resolution or thin the data.

• The filtering parameters allow you to select the area you want to convert. Normally you will do this graphically by dragging on the map. The vertical filtering parameters can normally be left unchanged — you can use these if you are using very noisy source data, and want to reject “impossible” points.
   
• The thinning option allows you to take only one in N of the points in the file.

Always back up your layout before pasting in large sets of terraindata!

• Press the “Convert dem” button

Landscape generated using srtm data for the Zermatt area

See SRTM Terrain Step-by-step for a practical example.

¹ http://srtm.usgs.gov/ (↑)

² GeoTiff is a version of the Tiff image format used for maps, satellite photos and the like, that includes geographical position metadata embedded in the file. If you edit a GeoTiff in a standard bitmap editor, you will probably destroy the metadata. (↑)

³ A proprietary format used by esri’s ArcInfo software (↑)

⁴ http://www.usna.edu/Users/oceano/pguth/website/microdem.htm (↑)

⁵ See http://www.gps.gov.uk/info.asp for a detailed guide to this process for the Ordnance Survey grid (↑)

⁶ See Rail3d Utilities (↑)

⁷ If your layout crosses a utm zone boundary, you can use “zone-forcing” to treat the points as though they are all in the same zone (↑)

⁸ The program will try to choose the most likely grid system, depending on the area covered by the dem — you can change this if it doesn’t correspond to your map (↑)

Mark Hodson

MRG 26/06/2013 11:11:08