These are experimental development notes to generate map tiles to be used offline as an XSCE content package. This is part of an ongoing effort to provide better maps on the XSCE.

See the design document.

To gain access to the map tile generation infrastructure, please get in touch with Anish Mangal or anyone from the XSCE development team.

Generating Map Tiles

Make a tileserver

Prerequisites

This document expects a basic understanding of OSM tile-serving to make sense to the reader. If you are just beginning in your quest to use map tiles, either go to the excellent OSM resources, or use one of the pre-built tilesets provided as part of the XSCE distribution.

You would ideally need a fast machine with a multicore processor, lots of RAM, and a 500+GB SSD running Ubuntu 14.04 to setup a tileserver capable of generating map tiles for the entire planet. If you have such infrastructure, great! If not, get in touch to gain access to the XSCE map development server, graciously hosted by Braddock Gaskill.

Create the tileserver

Follow the detailed instructions given here to set-up a functioning tile server.

There are a few issues with the default server, which you will have to work around if you want better looking maps. All these are optional steps/enhancements.

  1. Indic fonts do not render well on the default Mapnik version (2.x). You will need to install Mapnik 3 from source.
  2. If you are using Mapnik 3, you will also need to compile mod_tile (which provides renderd) from source.
  3. Optionally, you should upgrade to the latest boost libraries. Some Mapnik plugins like placing geojson components on a map won't work with boost < 1.56. Download source tarballs with instructions can be found on the boost website.
  4. If you want to have map tiles generated in your preferred language and not English by default, you will need the tags column in your postgres database tables. One approach to have that is by using the --hstore functionality provided by postgres. The steps involved are:
    1. While creating the postgres database. At the step Set up PostGIS on the PostgreSQL database in the OSM tileserver guide, create the hstore extension.
      CREATE EXTENSION hstore;
    2. While loading the osm.pbf file into the database use the --hstore switch.
      osm2pgsql --hstore --number-processes 4 -C 17000 -d gis planet.osm.pbf

Generate tiles

If all has gone well upto now, then you are ready to generate tiles. This is as simple as giving the render_list command.

sudo -u <username> render_list -a -n <num_cpu_processes> -z <min_zoom_level> -Z <max_zoom_level>

The above command will generate all map tiles between the zoom ranges enclosed in [z, Z]. In practice however, you will probably want to generate tiles for a specific geographic region. In order to have that, you will need the bounding box information of the geographical region(s).

Head to the OSM website and click the Export button. This will open up the bounding box selection overlay on the map. Select the region you wish to generate the tiles for and note the lower and higher latitude and longitude.

Input that information in the script below to obtain the commands which will generate the tiles.

latlong.py
#!/usr/bin/python
 
import math
# This function convers lat, long the desired zoom level into z/x/y
def deg2num(lat_deg, lon_deg, zoom):
    lat_rad = math.radians(lat_deg)
    n = 2.0 ** zoom
    xtile = int((lon_deg + 180.0) / 360.0 * n)
    ytile = int((1.0 - math.log(math.tan(lat_rad) + (1 / math.cos(lat_rad))) / math.pi) / 2.0 * n)
    return (xtile, ytile)
 
# Input the zooms levels you want to generate the render_list command for.
zoom_levels = [10, 11, 12, 13, 14, 15, 16, 17, 18]
 
# Input latitude and longitude information for bounding boxes in the data
# structure below. Make sure that lat1 < lat2, and lon1 < lon2. You may
# include as many bounding boxes as you wish.
#
# The example below are the bounding box regions for India, as selected by 
# using the export function from http://openstreetmap.org
bboxes = [ {'lat1' : 19.00, 'lon1': 67.41, 'lat2' : 30.00, 'lon2' : 98.53},\
           {'lat1' :  5.27, 'lon1': 71.10, 'lat2' : 20.00, 'lon2' : 87.54},\
           {'lat1' : 29.00, 'lon1': 71.81, 'lat2' : 37.51, 'lon2' : 83.67},\
           {'lat1' :  5.00, 'lon1': 90.79, 'lat2' : 15.71, 'lon2' : 96.06} ]
 
username = 'user'
 
for zoom in zoom_levels:
    for bbox in bboxes:
        x1, y1 = deg2num(bbox['lat1'], bbox['lon1'], zoom)
        x2, y2 = deg2num(bbox['lat2'], bbox['lon2'], zoom)
        print "sudo -u %s render_list -z %d -Z %d -a -n 4 -x %d -X %d -Y %d -y %d" % (username, zoom, zoom, x1, x2, y1, y2)

This will output commands of the form below, which can simply be entered at the linux commandline

sudo -u user render_list -z 10 -Z 10 -a -n 4 -x 703 -X 792 -Y 456 -y 422
sudo -u user render_list -z 10 -Z 10 -a -n 4 -x 714 -X 761 -Y 496 -y 453
sudo -u user render_list -z 10 -Z 10 -a -n 4 -x 716 -X 749 -Y 425 -y 396
sudo -u user render_list -z 10 -Z 10 -a -n 4 -x 770 -X 785 -Y 497 -y 466

Useful tweaks

Interesting hacks around common mapping problems go here.

Disputed borders

In some cases, local administrative departments require certain disputed map regions displayed in a particular manner. The OSM database includes information about disputed borders, and extra properties on boundaries which have tags like claimed_by. One strategy to get around displaying disputed borders as the local administrative body would like it is:

  • Do not generate any lines on tiles for disputed borders.
  • Overlay the claimed_by border of the administrative agency in question over the existing imported OSM data while rendering tiles.

There may be many methods to get around this problem. If you get stuck, seek help in the OpenStreetMap community. Usually, their IRC channel or mailing list may be good starting places.

Example: Disputed border between India, Pakistan, and India, China as required by maps displayed in India

This section requires some knowledge of editing OSM xml stylefiles.

For a quick visual reference. See the difference between the Indian and global map versions of this region. The disputed areas are the state of Jammu_and_Kashmir, and Arunachal_Pradesh.

In this example, we will make the following tweaks to the standard OSM setup:

  • Install the osm-boundaries package. Run the run.py script on the osm.pbf file. This will generate a file called osm_admin_x-y.osm.pbf, where x and y were admin_level arguments given to the script. Import this file into the already existing osm2pgsql database using the -a (append) flag. You should now have a table called carto_boundary in your database.
  • The hstore extension is used while loading the osm.pbf file into the postgres database. See the sections above for notes on how to do that.
  • Edit the xml stylefile which mod_tile and renderd use to generate the tiles. In this example, the OSMBright style is used, but one could use the standard mapnik-openstreetmap-carto as well.
    • Hide any borders which have the disputed tag set to yes.
    • Overlay our custom claimed_by (India) data on the map when rendering tiles.

For the second step above, find the OSMBright.xml style after cloning the repository and building the stylefile. Find the Layer definition for admin. These are the administrative boundaries we need to selectively hide. The existing xml would look something like this:

<Layer name="admin"
  srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
    <StyleName>admin</StyleName>
    <Datasource>
       <Parameter name="dbname"><![CDATA[osm]]></Parameter>
       <Parameter name="extent"><![CDATA[-20037508.34,-20037508.34,20037508.34,20037508.34]]></Parameter>
       <Parameter name="geometry_field"><![CDATA[way]]></Parameter>
       <Parameter name="id"><![CDATA[admin]]></Parameter>
       <Parameter name="key_field"><![CDATA[]]></Parameter>
       <Parameter name="project"><![CDATA[osm-bright-imposm]]></Parameter>
       <Parameter name="table"><![CDATA[( SELECT way, admin_level
  FROM planet_osm_line
  WHERE boundary = 'administrative'
    AND admin_level IN ('2','3','4')
) AS data]]></Parameter>
       <Parameter name="type"><![CDATA[postgis]]></Parameter>
    </Datasource>
  </Layer>

Edit it to look like this:

<Layer name="admin"
  srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
    <StyleName>admin</StyleName>
    <Datasource>
       <Parameter name="dbname"><![CDATA[gis]]></Parameter>
       <Parameter name="extent"><![CDATA[-20037508.34,-20037508.34,20037508.34,20037508.34]]></Parameter>
       <Parameter name="geometry_field"><![CDATA[geom]]></Parameter>
       <Parameter name="id"><![CDATA[admin]]></Parameter>
       <Parameter name="key_field"><![CDATA[]]></Parameter>
       <Parameter name="project"><![CDATA[osm-bright-imposm]]></Parameter>
       <Parameter name="table"><![CDATA[( SELECT geom, admin_level, disputed, maritime
  FROM carto_boundary
  WHERE admin_level IN ('1','2','3','4') 
  AND disputed = 0
) AS data]]></Parameter>
       <Parameter name="type"><![CDATA[postgis]]></Parameter>
    </Datasource>
  </Layer>

Explanation of the changes:

  • Instead of using planet_osm_line, use the carto_boundary table, which has the disputed information.
  • Only apply the style and render the boundary if disputed = 0.

In the next step, we will overlay the claimed_by information. For this, the overpass api may be used to generate the data. For the borders claimed by India, the API query looks like this.

Once you have overlay you require, export the data in geojson format, and overlay that file. Add the following definitions to the OSMBright.xml stylefile.

<Layer name="india_boundary" status="on"
  minzoom="7500"
  maxzoom="5000000000"
  srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
  <StyleName>india_boundary</StyleName>
  <Datasource>
       <Parameter name="layer"><![CDATA[OGRGeoJSON]]></Parameter>
       <Parameter name="type"><![CDATA[ogr]]></Parameter>
       <Parameter name="file"><![CDATA[/usr/local/share/maps/style/osm-bright-master/geojson/boundary_claimed_by_india/boundary_claimed_by_india.geojson]]></Parameter>
  </Datasource>
</Layer>

Additionally, you would need to create the india_boundary style, which should be a copy of the admin style. In our case it looks like this:

<Style name="india_boundary" filter-mode="first" opacity="0.5">
  <Rule>
    <MaxScaleDenominator>3000000</MaxScaleDenominator>
    <LineSymbolizer stroke-width="4" stroke-linecap="round" stroke-linejoin="round" stroke="#444466" />
  </Rule>
  <Rule>
    <MaxScaleDenominator>12500000</MaxScaleDenominator>
    <MinScaleDenominator>3000000</MinScaleDenominator>
    <LineSymbolizer stroke-width="2" stroke-linecap="round" stroke-linejoin="round" stroke="#444466" />
  </Rule>
  <Rule>
    <MaxScaleDenominator>50000000</MaxScaleDenominator>
    <MinScaleDenominator>12500000</MinScaleDenominator>
    <LineSymbolizer stroke-width="1.2" stroke-linecap="round" stroke-linejoin="round" stroke="#444466" />
  </Rule>
  <Rule>
    <MinScaleDenominator>50000000</MinScaleDenominator>
    <LineSymbolizer stroke-width="0.8" stroke-linecap="round" stroke-linejoin="round" stroke="#444466" />
  </Rule>
  <Rule>
    <LineSymbolizer stroke-linejoin="round" stroke="#444466" />
  </Rule>
</Style>

Once you have made these changes, begin the tile rendering process. You should now see the administrative boundaries as required. :-)

With inputs from Arun Ganesh and the OSM community

Localized maps

In most cases the default OSM tile rendering style generates placenames in English language. Depending on your needs, you may want to render the names in different languages of your choice. The steps involved in this process are:

  • Using the hstore extension is used while loading the osm.pbf file into the postgres database. See the sections above for notes on how to do that.
  • Edit the xml stylefile to display localized versions of the place names wherever possible.

For the second step, open the xml style file and find every instance where name is being displayed. Typically this will be in statements like the one below:

       <Parameter name="table"><![CDATA[( SELECT way, highway AS type, name, ref, oneway, CHAR_LENGTH(ref) AS reflen
    FROM planet_osm_line
    WHERE highway IN ('motorway', 'trunk')
      AND (name IS NOT NULL OR ref IS NOT NULL)
) AS data]]></Parameter>

In the SELECT statement above, you should replace name with COALESCE(tags->'name:hi',tags->'int_name',name) to get the desired effect. In this example, if there is a Hindi (hi) name available, it will be displayed instead of the default (English) name. The final edited xml will look like:

       <Parameter name="table"><![CDATA[( SELECT way, highway AS type, COALESCE(tags->'name:hi',tags->'int_name',name) AS name, ref, oneway, CHAR_LENGTH(ref) AS reflen
    FROM planet_osm_line
    WHERE highway IN ('motorway', 'trunk')
      AND (name IS NOT NULL OR ref IS NOT NULL)
) AS data]]></Parameter>

You may add more languages in the COALESCE statement to get different localized names. The generated tiles will be localized as per these settings.