Data Collection

Satellite imagery

We downloaded the cropped satellite image within the district boundary at AAAI-20 paper. (GitHub for this project : https://github.com/Sungwon-Han/READ )

ArcGIS REST API Service make this available, and here is a guide to follow up our work. If you have any question, do not hesitate to contact me!

E-mail ) segaukwa@gmail.com

Daytime - World Imagery

Objective

Download the daytime satellite images inside the district.


Requirements


Satellite image tile

We would import satellite image tile from World Imagery (For export). You may see details from below link:

https://www.arcgis.com/home/item.html?id=226d23f076da478bba4589e7eae95952

And each tile of the layer can be reachable by :

https://tiledbasemaps.arcgis.com/arcgis/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}?token={token}

Where z, y, x determines zoom level and coordinates (More details), and token is temporal token for your ArcGIS Developer Account.

You may check your token at Dashboard (https://developers.arcgis.com/dashboard ).


Boundary data (From Esri Demographics)

What we need for next is the boundary of each district, and how to download it!

As our paper referred, we have utilized Esri Demographics data which covers 130+ countries (doc).

You can check those 130+ countries at coverage, Most of the demographics data were collected by Michael Bauer Research(MBR).

In pjson format, the link:

https://geoenrich.arcgis.com/arcgis/rest/services/World/geoenrichmentserver/Geoenrichment/Countries?f=pjson

contains metadata of Esri Demographics of whole 130+ countries.

If you want to focus on a country, add country code to the link as:

https://geoenrich.arcgis.com/arcgis/rest/services/World/geoenrichmentserver/Geoenrichment/Countries/{country_code}?f=pjson

You can find country_code at:

https://developers.arcgis.com/rest/geoenrichment/api-reference/geoenrichment-coverage.htm

For example, South Korea has country_code KR, so the link would be...

https://geoenrich.arcgis.com/arcgis/rest/services/World/geoenrichmentserver/Geoenrichment/Countries/KR?f=pjson


NOTE!!! This link has an important role in overall progress, so I would name it as: CountryInfoLink


Let's take a look inside CountryInfoLink.

Inside the key "hierarchies", we only want the data boundary that defined by MBR research.

So please focus on the "datasets" that contains "MBR" at CountryInfoLink. REMEMBER THIS(*)! ex) KOR_MBR_2018 for South Korea

Just below the "datasets", "levelsInfo" contains "geographyLevels" which shows a list of unit area of collected demographics data.

The rightmost one -- in many case, 'Districts'-- is the smallest unit area. REMEMBER THIS(**)! ex) Districts for South Korea

Please keep datasetID(*), and the smallest_unit_area(**) in your mind!

From here, we will use ArcGIS GeoEnrichment API (doc).

You need following parameters as (key : value).

    • f : put pjson
    • sourceCountry : put country_code you are interested in. ex) KR
    • token : put your ArcGIS Developer token.
    • geographylayers : put country_code.Country. ex) KR.Country
    • geographyids : put ['01']
    • returnSubGeographyLayer : put true
    • subGeographyLayer : put country_code.smallest_unit_area. ex) KR.Districts
    • DataSetID : put datasetID. ex) KOR_MBR_2018
    • returnGeometry : put true

to make the API request.

You can visually check the response via Postman.

For example,

If you put in all parameters in your request, you can get a response from ArcGIS GeoEnrichment API.

As you can notice, each "features" matches with each district(smallest_unit_area), and "rings" returns coordinates of the boundary.

Each [A,B] implies [longitude,latitude] of the boundary.

This request returns whole district boundaries within the country(country_code).

This response is written in json format, so you could handle this boundary data information with additional codes.


Satellite imagery within the boundary

So far, we can download a satellite image for given [ zoomlevel / y tile index / x tile index ],

and we can get district boundaries, which are written in [latitude / longitude] for each district.

Now the problem is, we need a formula to convert two coordinates : (ZYX tile coordinate) <-> (latitude/longitude coordinate)

From OSM wiki, we can get a pseudocode for this.

Here is my python code example, for coordinate switching.

import math

def num2deg(xtile, ytile, zoom):
  n = 2.0 ** zoom
  lon_deg = xtile / n * 360.0 - 180.0
  lat_rad = math.atan(math.sinh(math.pi * (1 - 2 * ytile / n)))
  lat_deg = math.degrees(lat_rad)
  return (lat_deg, lon_deg)

def deg2num(lat_deg, lon_deg, zoom):
  lat_rad = math.radians(lat_deg)
  n = 2.0 ** zoom
  xtile = int((lon_deg + 180.0) / 360.0 * n)
  ytile = int((1.0 - math.log(math.tan(lat_rad) + (1 / math.cos(lat_rad))) / math.pi) / 2.0 * n)
  return (xtile, ytile)

From here, it is up to you to choose which images within the boundary.

In our paper, we decided to download the satellite image tile if more than three vertices contained inside the district boundary.

So in the sample figure, we download all image tiles as a daytime satellite imagery that "included" in each district.

reference : my AAAI-20 paper