Data Collection
Satellite imagery
We downloaded the cropped satellite image within the district boundary at AAAI-20 paper. (GitHub for this project : https://github.com/Sungwon-Han/READ )
ArcGIS REST API Service make this available, and here is a guide to follow up our work. If you have any question, do not hesitate to contact me!
E-mail ) segaukwa@gmail.com
Daytime - World Imagery
Objective
Download the daytime satellite images inside the district.
Requirements
- ArcGIS Developer Account (Get it from https://developers.arcgis.com/ )
- Postman (Not necessary)
Satellite image tile
We would import satellite image tile from World Imagery (For export). You may see details from below link:
https://www.arcgis.com/home/item.html?id=226d23f076da478bba4589e7eae95952
And each tile of the layer can be reachable by :
https://tiledbasemaps.arcgis.com/arcgis/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}?token={token}
Where z, y, x determines zoom level and coordinates (More details), and token is temporal token for your ArcGIS Developer Account.
You may check your token at Dashboard (https://developers.arcgis.com/dashboard ).
Boundary data (From Esri Demographics)
What we need for next is the boundary of each district, and how to download it!
As our paper referred, we have utilized Esri Demographics data which covers 130+ countries (doc).
You can check those 130+ countries at coverage, Most of the demographics data were collected by Michael Bauer Research(MBR).
In pjson format, the link:
contains metadata of Esri Demographics of whole 130+ countries.
If you want to focus on a country, add country code to the link as:
https://geoenrich.arcgis.com/arcgis/rest/services/World/geoenrichmentserver/Geoenrichment/Countries/{country_code}?f=pjson
You can find country_code at:
https://developers.arcgis.com/rest/geoenrichment/api-reference/geoenrichment-coverage.htm
For example, South Korea has country_code KR, so the link would be...
https://geoenrich.arcgis.com/arcgis/rest/services/World/geoenrichmentserver/Geoenrichment/Countries/KR?f=pjson
NOTE!!! This link has an important role in overall progress, so I would name it as: CountryInfoLink
Let's take a look inside CountryInfoLink.
Inside the key "hierarchies
", we only want the data boundary that defined by MBR research.
So please focus on the "datasets
" that contains "MBR" at CountryInfoLink. REMEMBER THIS(*)! ex) KOR_MBR_2018 for South Korea
Just below the "datasets
", "levelsInfo
" contains "geographyLevels
" which shows a list of unit area of collected demographics data.
The rightmost one -- in many case, 'Districts
'-- is the smallest unit area. REMEMBER THIS(**)! ex) Districts for South Korea
Please keep datasetID(*), and the smallest_unit_area(**) in your mind!
From here, we will use ArcGIS GeoEnrichment API (doc).
You need following parameters as (key : value).
- f : put pjson
- sourceCountry : put country_code you are interested in. ex) KR
- token : put your ArcGIS Developer token.
- geographylayers : put country_code.Country. ex) KR.Country
- geographyids : put ['01']
- returnSubGeographyLayer : put true
- subGeographyLayer : put country_code.smallest_unit_area. ex) KR.Districts
- DataSetID : put datasetID. ex) KOR_MBR_2018
- returnGeometry : put true
to make the API request.
You can visually check the response via Postman.
For example,
If you put in all parameters in your request, you can get a response from ArcGIS GeoEnrichment API.
As you can notice, each "features
" matches with each district(smallest_unit_area), and "rings
" returns coordinates of the boundary.
Each [A,B] implies [longitude,latitude] of the boundary.
This request returns whole district boundaries within the country(country_code).
This response is written in json format, so you could handle this boundary data information with additional codes.
Satellite imagery within the boundary
So far, we can download a satellite image for given [ zoomlevel / y tile index / x tile index ],
and we can get district boundaries, which are written in [latitude / longitude] for each district.
Now the problem is, we need a formula to convert two coordinates : (ZYX tile coordinate) <-> (latitude/longitude coordinate)
From OSM wiki, we can get a pseudocode for this.
Here is my python code example, for coordinate switching.
import math
def num2deg(xtile, ytile, zoom):
n = 2.0 ** zoom
lon_deg = xtile / n * 360.0 - 180.0
lat_rad = math.atan(math.sinh(math.pi * (1 - 2 * ytile / n)))
lat_deg = math.degrees(lat_rad)
return (lat_deg, lon_deg)
def deg2num(lat_deg, lon_deg, zoom):
lat_rad = math.radians(lat_deg)
n = 2.0 ** zoom
xtile = int((lon_deg + 180.0) / 360.0 * n)
ytile = int((1.0 - math.log(math.tan(lat_rad) + (1 / math.cos(lat_rad))) / math.pi) / 2.0 * n)
return (xtile, ytile)
From here, it is up to you to choose which images within the boundary.
In our paper, we decided to download the satellite image tile if more than three vertices contained inside the district boundary.
So in the sample figure, we download all image tiles as a daytime satellite imagery that "included" in each district.
reference : my AAAI-20 paper