Web Scraping and the IEA Database
Wait! What is web scraping? Learn more about it here.
As part of the COMPLIANCE project funded by the BMBF (the German Federal Ministry of Education and Research), I have been working on a research project that examines the evolution of the design of international environmental treaties (IEAs).
As part of this project, I have been using data from the IEA Database, which is an amazing database put together by Ronald Mitchell (University of Oregon) in 2002 which includes over 1,300 MEAs, over 2,200 BEAs, and 250 other environmental agreements that are constantly being revised and updated.
My first thought was to figure out a way to web scrap the data I needed.
I love web scraping. It has an entry cost, but it can save you a crazy amount of time and spare you a lot of efforts.
Unfortunately, the IEA Database was not designed to make things easy for me at the time. So instead of going through the hassle of collecting the data manually, I decided to complain about it (and ended up collecting the data manually anyway).
But luckily for the future generations, Prof Mitchell is committed to make our lives easier, and after a few exchanges about what could be improved in this regard, it is now possible to retrieve data from the IEA Database using simple web scraping techniques (You are welcome).
Hence, I wanted to propose a series of web scraping tutorials (alongside with the code) specifically designed to retrieve data from the IEA Database using Python.
The tutorials are available HERE.