MET Highlight Wikidata Mashup

Process

I started out by communicating with the Met API to return object data for the records that contained the 'isHighlight' code. I did this by including the request in the search url:

https://collectionapi.metmuseum.org/public/collection/v1/search?isHighlight=true&q=*

I used information from the Met API website to build this URL. Parsing through Met object records, I stored the returned data as individual JSON files in a directory named objects_json (using glob and os path). I then used glob to open up all of the object JSON files, creating a regular expression for extracting all of the object Wikidata QIDs from the data. I saved the Wikidata QIDs in their own JSON file.

In another script I formed my request to Wikidata's SPARQL endpoint using the triple quote string function in Python. I tested out this request in the SPARQL query service before inserting it into the code. I also used the QID JSON file and the python join function to include these QIDs in my SPARQL request. Additionally, I made sure that all of the Wikidata was returned as JSON. I then extracted the 'item', 'itemLabel', 'instanceOf', and 'instanceOfLabel' Wikidata properties from my list of QIDs.

Looping through the returned JSON to extract the values from the 'instanceOfLabel' key, I created an empty dictionary to count these values. I saved this dictionary into a separate JSON file. Finally, I made another script to sort the JSON count dictionary into individual dictionaries, making a different JSON file that was more digestible for Tableau visualization.

Challenges

My first challenge was constructing the query to the Met API. I had a lot of struggles with the format and the way the data is returned. The requests url has to include the relevant object ID in order to access the entire object record. This required making sure the url was not hardcoded with a particular object ID. I also had to figure out how to best recall all of the 'isHighlight' records when the API required the q parameter to be filled. Professor Matt Miller helped me with this in suggesting I populate that parameter with *.

I also struggled with the Wikidata endpoint. Despite my query functioning on the SPARQL website, it would not work in my python code. Professor Miller suggested replacing the GET request with a POST request, which finally returned the data.

Finally, I had never done data visualization before and had to teach myself how to use Tableau Public! In future iterations of the project I would definitely try for some more complex visualizations as I quite enjoyed the process.

Acknowledgements

This project would not have been possible without the help of Professor Matt Miller who teaches Pratt Institute School of Information's Programming for Cultural Heritage INFO 664 class!