How can language affect results for

cities in Google Images?

by Sam Rosevear

Project work with Yiping Zhong, Richard Tian, Mohamed Abdi Mohamed

Prof. Shilad Sen

Collective Intelligence

Macalester College

Fall 2020

Core Idea:

  • The goal of our project was to gauge how the perception of a city might change in Google Images depending on what language is used for the search term.

  • We set out to compare the image results of four different cities, using each of their own native languages as sample languages.

  • Our Cities:

    • Mogadishu, Somalia

    • Beijing, China

    • Istanbul, Turkey

    • New York City, USA

  • We searched each city in each language. For example, for Mogadishu we entered the following four searches:

    • moqadishu (Somali)

    • mogadishu (English)

    • mogadişu (Turkish)

    • 摩加迪沙 (Chinese)

  • We repeated this process for the other three cities. To the left is an image from our code showing each search term and its language.

Image Scraping:

  • For each of our search terms (each city in each language) we wanted to gather the first 100 image results:

    • We used the Selenium API for help with scraping, using it to access Google Search in our code.

    • We then entered one of our search terms and went to Images.

    • Then we inspected each image's source url, and saved the url's to a list.

mogadişu - Turkish

摩加迪沙 - Chinese

Now that we have our data, how do we actually measure the perception of the cities' images?

Our answer: Google Cloud Vision API

  • The Cloud Vision API is pre-trained, and provides image labeling. This basically shows users what the prominent entities in an image are.

  • Below is an example of how it works, including a Label and Confidence score:

  • By running the Cloud Vision API on each of our search terms' image result data, we were able to compare which labels occurred most frequently.

  • Comparing the popular labels returned by each search term showed us how the perception of the cities changes in Google Images depending on the language used.

  • We stored our image data in .csv files so that our code would run efficiently for this process.

Case Study: Mogadishu

  • We gathered the most popular labels returned for Mogadishu in each language.

  • Then, we visualized them with WordClouds (the bigger the word, the more popular the label) and bar graphs that measured the most frequent labels. We also included bar graphs of the average confidence scores of the top 10 labels.

  • Visualizations were created in Google Colab using Python with Pandas.

  • Below are our visualizations for the Mogadishu Image Label data:

Mogadishu - English

Mogadishu - Chinese

Mogadishu - Turkish

Mogadishu - Somali

Analysis:

As the WordClouds and graphs show, there are tangible differences in perception depending on the language used to search Mogadishu. For example, when we searched in Chinese or Turkish, the label 'Pollution' was the most frequent. However, when Mogadishu was searched in its native language (Somali), 'Pollution' doesn't even occur in the top 10 most frequent labels. There is also a lack of 'Military Person' and 'Military Uniform' labels for the Somali language visualizations, while they both occur in the top 10 most frequent labels of the Turkish and Chinese searches. Also notable is the English search data visualizations, which returned more of an emphasis on tourism and landmarks than any other search. Based on our visualizations, it is clear that searching in different languages leads to altered Google Image results for Mogadishu.

The visualizations of the image label data from our other three cities can be seen below:

Note: For our other cities, some search terms were the same in two languages. For example, a graph might be labeled "English and Somali" instead of only one language.

Istanbul

New York City

Beijing

Additional Thoughts:

  • A notable takeaway from our visualizations is that some cities are more homogenous than others, even with the different language usage. New York City's labels, for example, are similar across the four languages. The perception of New York City from a Google Image search doesn't seem to change much when searched in these languages.

  • We also found that the top couple of most frequent labels were generally not where we made our most interesting observations. As an example, the label 'Dome' is the top label for all three different search terms for Istanbul. This is expected, as some cities with a major, iconic landmark will return tons of image results showcasing said landmark. However, the slightly less frequent labels revealed interesting differences. For Istanbul, the English/Somali search term resulted in more of an emphasis on the city's waterways and watercraft, while the Chinese search term did not yield a water-related label in the top ten. We enjoyed noticing smaller differences in perception like that throughout our analysis.