Visualizing Algorithmic Selection in Social Media

Summary

Social media sites such as Facebook and Twitter use algorithms to filter information in order to reduce overload and selectively pick content for users. These algorithms create unique, individual, and isolated bubbles of information that users are not always aware of. We recommend that algorithmic awareness should be the first step in addressing the pitfalls of the filter bubble effect. We conducted an experimental study to investigate how simple visualizations can be used to achieve algorithmic awareness and to understand how it might influence users’ behavior. The visualizations did not lead to increased understanding of the algorithm per se, but its presence created interesting effects that will inform future studies.

I presented this preliminary study as a poster/extended abstract at ACM CSCW 2019.

Motivation

• If people are not aware that they are consuming information that has been algorithmically selected for them, they might consider perspectives they encounter online as the only side to an issue, causing an over attribution of truthfulness. Further online interactions that reflect this stance will only increase the echo chamber effect
• If people are aware of such algorithms and its effects, at least at a high level, they can contextualize the information they encounter online. Recommender systems such as Netflix and Amazon usually contain descriptive texts such as "Because you viewed/watched ...". This enables people to be aware of targeted content delivery even though they might not understand the underlying mechanisms by which it is achieved.

The interfaces and interactions in Facebook and Twitter do not make the the presence of algorithms obvious to its users.

Research Question

• How can simple visualizations of filtered content convey the presence and purpose of filtering algorithms to users?

To explore this RQ, I designed and conducted an experimental study within a simplified, simulated social media platform. The intervention was in the form of data visualizations that depicts how the algorithm in this platform works.

Study Design

Prototype

We developed a prototype using tweets from Members of the United States (US) Congress, the legislative branch of the US government. To minimize confounding factors, we only used tweets that pertained to the issue of healthcare insurance in the US, a topic of wide public interest where the two major US political parties have clear and conflicting viewpoints. We selected tweets from a particular time period when Congress Members were most vocal about the issue. We tagged the tweets as Republican or Democratic, based on which party the member tweeting it belonged to.

The left side of the prototype interface displays the tweets, the Congress member who tweeted it, and functions to like, re-tweet, and save for later (see Figure 1). The right side of the interface is either blank or features a donut chart visualization. The visualization is filled with two colors, blue for the Democratic party and red for the Republican party (see Figure 2). Since the first page of tweets starts with equal number of tweets from both parties, the visualization represents the two colors in equal proportions.

The prototype shows six pages of tweets in total with ten tweets per page. The user is prompted to like at least two tweets and re-tweet one in each page in order to progress to the next. If the user likes or re-tweets a party member’s tweets, the visualization reacts instantly, adding more of the corresponding party color. These interactions (with re-tweets weighted more than likes) increase the number of tweets that a user sees from that party in the next page of tweets. This selection is implemented through a simple sliding window algorithm and indicated by the visualization.

Method

• In a pre-experiment questionnaire, we gather data from the participants on their demographics, political leaning, and social media use.
• The visualization was hidden for the control group, and displayed to the experimental group. By comparing the data from these groups, we wanted to determine if our intervention enabled experimental group users to understand that content was curated for them.
• In the post-experiment questionnaire, we test understanding of the prototype, its algorithm, and the visualization.
• Detailed log data provides an account of participant activity during the experiment.
• The study was conducted with eight participants using a convenience sample of undergraduate or graduate students from the university.

Findings

1. Three out of four participants in the control group (without the visualization) stated they saw tweets from both parties equally, though they were shown more tweets from one party (43 tweets from one party, 17 from the other, over 6 pages). The fourth participant was able to discern that they had seen more of one party’s tweets.
2. Two out of four participants in the experimental group answered that they had seen both parties tweets equally. The third participant figured out which party’s tweets they had seen more of, but the fourth participant guessed the wrong party though they knew content was algorithmically selected.

We could not confirm our hypothesis (p-value > 0.05).

Discussion

A major drawback of running an analysis in this stage was having a small no.of participants (at least 20).

We wanted to know what was going on here since there were mixed responses from the participants in the experimental group. We wanted to find out how participants perceived the visualization. The debrief session of the experiment helped us answer the questions we had.

• Two participants said they found the visualization particularly useful when they did not get enough meaning or context from the tweets. We found from the logs that they would like and unlike a tweet to understand how it was politically leaning from the way the colors moved in the visualization. The visualization made it easier for them to figure out which tweets were coming from which party!
• Another participant used the visualization to ensure that they were not accidentally supporting tweets that did not align with their views by checking the colors in the visualization. It made them confident of their endorsement.
• There seems to be uncertainty over whether the visualization exactly represented - the participant’s interactions (like/re-tweet) or the feed’s ratio of tweets.
• We did not observe any behavioral change. Our debrief session helped us understand that it was important for participants to continue supporting tweets that aligned with their values and convictions, similar to what has been reported by previous studies.

Next Steps

Though we could not test our hypothesis, we gathered useful data that explained users’ mental model of how the visualization might have worked and what they think its purpose might have been.

A few future research directions:

1. People hold their values and convictions very closely. Sometimes they do not want to be exposed to diverse views or rather, views that are contrary to their beliefs. How can we decouple that from this research study?
2. Investigating how visualizations can be interpreted definitely especially when used in such contexts. For example, it was reinforcing confirmation bias in this study.
3. What are other data and/or research methods that we can use to study such questions? Experiments such as this one are only a simulation of real-world practices and behaviors.