At the start of my second year in graduate school, I took a qualitative research methods course taught by my faculty advisor, Dr. Ruth Walker. Many graduate programs do not offer a strictly qualitative course, which made me excited to gain advanced knowledge and experience to add more methodologies to my current research skills. Qualitative research offers a unique lens to analyze data and provides rich data to better understand individual and group experiences as well as discover underlying meanings of certain phenomena. Researchers can reveal hidden patterns, motivations, and influences that quantitative approaches may overlook. In this post, I am going to provide you with more information on a specific qualitative method that I both learned and also conducted a study using - content analysis. Our class was divided into research teams based upon our interests, and each team had a different project using a different qualitative method. My team consisted of myself and one other researcher using a content analysis to better understand what public universities are communicating about the LGBTQ+ community on their social media accounts. 

Our Research Process (Lune & Berg, 2017)

1) Define our research question/objective: Gain a better understanding of what universities are posting (and not posting) about the LGBTQ+ community on the social media accounts.

2) Select content to analyze: We conducted (with assistance from research assistants in Dr. Walker's lab) web scrapping of 100 public university Instagram accounts. 

3) Reflexivity

1. • Qualitative researchers practice reflexivity to recognize and address their biases to foster transparency and validity by evaluating how personal assumptions may influence the research process and findings. 

   • “Maintaining a self-critical attitude and questioning taken-for-granted assumptions regarding the political nature of our work and its (intended and unintended) effects, as well as the social distribution of these effects” (Poland et al., 2006, p. 61)

   • More information on how we engaged in this process is included below.

4) Familiarized ourselves with the data.

5) Create codes to best identify our research question.

6) Coded data from established codes.

7) Conducted frequency counts while coding.

1. • Content analysis can encompass both quantitative and qualitative components. Frequency counts on each code can aid in a better understanding and interpretation of themes and concepts within the data.

8) Reevaluated the content and make our codes more concise.

9) Analyzed coded data to identify themes, trends, or patterns (i.e., manifest vs latent content).

10) Interpreted our findings and wrote up the analysis. 

1. • Tied in previous research.

   • Drew conclusions and practical implications.

** For more in-depth information on the steps involved within a content analysis, please read “Qualitative Research Methods for the Social Sciences” by Lune & Berg (2017) 

I am now going to give you a closer look into the methods of my group research project titled, “Visibility Matters: A Content Analysis of LGBTQ+ Messages on Public University Instagram Accounts”

My research partner and I (John DiClementi) conducted a content analysis to gauge college university LGBTQ+ friendliness towards students on social media (i.e., Instagram). Before collecting Instagram posts, universities were determined by looking at a geographic map of the U.S. Our sample selection was 100 Instagram accounts from public universities in the U.S. We selected two public universities from each state. However, there was an exception, as Wyoming only has one public university, so a third university was added from the state of Texas. The data collection period was from the end of 2021 (11/1/21) through the end of 2022 (10/31/22).   

We downloaded all posts from the campus universities' Instagram accounts (all IG accounts were set as public), including photos, videos, and captions, and we stored them in an Excel database file. Each university in the sample was assigned a unique identification number from 1 to 100. We also collected and recorded metrics such as likes, comments, video views, post format (single image, multiple images, video, or a combination), and post text, including hashtags, emojis, and links embedded within the text along with the date of the post. Private institutions were excluded from this study to prevent potential ethical and policy-related complexities associated with private universities. Additionally, we did not collect Instagram stories (temporary photos or videos on a user’s profile that disappear after 24 hours) due to them vanishing after a 24-hour period for reliability purposes.

Reflexivity

During the content analysis, both John and I practiced reflexivity, maintaining self-awareness to ensure the transparency and validity of our findings. This involved critically examining how our own perspectives might influence data interpretation. In the write up of our results, we included the following author positionally statement, “In the exploration of LGBTQ+ friendliness across public universities in the U.S., we recognize that our understanding and interpretation of how these institutions promote LGBTQ+ students, faculty, staff, and the community, are shaped by our individual identities, diverse backgrounds, and experiences. Both authors are graduate students and teach at a public university that has an annual enrollment of more than 11, 000 undergraduate and graduate students. The two authors are in-state residents, one is a first-generation college student and one is a third-generation college student. Both authors identify as young, white, cisgender adults, one man and one woman. One identifies as straight and one as a member of the LGBTQ+ community."

Data Analysis  

We used content analysis to categorize, quantify, and describe the content within our dataset, such as text, images, or audio, to better understand patterns, themes, and underlying characteristics within the data content. The data analysis process for this project involved weekly meetings between the research team to code the text captions and images and videos of each post. During each meeting we analyzed approximately 15 posts, for a total of seven meetings to code all the data. Our coding was conducted primarily through a manifest content analysis (i.e., explicit) where we closely examined the exact words, phrases, and content in the photos and videos, to describe what was being explicitly expressed. In our manifest analysis, we systematically moved through the data to identify categories and themes, using the exact language and content. This approach allowed us to emphasize a conscious awareness of referencing the original text to closely adhere to the original meanings and contexts. John and I independently coded the data, utilizing our own codebook. Simultaneously, we engaged in collaborative discussions, addressing and reconciling any agreements or disagreements among grouping of the codes, while also establishing new codes. This approach served as a crucial step to establish interrater reliability (which is the level of agreement between the researchers when independently coding). Cohen’s Kappa was calculated to rate interrater reliability (k = 0.94), demonstrating consistency in our data interpretation. Frequency counts were also utilized to assess the number of universities associated with each code.  

The next step in our data analysis consisted of a latent content analysis to explore the underlying meanings of universities that remained inactive or vague within posts related to promoting and supporting LGBTQ+ students, faculty, staff, and the community on campus. Latent content analysis (i.e., implicit meaning) involves a systematic exploration of textual or visual data to reveal concealed themes, patterns, and meanings. This method empowers researchers to gain novel insights into the underlying concepts, beliefs, or motivations embedded within the content. We analyzed the posts’ number of likes, number of institutions present for each code, and means and standard deviation across the universities. Latent content analysis goes beyond mere quantitative measures, such as counting likes and the frequency of the institutions for each subsequent code; it goes beyond to reveal concealed meanings and provide valuable insights within the data. Thus, resulting in a richer and more nuanced comprehension of rating LGBTQ+ acceptance, ongoing community support, and commitment to inclusivity of these public universities. 

Results from Content Analysis

After analyzing the data, we categorized our results into three main categories: (1) active LGBTQ+ relevant messages, (2) passive LGBTQ+ relevant messages, and (3) failure to engage in LGBTQ+ relevant messaging. Please see below for the poster we presented at the Association for Women in Psychology (AWP) 2024 Conference for more detailed information of our findings.

Content Analysis Example https://static1.squarespace.com/static/5b0ee82df793927c77add8b6/t/610574ad7391264f2aed0392/1627747501781/Abusive%20Beats%20Garland%20Revised.pdf

Latent Content Study

https://palliativereferences.sites.stanford.edu/publications/hispaniclatino-americans/promoting-health-pulpit-process-evaluation-hiv-sermons-reduce

References

• https://www.researchgate.net/publication/295303146_How_to_plan_and_perform_a_qualitative_study_using_content_analysis

• https://www.tandfonline.com/doi/abs/10.1080/07399339209516006

• https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1468-2958.2004.tb00738.x