With the growing use of computational text analysis in social sciences, topic modeling has become a key method for uncovering latent themes in textual data. However, its largely automated and inductive nature raises concerns about the validity of its results. Through a systematic review of 792 studies, this dissertation examines whether the field has converged toward standardized validation practices. Findings reveal a lack of consensus on validation methods, highlighting a mismatch between the qualitative nature of topic modeling and the quantitative tradition of standardized validation. The study advocates for greater transparency, detailed reporting, and qualitative validation approaches to enhance the credibility of findings in computational social science research.

As part of the DIGITIZE project (WP3), this research focuses on improving text representation in computational communication science. Using a dataset of over 5 million articles from Austrian media (2010–2022), we developed custom word embeddings with fastText, incorporating sub-word information and conducting extensive hyperparameter searches to ensure model robustness. Models were validated through semantic, syntactic, and classification tasks and compared with off-the-shelf models. A use case on gender bias demonstrated how model choice affects results in sensitive social issue analysis. This work highlights the importance of rigorous validation for credible text analysis in social sciences.

In connection with DIGITIZE (WP2 & WP3), this research examines how responses to open-ended survey questions differ from closed-ended ones. Analyzing 562 responses on trust in the Austrian coalition government, we identified significant discrepancies between the two formats, raising questions about the validity of traditional trust measures. Using large language models (LLMs) like GPT-4omini, we efficiently analyzed sentiment, actors, and topics in open-ended responses, revealing their unique ability to capture nuanced perspectives. This work underscores the value of open-ended questions in enriching survey data and improving the interpretation of public opinion.

This project examines how the Austrian media discourse on paternity leave has evolved over the past 30 years, focusing on the topics and narratives shaping public debate. Using Topic Modeling and Large Language Models (LLMs), we analyze trends and shifts in the conversation, providing insights into the societal and cultural dynamics surrounding paternity leave.

The AUTNES Media Side analyzes media coverage of Austrian politics during the eight weeks before elections. It examines how campaign themes and political personas are shaped by media and explores the impact of media messaging on voters compared to other communication channels. A key focus is the role of online political information in shaping public awareness and opinions.

Within the ySkills project, this study investigates how peer relationships shape the development of digital skills among children and adolescents. Surveying 212 classrooms across three European countries, we found that digital skills are shared through peer networks, often along gender lines—girls seek advice more but offer it less than boys. Perceived skills, not measured abilities, drive these interactions, while restrictive parental control limits participation in advice exchanges. The findings emphasize the need to harness peer learning in education and address gender divides in digital skills education.

This project explores how multilingual language embeddings (MLE) respond to linguistic and cultural differences in measuring gendered biases in Austrian and Israeli media discourses on political scandals. Analyzing German and Hebrew texts, we examine the semantic, pragmatic, and contextual equivalence in MLE mappings, identifying where and why linguistic and cultural differences lead to non-equivalence. Through a case study on the scandalization and resignation of male and female political officials, we aim to uncover the limitations of MLE for cross-lingual measurement and AI-based information processing.