July 17, 2019 @ University of Amsterdam, The Netherlands


Tutorial at the International Conference on Computational Social Science  

Human values shape our decisions, our views of the world, and ourselves. This tutorial brings to light latest research on modeling human values through technological footprints, revealing individual, societal, and cultural forces. How these forces can be captured and utilized in the design of persuasive technologies, promoting fairness, sensitivity and inclusion is an ongoing conversation.

Kyriaki Kalimeri
Ph.D. Brain and Cognitive

Yelena Mejova
Ph.D. Computer



Understanding human values with an empirical approach, both from a qualitative and quantitative point of view, allows us to better model behaviours, actions, and attitudes towards social phenomena. It is invaluable in the design of, for instance, effective health interventions - such as encouraging vaccination - or even appropriate communication campaigns for policy making - such as sensibilization towards pro-environmental attitudes. This is important since public debate on human values often focuses on perceived threats to different values while rarely understanding or articulating how values are inferred from people’s behaviors and judgements.

In this tutorial, we give an overview of how the basic human and moral values are interpreted according to the psychological literature
, as a combination of individual, societal, and cultural forces. We discuss the latest research in assessing these through both traditional methods, as well as through quantitative methods applied to digital data. In the first part, we provide an overview of traditional survey methods, and discuss their applicability to the new forms of discourse, the validity of recruitment using the Internet and new opportunities this medium holds.

In the second part, we consider several case studies of applying computational methods to large amounts of social media data for understanding values associated with specific domains, including politics
, health, charitable giving, and privacy, and discuss how social media can capture the behavioral differences in large populations of different values. Here, we introduce methodologies for large scale data analysis including topic discovery, topic refinement, grounded theory labeling, network science, and regression modeling.

We conclude with the discussion of ethical use of such modeling, including data and model bias, informed consent, intervention design, and the use of persuasive technology.

Kyriaki Kalimeri is a Researcher at the ISI Foundation, Turin, Italy. She received her PhD in Brain and Cognitive Sciences from the University of Trento and her Diploma in Electrical and Computer Engineering from the Technical University of Crete. Her research lies at the intersection of computational social science, social media analysis, and machine learning. The focus is on the automatic prediction of psychological characteristics and moral worldviews from digital data, employing machine learning techniques, translating data into insights for the design of effective communication strategies. She co-organized the Social Media and Health workshop in ICWSM’18.

Yelena Mejova is a Research Leader at the ISI Foundation, Turin, Italy, part of the Digital Epidemiology group. Previously, she was a Scientist in the Social Computing Group at the Qatar Computing Research Institute (QCRI). Specializing in social media analysis and mining, her work concerns the quantification of health and wellbeing signals in social media, as well as tracking of social phenomena, including politics and news consumption. She co-edited a volume on the use of Twitter for social science in Twitter: A Digital Socioscope and has recently organized tutorials on the use of social media for health-related research at ICWSM’17 and IC2S2’17, as well as the Health workshop at ICWSM’18. She has given talks on the subject internationally, including Yandex’s Yet Another Conference in Moscow and American University of Beirut.


  • Shalom H Schwartz. 2012. An overview of the Schwartz theory of basic values.Online Readings in Psychology and Culture 2, 1 (2012), 11.
  • Haidt, J., 2003. The moral emotions. Handbook of affective sciences, 11(2003), pp.852-870.
  • Jonathan Haidt and Craig Joseph. 2004. Intuitive ethics: How innately prepared intuitions generate culturally variable virtues. Daedalus 133, 4 (2004), 55–66.
  • Graham, J., Haidt, J., Koleva, S., Motyl, M., Iyer, R., Wojcik, S.P. and Ditto, P.H., 2013. Moral foundations theory: The pragmatic validity of moral pluralism. In Advances in experimental social psychology (Vol. 47, pp. 55-130). Academic Press.
  • Iyer, R., Koleva, S., Graham, J., Ditto, P. and Haidt, J., 2012. Understanding libertarian morality: The psychological dispositions of self-identified libertarians. PloS one, 7(8), p.e42366.
  • Feldman, G. Personal values and moral foundations: Examining relations and joint prediction of moral variables.
  • McAdams, D., & Pals, J. (2006). A new big five: Fundamental principles for an integrative science of personality. American Psychologist , 61 , 204
  • Wolsko, C., Ariceaga, H., & Seiden, J. (2016). Red, white, and blue enough to be green: Effects of moral framing on climate change attitudes and conservation behaviors. Journal of Experimental Social Psychology, 65 , 7 - 19. http://www.sciencedirect.com/science/article/pii/S0022103116301056. doi:http://dx.doi.org/10.1016/j.jesp.2016.02.005.
  • Low, M., & Wui, M. G. L. (2015). Moral foundations and attitudes towards the poor. Current Psychology, (pp. 1 - 11). URL: http://dx.doi.org/10.1007/s12144-015-9333-y. doi:10.1007/s12144-015-9333-y.
  • Amin, A. B., Bednarczyk, R. A., Ray, C. E., Melchiori, K. J., Graham, J., Huntsinger, J. R., & Omer, S. B. (2017). Association of moral values with vaccine hesitancy. Nature Human Behaviour , 1 , 873 - 880. doi:10.1038/s41562-017-0256-5.
  • Kalimeri, K., Beiró, M.G., Urbinati, A., Bonanomi, A., Rosina, A., & Cattuto, C. (2019). Human Values and Attitudes towards Vaccination in Social Media. WWW.
  • Hoover, J., Johnson, K., Boghrati, R., Graham, J., & Dehghani, M. (2018). Moral framing and charitable donation: Integrating exploratory social media analyses and confirmatory experimentation. Collabra: Psychology, 4(1).
  • Santos et al. Global Increases in Individualism. Psychological Science, 2017
  • Miriam J. Metzger, Privacy, Trust, and Disclosure: Exploring Barriers to Electronic Commerce, Journal of Computer-Mediated Communication, Volume 9, Issue 4, 1 July 2004, JCMC942, https://doi.org/10.1111/j.1083-6101.2004.tb00292.x
  • Heath,  R. L. & Bryant,  J. (1992). Human communication theory and research . Hillsdale, NJ: Lawrence Erlbaum.
  • Roloff,  M. E. (1981). Interpersonal communication: The social exchange approach . Beverly Hills, CA: Sage.
  • Snyder, M., Tanke, E. D., & Berscheid, E. (1977). Social perception and interpersonal behavior: On the self-fulfilling nature of social stereotypes. Journal of Personality and Social Psychology, 35(9), 656-666.
  • Stone,  E. F., Gueutal,  H. G., Gardner,  D. G., & McClure,  S. (1983). A field experiment comparing information-privacy values, beliefs, and attitudes across several types of organizations. Journal of Applied Psychology , 68(3), 459–468.
  • Nyhan, B., Reifler, J., Richey, S. & Freed, G. L. Effective messages in vaccine promotion: a randomized trial. Pediatrics 133, E835–E842 (2014).
  • Opel, D. J. et al. The influence of provider communication behaviors on parental vaccine acceptance and visit experience. Am. J. Public Health 105, 1998–2004 (2015).
  • Inglehart, R., C. Haerpfer, A. Moreno, C. Welzel, K. Kizilova, J. Diez-Medrano, M. Lagos, P. Norris, E. Ponarin & B. Puranen et al. (eds.). 2014. World Values Survey: Round Six - Country-Pooled Datafile Version: http://www.worldvaluessurvey.org/WVSDocumentationWV6.jsp. Madrid: JD Systems Institute.
  • Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.F. and Rahwan, I., 2018. The moral machine experiment. Nature, 563(7729), p.59.
  • Melissa A Baker, Paul Fox, and JD Twila Wingrove. Crowdsourcing as a forensic psychology research tool. American Journal of Forensic Psychology, 34(1):37, 2016.
  • Tara S Behrend, David J Sharek, Adam W Meade, and Eric N Wiebe. The viability of crowdsourcing for survey research. Behavior research methods, 43(3):800, 2011.
  • Michael Buhrmester, Tracy Kwang, and Samuel D Gosling. Amazon’s mechanical turk: A new source of inexpensive, yet high-quality, data? Perspectives on psychological science, 6(1):3–5, 2011.
  • Matthew J C Crump, John V Mcdonnell, and Todd M Gureckis. Evaluating Amazon’s Mechanical Turk as a Tool for Experimental Behavioral Research. PloS one, 8(3), 2013.
  • Laura Germine, Ken Nakayama, Bradley C. Duchaine, Christopher F. Chabris, Garga Chatterjee, and Jeremy B. Wilmer. Is the web as good as the lab? comparable performance from web and lab in cognitive/perceptual experiments. Psychonomic Bulletin & Review, 19(5):847–857, 2012.
  • Samuel D Gosling and Winter Mason. Internet research in psychology. Psychology, 66,2015.
  • Edith Law, Andrea Wiggins, Mary L Gray, and Alex Williams. Crowdsourcing as a Tool for Research : Implications of Uncertainty. In Proceedings of the 20th ACM Conference on Computer Supported Cooperative Work and Social Computing (To appear). ACM, 2017.
  • Winter Mason and Siddharth Suri. Conducting behavioral research on Amazon’s Mechanical Turk. Behavior research methods, 44(1):1–23, 2012.
  • Golder, S. A., & Macy, M. W. (2014). Digital footprints: Opportunities and challenges for online social research. Annual Review of Sociology, 40, 129-152.
  • Macy, M. W. (2015). An Emerging Trend: Is Big Data the End of Theory?. Emerging Trends in the Social and Behavioral Sciences: An Interdisciplinary, Searchable, and Linkable Resource, 1-14.
  • Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016). Social data: Biases, methodological pitfalls, and ethical boundaries. Methodological Pitfalls, and Ethical Boundaries (December 20, 2016).
  • Kyriaki Kalimeri, Mariano G. Beiro, Andrea Bonanomi, Alessandro Rosina & Ciro Cattuto. Evaluation of Biases in Self-reported Demographic and Psychometric  Information: Traditional versus Facebook-based Surveys arXiv URL:  https://arxiv.org/abs/1901.07876
  • Zeynep Tufekci. Big questions for social media big data: Representativeness, validity and other methodological pitfalls. ICWSM, 14:505–514, 2014.
  • Hoover, J., Portillo-Wightman, G., Yeh, L., Havaldar, S., Davani, A. M., Lin, Y., … Dehghani, M. (2019, April 10). Moral Foundations Twitter Corpus: A collection of 35k tweets annotated for moral sentiment. https://doi.org/10.31234/osf.io/w4f72
  • Boyd, R. L., Wilson, S. R., Pennebaker, J. W., Kosinski, M., Stillwell, D. J., & Mihalcea, R. (2015, April). Values in words: Using language to evaluate and understand personal values. In Ninth International AAAI Conference on Web and Social Media.
  • Novak, P. K., Smailović, J., Sluban, B., & Mozetič, I. (2015). Sentiment of emojis. PloS one, 10(12), e0144296.
  • Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends® in Information Retrieval, 2(1–2), 1-135.
  • Araque, O., Gatti, L., & Kalimeri, K. (2019). MoralStrength: Exploiting a Moral Lexicon and Embedding Similarity for Moral Foundations Prediction. arXiv preprint arXiv:1904.08314.
  • Chen, J., Hsieh, G., Mahmud, J. U., & Nichols, J. (2014, February). Understanding individuals' personal values from social media word use. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (pp. 405-414). ACM.
  • Kalimeri, K., Beiró, M. G., Delfino, M., Raleigh, R., & Cattuto, C. (2019). Predicting demographics, moral foundations, and human values from digital behaviours. Computers in Human Behavior, 92, 428-445.
  • Michael F. Schober, Josh Pasek, Lauren Guggenheim, Cliff Lampe, and Frederick G. Conrad. Social media analyses for social measurement. Public Opinion Quarterly, 80(1):180–211, 2016
  • Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111(24), 8788–8790.
  • Ratkiewicz, Jacob, et al. "Truthy: mapping the spread of astroturf in microblog streams." Proceedings of the 20th international conference companion on World wide web. ACM, 2011.
  • Davis, Clayton Allen, et al. "Botornot: A system to evaluate social bots." Proceedings of the 25th International Conference Companion on World Wide Web. International World Wide Web Conferences Steering Committee, 2016.
  • De Choudhury, M., Kiciman, E., Dredze, M., Coppersmith, G., & Kumar, M. (2016, May). Discovering shifts to suicidal ideation from mental health content in social media. In Proceedings of the 2016 CHI conference on human factors in computing systems (pp. 2098-2110). ACM.
  • Stephens-Davidowitz, Seth, and Andrés Pabon. Everybody lies: Big data, new data, and what the internet can tell us about who we really are. New York, NY: HarperCollins, 2017.
  • Mooijman, M., Hoover, J., Lin, Y., Ji, H., & Dehghani, M. (2018). Moralization in social networks and the emergence of violence during protests. Nature Human Behaviour, (p. 1).
  • Magdy, Walid, Kareem Darwish, and Ingmar Weber. "#FailedRevolutions: Using Twitter to study the antecedents of ISIS support." First Monday 21(2) (2016).
  • Angus Campbell, Converse Philip, Miller Warren, and Stokes Donald. 1960. The American Voter. University of Chicago Press.
  • Le, H. T., Boynton, G. R., Mejova, Y., Shafiq, Z., & Srinivasan, P. (2017, May). Revisiting the american voter on twitter. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 4507-4519). ACM.
  • Mejova, Y., Haddadi, H., Noulas, A., & Weber, I. #Foodporn: Obesity patterns in culinary interactions. In Proceedings of the 5th International Conference on Digital Health 2015 (pp. 51-58). ACM.
  • Simpson, Courtney C., and Suzanne E. Mazzeo. "Skinny is not enough: A content analysis of fitspiration on Pinterest." Health communication 32, no. 5 (2017): 560-567.
  • Yelena Mejova. Information Sources and Needs in the Obesity and Diabetes Twitter Discourse. ACM International Conference on Digital Health (DH), 2018.
  • Yelena Mejova, Youcef Benkhedda, Khairani. #Halal Culture on Instagram. Frontiers in Digital Humanities: Big Data, 02. 2017.
  • Norah Abokhodair, Sofiane Abbar, Sarah Vieweg, Yelena Mejova. Privacy and Social Media Use in the Arabian Gulf: Saudi Arabian & Qatari Traditional Values in the Digital World. The Journal of Web Science 3 (1), 2017.
  • Magdy, Walid, Kareem Darwish, Norah Abokhodair, Afshin Rahimi, and Timothy Baldwin. "# isisisnotislam or# deportallmuslims?: Predicting unspoken views." In Proceedings of the 8th ACM Conference on Web Science, pp. 95-106. ACM, 2016.
  • Swigger, N. (2013). The online citizen: Is social media changing citizens’ beliefs about democratic values? Political Behavior, 35(3), 589–603.
  • Kurpad, Sunita Simon. "Ethics in psychosocial interventions." Indian journal of psychiatry 60.Suppl 4 (2018): S571.