CMPS 290G Winter 2014

Instructor: Luca de Alfaro

Email: luca@ucsc.edu

Chat, hangouts: luca@ucsc.edu

Place and time: Porter Acad 241 MW from 2-3:45pm.

Office hours: Mondays 4-5 or by appointment

This course explores the foundations of crowdsourcing and computer-mediated collaboration. The course covers the algorithmic and statistical foundations of crowdsourcing, introducing an analyzing algorithms, and experimenting with concrete systems. The algorithms presented will include algorithms for information classification, tagging, ranking, and acquisition.

The course also provides an introduction to computational systems for mediating user interaction and collaboration. In particular, the course will provide a foundation to the design of user reputation systems, and will discuss the issues of incentives to collaboration, and social dynamics induced by algorithm choices. The course will consist of a mix of theoretical lectures and paper discussions. The assigned homework will be a mix of theoretical assignments, data analysis assignments, and programming problems where students are asked to implement and experiment with algorithms.

Readings

These readings are from last year's edition of the class, and they will be updated as the class proceeds.

• Papers that will be discussed in class
• Class notes
  • Data on CrowdSourcing experiments (shared with individual students only).
  • The Vancouver algorithm code, in GitHub

Photos of the blackboard.

You need to be logged in via your @ucsc.edu account to access the above folders.

Lectures

• Lecture 1 (Jan 13): Content-driven reputation. Read this paper.

Syllabus

An approximate syllabus is as follows:

• Bayesian approach to user reputation modeling.
• Learning user reputation in labeling and tagging tasks.
• Graph-theoretical approaches to reputation: PageRank, HITS, and variants.
• Attacks and defenses to graph reputation systems.
• Chronological reputation systems: the “useful work” principle, robust ranking.
• Open problems: democracy vs. oligarchy in online communities, size of community and of information.
• Introduction to crowdsourcing
• Crowdsourcing a rank: algorithms and statistical analysis
• Crowdsourcing quality ratings
• Crowdsourcing labeling and tagging tasks
• Incentives in crowdsourcing.
• Mechanism design in game theory: a brief introduction. The revelation principle, VCG framework.
• Truthful incentives for crowdsourced ranking.
• Reputation systems and crowdsourcing: leveraging user information in task assignment and result aggregation.
• The role of machine learning in crowdsourcing. Practical approaches to machine-learning for user and task classification.