CMPS 290L Fall 2014

Instructor: Luca de Alfaro

Email: luca@ucsc.edu

Chat, hangouts: luca@ucsc.edu

Place and time: Soc Sci 2 363 TTh 2-3:45 (note new location)

Office hours: Mondays 4-5 or by appointment

Discussion Group: Signup, Piazza group

This course explores the foundations of crowdsourcing and computer-mediated collaboration. The course covers the algorithmic and statistical foundations of crowdsourcing, introducing an analyzing algorithms, and experimenting with concrete systems. The algorithms presented will include algorithms for information classification, tagging, ranking, and acquisition.

The course also provides an introduction to computational systems for mediating user interaction and collaboration. In particular, the course will provide a foundation to the design of user reputation systems, and will discuss the issues of incentives to collaboration, and social dynamics induced by algorithm choices. The course will consist of a mix of theoretical lectures and paper discussions. The assigned homework will be a mix of theoretical assignments, data analysis assignments, and programming problems where students are asked to implement and experiment with algorithms.

Readings

List of advised reading corresponding to lectures.

Also, here is material from last year's edition of the class, which may still be useful.

• Papers that will be discussed in class
• Class notes
  • Data on CrowdSourcing experiments (shared with individual students only).
  • The Vancouver algorithm code, in GitHub

You need to be logged in via your @ucsc.edu account to access the above folders.

Project

• List of projects
• Folder with project information

Please add a description of your project to the above list of projects, and I will add pointers to data and code to that document.

You need to be logged in via your @ucsc.edu account to access the document.

You can use the folder to create a document just for your project. If you write in that documents which experiments you have done and the results you have produced, I can read the document and keep up with the progress, and give you suggestions for things to try.

Homework

• Homework 1, due Thursday October 9 in class.

Syllabus

An approximate syllabus is as follows:

• Bayesian approach to user reputation modeling.
• Learning user reputation in labeling and tagging tasks.
• Graph-theoretical approaches to reputation: PageRank, HITS, and variants.
• Attacks and defenses to graph reputation systems.
• Chronological reputation systems: the “useful work” principle, robust ranking.
• Open problems: democracy vs. oligarchy in online communities, size of community and of information.
• Introduction to crowdsourcing
• Crowdsourcing a rank: algorithms and statistical analysis
• Crowdsourcing quality ratings
• Crowdsourcing labeling and tagging tasks
• Incentives in crowdsourcing.
• Mechanism design in game theory: a brief introduction. The revelation principle, VCG framework.
• Truthful incentives for crowdsourced ranking.
• Reputation systems and crowdsourcing: leveraging user information in task assignment and result aggregation.
• The role of machine learning in crowdsourcing. Practical approaches to machine-learning for user and task classification.