Workshop Schedule

Sunday, June 6, 2010
9:00am    Morning Session
       Creating Speech and Language Data With Amazon’s Mechanical Turk
       Chris Callison-Burch and Mark Dredze

9:10am   Invited Talk by CrowdFlower

10:10am    Morning Presentation Session
        Corpus Creation for New Genres: A Crowdsourced Approach to PP Attachment
        Mukund Jha, Jacob Andreas, Kapil Thadani, Sara Rosenthal and Kathleen McKeown

10:30am    Break

11:00am    Morning Poster Session

Note: morning talks will also be presented as posters in this session.

Clustering dictionary definitions using Amazon Mechanical Turk

Gabriel Parent and Maxine Eskenazi

Semi-supervised Word Alignment with Mechanical Turk
Qin Gao and Stephan Vogel

Rating Computer-Generated Questions with Mechanical Turk
Michael Heilman and Noah A. Smith

Crowdsourced Accessibility: Elicitation of Wikipedia Articles
Scott Novotney and Chris Callison-Burch

Document Image Collection Using Amazon’s Mechanical Turk
Audrey Le, Jerome Ajot, Mark Przybocki and Stephanie Strassel

Using Amazon Mechanical Turk for Transcription of Non-Native Speech
Keelan Evanini, Derrick Higgins and Klaus Zechner

Exploring Normalization Techniques for Human Judgments of Machine Translation Adequacy Collected Using Amazon Mechanical Turk
Michael Denkowski and Alon Lavie

Can Crowds Build parallel corpora for Machine Translation Systems?
Vamshi Ambati and Stephan Vogel

Turker-Assisted Paraphrasing for English-Arabic Machine Translation

Michael Denkowski, Hassan Al-Haj and Alon Lavie

Annotating Large Email Datasets for Named Entity Recognition with Mechanical Turk

Nolan Lawson, Kevin Eustice, Mike Perkowitz and Meliha Yetisgen-Yildiz

Annotating Named Entities in Twitter Data with Crowdsourcing

Tim Finin, William Murnane, Anand Karandikar, Nicholas Keller, Justin Martineau and Mark Dredze

MTurk Crowdsourcing: A Viable Method for Rapid Discovery of Arabic Nicknames?

Chiara Higgins, Elizabeth McGrath and Laila Moretto

An Enriched MT Grammar for Under $100

Omar Zaidan and Juri Ganitkevitch

12:30pm    Lunch

1:30pm    Afternoon Presentation Session 1
Using the Amazon Mechanical Turk to Transcribe and Annotate Meeting Speech for Extractive Summarization
Matthew Marge, Satanjeev Banerjee and Alexander Rudnicky

Using Mechanical Turk to Annotate Lexicons for Less Commonly Used Languages
Ann Irvine and Alexandre Klementiev

Opinion Mining of Spanish Customer Comments with Non-Expert Annotations on Me
chanical Turk
Bart Mellebeek, Francesc Benavent, Jens Grivolla, Joan Codina, Marta R. Costa-Jus and Rafael Banchs

Crowdsourcing and language studies: the new generation of linguistic data

Robert Munro, Steven Bethard, Victor Kuperman, Vicky Tzuyin Lai, Robin Melnick, Christopher Potts, Tyler Schnoebelen and Harry Tily

Not-So-Latent Dirichlet Allocation: Collapsed Gibbs Sampling Using Human Judgments
Jonathan Chang

3:10pm    Break

3:30pm    Afternoon Presentation Session 2
Collecting Image Annotations Using Amazon’s Mechanical Turk
Cyrus Rashtchian, Peter Young, Micah Hodosh and Julia Hockenmaier

Non-Expert Evaluation of Summarization Systems is Risky
Dan Gillick and Yang Liu

Shedding (a Thousand Points of) Light on Biased Language

Tae Yano, Philip Resnik and Noah A. Smith

4:30pm    Afternoon Poster Session
Note: afternoon talks will also be presented as posters in this session.

Evaluation of Commonsense Knowledge with Mechanical Turk
Jonathan Gordon, Benjamin Van Durme and Lenhart Schubert

Cheap Facts and Counter-Facts

Rui Wang and Chris Callison-Burch

The Wisdom of the Crowds Ear: Speech Accent Rating and Annotation with Amazon Me
chanical Turk
Stephen Kunath and Steven Weinberger

Crowdsourcing Document Relevance Assessment with Mechanical Turk

Catherine Grady and Matthew Lease

Preliminary Experiments with Amazon’s Mechanical Turk for Annotating Medical Named
Meliha Yetisgen-Yildiz, Imre Solti, Fei Xia and Scott Halgrim

Tools for Collecting Speech Corpora via Mechanical-Turk

Ian Lane, Matthias Eck, Kay Rottmann and Alex Waibel

Measuring Transitivity Using Untrained Annotators

Nitin Madnani, Jordan Boyd-Graber and Philip Resnik

Amazon Mechanical Turk for Subjectivity Word Sense Disambiguation
Cem Akkaya, Alexander Conrad, Janyce Wiebe and Rada Mihalcea

Non-Expert Correction of Automatically Generated Relation Annotations

Matthew R. Gormley, Adam Gerber, Mary Harper and Mark Dredze

Using Mechanical Turk to Build Machine Translation Evaluation Sets

Michael Bloodgood and Chris Callison-Burch

Creating a Bi-lingual Entailment Corpus through Translations with Mechanical Turk:
$100 for a 10-day Rush
Matteo Negri and Yashar Mehdad

Error Driven Paraphrase Annotation using Mechanical Turk

Olivia Buzek, Philip Resnik and Ben Bederson