BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//University of Liverpool Computer Science Seminar System//v2//EN
BEGIN:VEVENT
DTSTAMP:20260922T022148Z
UID:Seminar-dept-344@lxserverM.csc.liv.ac.uk
ORGANIZER:CN=Lutz Oettershagen:MAILTO:Lutz.Oettershagen@liverpool.ac.uk
DTSTART:20140225T160000
DTEND:20140225T170000
SUMMARY:School Seminar Series
DESCRIPTION:Ulle Endriss: Collective Annotation: From Crowdsourcing to Social Choice\n\nCrowdsourcing is an important tool, e.g., in  computational linguistics and computer vision, to efficiently label large amounts of data using nonexpert annotators. The individual annotations collected then need to be aggregated into a single collective annotation that can serve as a new gold standard. In this talk, I will introduce the framework of collective annotation, in which we view this problem of aggregation as a problem of social choice, similar to the problem of aggregating the preferences of individual voters in an election. I will present both a formal model for collective annotation in which we can express desirable properties of diverse aggregation methods as axioms, and I will report on the empirical performance of several such methods on annotation tasks in computational linguistics using data we collected by means of crowdsourcing. The talk is based on joint work with Raquel Fernandez, Justin Kruger and Ciyang Qing.\n\nhttps://www.csc.liv.ac.uk/research/seminars/abstract.php?id=344
LOCATION:Ashton Lecture Theater
END:VEVENT
END:VCALENDAR