CLIN33 Shared Task

 Introduction

Welcome to the CLIN33 shared task on the Automatic Detection of AI-Generated Texts! 

With the surge of powerful automatic content generation comes the important task of distinguishing  texts generated by language models from texts written by humans. Therefore, the CLIN33 shared task invites you to contribute to the development of automatic detection systems of AI-generated texts. The shared task consists of two tracks.

Detection Track. In this track you will develop a system that detects whether a document is generated by a language model or written by a human. The system will be evaluated on a test dataset that will remain hidden. For tuning your system you will be provided with a sample of the test data as a development set.

• Your system will be submitted as a notebook (colab or jupyter) and be self-contained

• The test data will not be shared

• You will be provided with a subset of the test data for validation and development

• No training data will be provided, you are expected to create your own when using Machine Learning

• There will be a scoreboard for testing your model on the validation data that you can use to monitor progress during development

• There will be three genres of text in the development and test: newspaper text, tweets, and reviews generated with two different language models

• In addition the test data will contain two additional genres (poetry and a mystery genre) and text generated with an additional open source model

• The generated text will be produced using different prompt engineering strategies 

• There will be an English and a Dutch subtrack. Participants are expected to participate in both.

There will be a prize for the team that develops the best performing system averaged over both languages and all conditions.

Explanation track (optional). In this track, you will provide a qualitative analysis, empirically supported by the output of your system on the validation set, of the knowledge used by your model to solve the task. This explanation can be in terms of statistical properties or in terms of linguistic patterns and phenomena and will the form of a short paper (4 pages max). This track will be evaluated by the organizers, extended with additional judges representing the users of detection systems (for example teachers or editors).  The most insightful submission will be awarded a special prize.

To participate, please fill in the form.  This form is no longer active. Contact walter.daelemans@uantwerpen.be for questions.