Call for Abstracts

Please note that the submission deadline has passed.

We welcome 2-page extended abstracts on topics relating to challenges in machine learning. Selected papers will be presented primarily as posters, but exceptional contributions will be given oral presentations. Abstract should be submitted by September 16th, 2019 by sending email to ciml2019@chalearn.org. [You can use the NeurIPS template for your submissions; submission need NOT be anonymized; and extra page can be used for references and acknowledgements]. The best contributions will be invited to contribute a book chapter in the Springer series on challenges in Machine Learning.

Topics of interest include, but are not limited to:

Methods:

  • Novel or atypical challenge protocols, particularly relating to research.

  • Novel or atypical challenge protocols to tackle complex tasks with very large datasets, multi-modal data, and data streams.

  • Methods and metrics of entry evaluation, quantitative and qualitative challenges.

  • Methods of data collection, "ground-truthing", and preparation including bifurcation/anonymization, data generating models.

  • Teaching challenge organization.

  • Hackatons and on-site challenges.

  • Challenge indexing and retrieval, challenge recommenders.

Theory:

  • Experimental design, size data set, data split, error bounds, statistical significance, violation of typical assumptions (e.g. i.i.d. data).

  • Game theory applied to the analysis of challenge participation, competition and collaboration among participants.

  • Diagnosis of data sanity, artifacts in data, data leakage.

Implementation:

  • Re-usable challenge platforms, innovative software environments.

  • Linking data and software repositories to challenges.

  • Security/privacy, intellectual property, licenses.

  • Cheating prevention and remedies (i.e. leaderboard climbing).

  • Issues raised by requiring code submission.

  • Challenges requiring user interaction with the platform (active learning, reinforcement learning).

  • Dissemination, fact sheets, proceedings, crowdsourced papers, indexing post-challenge publications.

  • Long term impact, on-going benchmarks, metrics of impact.

  • Participant rewards, stimulation of participation, advertising, sponsors.

  • Profiling participants, improving participant professional and social benefits.

Applications:

  • Challenges for the benefit of society, as a scientific research tool, for up-skilling, or to solve industry problems.

  • Where to venture next: opportunities for challenge organizers to organize challenges in new domains with high societal impact.

  • Successful challenge leading to significant breakthrough or improvement over the state-of-the-art or unexpected interesting results.

  • Rigorous study of the impact of challenges, analyzing topics and tasks lending themselves to high impact machine learning challenges.

  • Challenges organized or supported by Government agencies, funding opportunities.

Important Dates

  • Abstract submission deadline: September 16th, 2019 (submission closed).

  • Acceptance decisions: October 1st, 2019.

  • Finalized program on website: October 7th, 2019.