Schedule

2018 Challenges in Machine Learning workshop@NIPS:

Machine Learning Challenges "in the wild"

Saturday December 8, 2018

NeurIPS 2018 workshops. Palais des congres de Montreal, Canada

[Home][Schedule][Invited Speakers][Committee][Organizers][NIPS workshop book]

Morning session (8:00 am-12:00 pm): New platforms and innovative designs

8:00 am (10 min) - Morning welcome and introduction

8:10 (30 min) - Esteban Arcaute and Umut Ozertem - Facebook project on developing benchmarks of algorithms in realistic settings

8:40 (30 min) - Laura Seaman - Project Alloy – Machine Learning Challenges for Researching Human-Machine Teaming (slides)

9:10 (90 min) - Live competition: Pommerman (slides)

10:40 - 11:00 (20 min) Break, poster viewing

11:00 (30 min) - Julien Hay, Bich-Liên Doan, Fabrice Popineau - Renewal news recommendation platform (slides)

11:30 - 12:00 (30 min) Panel discussion. Design principles and implementation issues. (3 panelists TBA + 1 moderator)

Lunch break (12:00 pm-1:00pm)

Lunch served in the room for participants.

Discussions and poster viewing.

Afternoon session (1:00 pm-6:30 pm): Robotics, RL, and interactive challenges

1:00 - Afternoon welcome and announcements (organizers)

1:10 (30 min) - Mikhail Burtsev and Varvara Logacheva - Wild evaluation of chat-bots (slides)

1:40 (30 min) - Larry Jackel - Measuring Progress in Robotics

2:10 (30 min) - Daniel Polani - Competitions to Challenge Artificial Intelligence: from the L-Game to RoboCup

2:40 (30 min) - Antoine Marot - Learning to run a power network

3:10 - 3:30 (20 min) Break, poster viewing

3:30 (90 min) - Live competition: The driving AI Olympics (slides)

5:00 - 5:30 (30 min) - Panel discussion: Opportunities to organize new impactful challenges. (3 panelists TBA + 1 moderator)

5:30 - Adjourn.

6:30 - Closing banquet.

Posters

Posters dimensions are: 36 x 48 in. (91cm x 122cm); masking tape will be provided. All posters can be presented at all poster sessions.

New platforms and innovative designs:

Beyond the Leaderboard, Adriënne M. Mendrik, Stephen R. Aylward

Grand-Challenge.org, James A. Meakin Bram van Ginneken

Corpus for AutoML Pipelines, Richard Lippmann, Swaroop Vattam, Pooya Khorrami, and Cagri Dagli

How to fail hosting data science contests with images, Evgeny Nizhibitsky, Moscow State Univ. Artur Kuzin, Dbrain, Moscow Institute of Physics and Technologies, Russia

ML Benchmark Tools Package, Ryan Turner, Uber AI Labs

New impactful challenges:

NASA Frontier Development Lab 2018, A. Bell, A. Chopra, W. Fawcett, R. Talebi, D. Angerhausen, A. Berea, N.A. Cabrol, C. Kempes, M. Mascaro

AutoDL challenge design and beta tests, Zhengying Liu, Olivier Bousquet, Andre Elisseeff, Isabelle Guyon, Adrien Pavao, Lisheng Sun-Hosoya, and Sebastien Treguer

TrackML, a Particle Physics Tracking Machine Learning Challenge, Jean-Roch Vlimant (Caltech), Vincenzo Innocente, Andreas Salzburger (CERN), Isabelle Guyon (ChaLearn), Sabrina Amrouche, Tobias Golling, Moritz Kiehn (Geneva University),David Rousseau∗, Yetkin Yilmaz (LAL-Orsay), Paolo Calafiura, Steven Farrell, Heather Gray (LBNL), Vladimir Vava Gligorov (LPNHE-Paris), C ́ecile Germain, Victor Estrade(LRI-Orsay),Edward Moyse (University of Massachussets), Mikhail Hushchyn, Andrey Ustyuzhanin (Yandex, HSE)

L2RPN: Learning To Run a Power Network Competition, Antoine Marot, Balthazar Donon, Isabelle Guyon, Benjamin Donnot.

Multi-Agent RL in MalmÖ(MARLÖ) Competition, Diego Perez-Liebana Katja Hofmann Sharada Prasanna Mohanty Noburu Kuno Andre Kramer Sam Devlin Raluca D. Gaina Daniel Ionita

Live competitions:

Pommerman

Competition summary: Train a team of communicative agents to play Bomberman. Compete against other teams.

Organizers:

Cinjon Resnick, NYU, cinjon@nyu.edu

David Ha, Google Brain, hadavid@google.com

Denny Britz, Prediction Machines, dennybritz@gmail.com

Jakob Foerster, Oxford, jakobfoerster@gmail.com

Jason Weston, Facebook FAIR, jase@fb.com

Joan Bruna, NYU, bruna@cims.nyu.edu

Julian Togelius, NYU, julian@togelius.com

Kyunghyun Cho, NYU, kyunghyun.cho@nyu.edu

Webpage: https://www.pommerman.com/

The AI diving Olympics

Competition summary: Machine Learning (ML), deep learning, and deep reinforcement learning have shown remarkable success on a variety of tasks in the very recent past. However, the ability of these methods to supersede classical approaches on physically embodied agents is still unclear. In particular, it remains to be seen whether learning-based approached can be completely trusted to control safety-critical systems such as self-driving cars. This live competition, presented by the Duckietown Foundation, is designed to explore which approaches work best for what tasks and subtasks in a complex robotic system. The participants will need to design algorithms that implement either part or all of the management and navigation required for a fleet of self-driving miniature taxis. There will be a set of different trials that correspond to progressively more sophisticated behaviors for the cars. These vary in complexity, from the reactive task of lane following to more complex and “cognitive” behaviors, such as obstacle avoidance, point-to-point navigation, and finally coordinating a vehicle fleet while adhering to the entire set of the “rules of the road”. We will provide baseline solutions for the tasks based on conventional autonomy architectures; the participants will be free to replace any or all of the components with custom learning-based solutions.The competition will be live at NIPS, but participants will not need to be physically present—they will just need to send their source code packaged as a Docker image. There will be qualifying rounds in simulation and we will make available the use of “robotariums,” which are facilities that allow remote experimentation in a reproducible setting.

Organizers:

Andrea Censi, nuTonomy and ETH Zürich, acensi@idsc.mavt.ethz.ch

Liam Paull, Université de Montréal, paulll@iro.umontreal.ca

Jacopo Tani, ETH Zürich, tanij@ethz.ch

Scott Livingston, q@rerobots.net

Julian Zilly, ETH Zürich, jzilly@ethz.ch

Ruslan Hristov, nuTonomy, rusi@nutonomy.com

Oscar Beijbom, nuTonomy, oscar@nutonomy.com

Eryk Nice, nuTonomy, eryk.nice@nutonomy.com

Sunil Mallya, Amazon, smallya@amazon.com

Justin De Castri, Amazon, decastri@amazon.com

Hsueh-Cheng (Nick) Wang, National Chiao Tung University, hchengwang@gmail.com