Overview

In continuous black-box optimization, there are a number of benchmark problem sets and competitions. However, the focus has mainly been on the performance and comparison of algorithms on artificial problems. The aim of this workshop is to instead make a class of real-world optimization problems the center of focus, bringing together researchers to discuss and develop deeper insights into the structure and difficulty of the problems, as well as experimental methodology (including algorithms). We focus on optimization problems found in machine learning (ML) and data analysis (DA) because:

• They are important in many real-world applications of machine learning
• They have several properties that are convenient for benchmarking and landscape analysis
• They are closely related to the methodologies and knowledge of black-box optimization researchers (i.e. not strongly reliant on domain expert knowledge from a very different discipline

Several problem classes (and specific problem instances) from the area of machine learning and data analysis will be proposed in advance of the workshop submission deadline. Participants will be invited to submit a brief paper that shows new insights into the problems, for example via exploratory landscape analysis, algorithm performance (with a focus on "why") or analysis of the quality/diversity of solutions present in the problem instances. Contributions around open source code, data or other shared resources for the research community would be particularly welcome.

Topics of Interest

Interested participants are invited to make submissions to the workshop that are broadly aligned with its theme. We encourage a variety of submission types including work-in-progress, position statements, new discussion/analysis of previous work and proposals for new benchmark problems. Specific topics include (but are not limited to):

• Exploratory Landscape Analysis - calculation, analysis and comparison of problem fitness landscapes.
• Techniques for improved benchmarking, evaluation and comparison of black-box optimization algorithms.
• Evaluating optimization algorithms on ML and DA optimization problems, with a focus on insight into the problem and assessment of the solutions found.
• Proposals for new benchmark problems, software frameworks, datasets or other resources to support problem analysis and algorithm benchmarking.
• Evaluation of algorithm selection and configuration techniques.
• Meta-analyses of reported experimental results and methodologies from the literature.

Problem Set

Submission of relevance to the workshop are welcome. However, as a focus for the workshop, we have suggested a set of specific problem instances. Researchers are encouraged to use these problem instances as part of their submissions to the workshop.

• MLDA Problem Set v1.0. Refer to the document for links to some code and datasets.
• Google drive share for the workshop.

Submission

Submissions should take the form of extended abstracts of up to 2 pages long, preferably formatted according to the plain article LaTeX style. Abstracts should be submitted as a PDF attachment to marcusg@uq.edu.au

Important Dates and Location

Abstract submission deadline: 28th June, 2018

Notification of acceptance: 2nd July, 2018

The workshop will be held as part of the 15th International Conference on Parallel Problem Solving from Nature (PPSN 2018), Coimbra, Portugal. It will be a half-day workshop, to be scheduled on either the 8th or 9th of September, 2018. PPSN runs from 8-12th September, 2018.

Organizers

Marcus Gallagher, University of Queensland, Australia

Mike Preuss, University of Munster, Germany

Pascal Kerschke, University of Munster, Germany

Technical Committee

Mario Andres Munoz Acosta, University of Melbourne, Australia

Aldeida Aleti, Monash University, Australia

Hans-Georg Beyer, Vorarlberg University of Applied Sciences , Austria

Peter A. N. Bosman, Centrum Wiskunde & Informatica (CWI), The Netherlands

Jose Lozano, University of the Basque Country, Spain

Katherine Malan, University of South Africa, South Africa

Gabriela Ochoa, University of Stirling, Scotland

Sebastian Risi, IT University of Copenhagen, Denmark

Marc Schoenauer, INRIA, France

Thomas Stuetzle, Universite Libre de Bruxelles, IRIDIA, Belgium

Markus Wagner, University of Adelaide, Australia