Rules and Guidelines

You can find the datasets and the evaluation code in this GitHub repository: https://github.com/plspeziali/ara25

The three datasets provided have the following structure:

1. graph_dict_osdpm.pickle is a dictionary where:

   • Keys: Node identifiers (e.g., 'n_0', 'n_1', 'n_27992', etc.)

   • Values: Another dictionary representing connections to neighboring nodes in this structure where:

     • Keys: Neighboring node (e.g., 'n_1')

     • Values: A tuple containing the float values (total length, num. of crossings, walk length, bike length)

2. node_dict_osdpm.pickle is a dictionary where:

• Keys: Node identifiers (e.g., 'n_0', 'n_1', 'n_27992', etc.)

• Values: A tuple containing the coordinates of the node in the Dutch RD New coordinates format 

3. pareto_front_osdpm.pickle is a list of 10000 dictionary entries, every entry is a Pareto front with three keys:

• 'source': The node identifier of the source of those routes

• 'destination': The node identifier of the destination of those routes

• 'pareto_set': A list of tuples each one representing a (weights_list, nodes_list) for a specific route connecting the source to the target

In order to ensure a fair comparison between participants we expect all code to be written in Python, with the use of a limited set of standard libraries such as NumPy, Pandas, geoPandas, NetworkX, math, Pickle, and Torch.

We expect the submissions to be open source. Teams will be responsible for their own repository maintenance. We will publish a website at the VUB university domain where we will publish the competition and related code, announce the results, and link to the repositories of all participating teams. 

The code is to be submitted by sharing a Github repository with the organisers before the submission deadline. The organisers will check as soon as possible whether the code runs. If it does not run, the participants may still correct this, if and only if the submission deadline has not been reached. At the submission deadline, the code will be frozen and entered into the competition. The participants may choose to make their repositories public at this point, or wait until after the results are announced. By entering the competition, however, the participants agree to open-sourcing their submissions.  

The competition will have two tracks with two different evaluation criteria. 

Track 1: The budgeted Pareto-front Finding. To efficiently find the full Pareto front, i.e. all non-dominated solutions. This is the standard setting for Multi-objective path planning and can be seen as a possible first subroutine that will lead to complete input for the user interaction for accessible route planning. For shorter routes this may be done in a reasonable time, but we note that for long routes, the Pareto front might be too large, leading to users who have to wait too long before interaction can start. The two main criteria for this track are optimality/coverage and computational efficiency.   

Optimality and Coverage: The found routes should accurately approximate the true Pareto front, covering a diverse range of trade-offs across objectives. 

Computational Efficiency: The route planning method should demonstrate efficiency in terms of runtime and resource utilization. This will be implemented through a computational budget (sufficient for small problems, but possibly challenging for longer routes - we will fine-tune this before the competition). 

Robustness: The route planning method should be adaptable to different problem instances without significant loss of performance. This will be implemented by testing the algorithm on various problem instances.  

Reproducibility: Participants must provide clear documentation to ensure that results can be validated and reproduced by the competition organizers. This will be implemented by a) open-sourcing the code, and b) submitting the code to the organisers who will run the competition. 

Track 2: Maximum Expected Utility. The second criterion is inspired by a clear wish from users and interaction designers at the Municipality of Amsterdam. Specifically, sometimes users do not or cannot spend a lot of time planning a route at their leisure. At such times, they maybe have time to compare three routes at best and will go for the best of those. Luckily we may have a prior belief (expressed as a distribution over utility functions) that forms an educated guess regarding the preferences of the user. In this scenario, it may not be useful to pre-compute the entire Pareto front. Instead, we just need to output the 3 routes that will provide the best option in terms of expected utility, given that a user will 1) draw a utility function from the prior, and 2) subsequently pick the alternative (from the three presented alternatives) that will maximize their utility.  

This track will have a strict time budget (much stricter than track 1) and will be evaluated on the maximum expected utility given the prior. 

According to the competition rankings, we will award certificates to the top 5 teams and invite them to present their work on-site at the AAMAS 2025 conference.