The Longest6 benchmark consists of 36 routes with an average route length of 1.5km, which is similar to the average route length of the official leaderboard (~1.7km). During evaluation, we ensure a high density of dynamic agents by spawning vehicles at every possible spawn point permitted by the CARLA simulator. Following the NEAT evaluation benchmark, each route has a unique environmental condition obtained by combining one of 6 weather conditions (Cloudy, Wet, MidRain, WetCloudy, HardRain, SoftRain) with one of 6 daylight conditions (Night, Twilight, Dawn, Morning, Noon, Sunset).
To evaluate an agent on Longest6, please use the following environment variables in leaderboard/scripts/local_evaluation.sh:
export SCENARIOS=${WORK_DIR}/leaderboard/data/longest6/eval_scenarios.json
export ROUTES=${WORK_DIR}/leaderboard/data/longest6/longest6.xmlWe additionally provide a separate xml file for each of the 36 routes in the benchmark in the folder longest6_split. To get the driving score of a single route, you can use the environment variables:
export SCENARIOS=${WORK_DIR}/leaderboard/data/longest6/eval_scenarios.json
export ROUTES=${WORK_DIR}/leaderboard/data/longest6/longest6_split/longest_weathers_#.xml |
 |
 |
 |
 |
 |
|---|---|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |