Questions about optimal/sub-optimal/recovery trajectories and `optimal_steps` mismatch

[Mars-xyx]

Hi, thanks for releasing ColorBench.

I have two questions:

1. In the paper, ColorBench is described as covering optimal, sub-optimal, and recovery paths.

   • Did you release any official trajectory annotations for these (e.g., per-task optimal trajectory / sub-optimal trajectory / recovery trajectory)?
   • Or are users expected to reconstruct them from graph.json and milestones?

2. I reconstructed task-conditioned trajectories from data/graph.json + data/tasks.json, and found that for some tasks, the shortest feasible path is mismatch with optimal_steps in tasks.json.

   • For example, task_id 127 can reach the milestone-defined goal in about 5 steps, while optimal_steps is larger.
   • Is this expected? If yes, what is the intended meaning of optimal_steps (strict ground-truth optimum vs approximate reference length)?

[norsheep]

We have updated the ground-truth optimal_steps based on manual statistics. Sorry for the earlier incorrect values—they may have affected your judgment.

Due to the dynamic nature of page transitions in the benchmark (e.g., occasional pop-ups such as ads), we have not provided a fixed optimal trajectory at this stage. You may instead refer to the graph.json and milestones to guide your evaluation. If needed, we can consider providing a reference optimal path in the future.