Autonomous robots are increasingly adopted in goods delivery services to solve issues that affect last-mile delivery, such as traffic congestion, pollution, and high operational costs. In these systems, clients order goods through a mobile application, and a scheduler allocates autonomous robots to serve the different orders. The scheduler is an optimisation algorithm that adopts different heuristics with the aim of maximising the number of delivered goods. Assessing the quality of the scheduler requires to evaluate it under different sets of orders. The state-of-the-practice for these systems consists in randomly generating orders. However, such techniques are not able to expose the deficiencies of the scheduler. To fill this gap, in this paper, we propose a search-based approach that searches for test scenarios defined as set of orders. The goal of the approach is to find scenarios in which the proportion of successfully delivered orders is minimised; at the same time, the approach tries to minimise the total number of orders in the scenario, to guarantee that low performance is due to poor performance of the scheduler and not overload of the system. Finally, the approach tries to generate set of orders that are realistic, i.e., that could occur in the real world. We applied the approach to test the scheduling algorithm developed by Panasonic for the operation of autonomous delivery robots in three regions in Japan, among which the Fujisawa Sustainable Smart Town. We did this using a simulator that allows to assess the system performance under different conditions. Experimental results show that the approach is statistically significantly better at generating challenging test scenarios than a random generation approach. Moreover, results show that the approach is able to generate challenging tests for different loads of the system.
- Thomas Laurent https://laurenttho3.github.io/
- Paolo Arcaini http://group-mmm.org/~arcaini/
- Fuyuki Ishikawa http://research.nii.ac.jp/~f-ishikawa/en/