From the official forward code, they normalize the input speed to one.
https://github.com/carla-simulator/imitation-learning/blob/master/agents/imitation/imitation_learning.py#L141
And from the released dataset, it seemed that the speed is raw km/h (from 0 to 90)?
Have you considered this normalization?