CLI format run benchopt

[tomMoral]

A first POC to run this benchmark with benchopt.

The results looks like these after running benchopt run benchopt_benchmark/ -n 100 -s SGD[max_epochs=20,learning_rate=[2e-3,1e-3]] -d MNIST[seed=42,activation=ReLU] -r 2

(note that you probably need to install the dev version of benchopt as pip install git+https://github.com/benchopt/benchopt to have the plot running.)

There are several issues with this implem:

• There are no checkpoints yet.
• The test loss corresponds to the last and not the best
• The train loss is incorrect as I did not know how to access it from the Trainer.
• There are several things that I changed directly in FOB to avoid having to mess too much with the config file. We could do either depending on what you thing make more sense.

But that is already a good basis to discuss if this makes sense and if we should continue further :)
Happy to chat IRL in ICML if you are going!