backflipping expert hyperparameters

[Howuhh]

Hi, I am interested in training a model on backflip halfcheetah for my work. I have tried training PPO, however I have not been able to achieve results as on gifs, usually the agent just jumps backwards, no flips.

The paper says that SAC was trained. Do you still have the hyperparameters for the expert with which the trajectories were collected? I would be happy if they were made public (or even dataset).

Thanks!

[frt03]

@Howuhh
Thank you for your interest! I trained backflip-cheetah with SAC:

steps: 3000000  # Number of env steps.
actor:
  lr: 3e-4
  nn_size: 256  # Sizes of hidden layers are [nn_size, nn_size]
agent:
  alpha_lr: 3e-4  # Learning rate of alpha.
  batch_size: 256  # Size of each mini-batch.
  gamma: 0.99  # discount factor gamma
critic:
  lr: 3e-4
  nn_size: 256
replay_buffer:
  start_size: 10000
  size: 1000000

I used my own torch codebase, but probably any implementation would work. While I feel positive to release the backflip-cheetah dataset, it is undecided for now.

[Howuhh]

Thanks! All arguments in BackflipCheetahEnv should be default? forward_reward_weight=1.0 for example