I am reproducing the experiments from your paper "Subtask-Aware Visual Reward Learning from Segmented Demonstrations" using the provided code, and I encountered an issue I would like to consult you about.
Experiment Setup
I followed the two-stage process:
Stage 1 (Reward Model Training)
bash scripts/train_reds_metaworld.sh door-open 2 0 3000 100000 50 50 /root/autodl-tmp/
Stage 2 (Policy Training)
DEVICE_ID=0
TASK_NAME=door-open
SEED=0
BASE_PATH=/root/autodl-tmp
XLA_PYTHON_CLIENT_PREALLOCATE=false LD_PRELOAD=""
CUDA_VISIBLE_DEVICES=${DEVICE_ID} python scripts/train_dreamer.py
--configs=reds_prior_rb metaworld
--reward_model_path=${BASE_PATH}/reds_logdir/REDS/metaworld-${TASK_NAME}/${TASK_NAME}_phase2/s0/last_model.pkl
--logdir=${BASE_PATH}/exp_local/${TASK_NAME}reds_seed${SEED}
--task=metaworld${TASK_NAME}
--env.metaworld.reward_type=sparse
--seed=${SEED}
Observed Issue
In the final evaluation logs (e.g., 20250805T210535_249_e0bd_failure.npz), the success rate is always 0.
However, in the rollout videos, the task appears to be completed (e.g., the door opens), so I suspect it might not be reaching the success threshold defined in the environment.
Task: door-open, reward type: sparse.
Checks Already Performed
Verified that reward_model_path exists and loads correctly.
Ensured camera_keys, window_size, and skip_frame match between Stage 1 and Stage 2.
Used NUM_FAILURE_DEMOS=50 for negative demonstrations (also tried reducing or disabling).
Confirmed that Dreamer evaluation uses the environment’s sparse success criterion.
Question
Could you please advise if there are any recommended methods or parameter settings in your implementation that would allow this task to be recognized as successful during evaluation, and achieve a higher success rate?
I am reproducing the experiments from your paper "Subtask-Aware Visual Reward Learning from Segmented Demonstrations" using the provided code, and I encountered an issue I would like to consult you about.
Experiment Setup
I followed the two-stage process:
Stage 1 (Reward Model Training)
bash scripts/train_reds_metaworld.sh door-open 2 0 3000 100000 50 50 /root/autodl-tmp/
Stage 2 (Policy Training)
DEVICE_ID=0
TASK_NAME=door-open
SEED=0
BASE_PATH=/root/autodl-tmp
XLA_PYTHON_CLIENT_PREALLOCATE=false LD_PRELOAD=""
CUDA_VISIBLE_DEVICES=${DEVICE_ID} python scripts/train_dreamer.py
--configs=reds_prior_rb metaworld
--reward_model_path=${BASE_PATH}/reds_logdir/REDS/metaworld-${TASK_NAME}/${TASK_NAME}_phase2/s0/last_model.pkl
--logdir=${BASE_PATH}/exp_local/${TASK_NAME}reds_seed${SEED}
--task=metaworld${TASK_NAME}
--env.metaworld.reward_type=sparse
--seed=${SEED}
Observed Issue
In the final evaluation logs (e.g., 20250805T210535_249_e0bd_failure.npz), the success rate is always 0.
However, in the rollout videos, the task appears to be completed (e.g., the door opens), so I suspect it might not be reaching the success threshold defined in the environment.
Task: door-open, reward type: sparse.
Checks Already Performed
Verified that reward_model_path exists and loads correctly.
Ensured camera_keys, window_size, and skip_frame match between Stage 1 and Stage 2.
Used NUM_FAILURE_DEMOS=50 for negative demonstrations (also tried reducing or disabling).
Confirmed that Dreamer evaluation uses the environment’s sparse success criterion.
Question
Could you please advise if there are any recommended methods or parameter settings in your implementation that would allow this task to be recognized as successful during evaluation, and achieve a higher success rate?