[Bug]: Reproduce results in swebench

[Hodge931]

Is there an existing issue for the same bug?

• I have checked the existing issues.

Describe the bug and reproduction steps

Is it sufficient to reproduce the swebench results (53.8% in verified set) by following the readme at https://github.com/All-Hands-AI/OpenHands/tree/main/evaluation/benchmarks/swe_bench?

Thanks so much!

OpenHands Installation

Docker command in README

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response

[neubig]

Hi @Hodge931 , yes that should be sufficient, although it might be best to use the exact commit that we used when we generated those results (maybe @xingyaoww can provide that).

[fishmingyu]

There is one bug I just reported, may you check Bug? @neubig

[fishmingyu]

Another bug I just reported at new issue.