Skip to content

Conversation

@burtenshaw
Copy link
Collaborator

I've been trying out the library and implemented this simple script to test models on the kuhn_poker task in open spiel.

It tests two models on the task, using the openai inference client, so it's provider agnostic.I used it with HF inference providers, but you could vLLM, openai, etc.

The example script follows the style of the others and prints out a report:

============================================================
COMPARATIVE RESULTS
============================================================

Rank   Model                                    Avg $/game   Win Rate   Record         
------ ---------------------------------------- ------------ ---------- ---------------
🥇 1   DeepSeek-V3.1-Terminus                   +0.020         55.0%     55W-45L-0P     
🥈 2   Qwen2.5-72B-Instruct                     -0.200         40.0%     40W-60L-0P     

🏆 Best performer: DeepSeek-V3.1-Terminus
   Chips/game: +0.020
   Win rate: 55.0% (55/100)

📊 Strategy Analysis:
   DeepSeek-V3.1-Terminus           5.0% PASS /  95.0% BET
   Qwen2.5-72B-Instruct            50.5% PASS /  49.5% BET

============================================================

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 22, 2025
base_url=API_BASE_URL,
api_key=API_KEY,
)
env = OpenSpielEnv(base_url=OPENSPIEL_SERVER_URL)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you want to use from_docker_image("ghcr...") just so the person playing with this example does not have to bring this up and this example works out of the box?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense.

Copy link
Collaborator Author

@burtenshaw burtenshaw Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made the change, but I couldn't test on ghcr.io/meta-pytorch/openspiel-env:latest because it's private. I tested on a local build of openspiel-env though. Tested now.

I also opened this from the process

@init27
Copy link
Contributor

init27 commented Oct 23, 2025

@burtenshaw I just came to say you're a legend, thanks for so many fast contributions!!!!

@burtenshaw
Copy link
Collaborator Author

@burtenshaw I just came to say you're a legend, thanks for so many fast contributions!!!!

No worries @init27 . I'm having so much fun with openenv.

@jspisak jspisak added the enhancement New feature or request label Oct 27, 2025
@pankit-eng pankit-eng merged commit 06bfcee into meta-pytorch:main Oct 27, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot. enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants