|
| 1 | +# Braintrust SDK Agent Guide |
| 2 | + |
| 3 | +Guide for contributing to the Braintrust Python SDK repository. |
| 4 | + |
| 5 | +## Defaults |
| 6 | + |
| 7 | +- Use `mise` as the source of truth for tools and environment. |
| 8 | +- Prefer `py/` commands over root `make` targets when working on the SDK itself. |
| 9 | +- Keep changes narrow and run the smallest relevant test session first. |
| 10 | +- Do not rely on optional provider packages being installed unless the active nox session installs them. |
| 11 | + |
| 12 | +## Repo Map |
| 13 | + |
| 14 | +- `py/`: main Python package, tests, examples, nox sessions, release build |
| 15 | +- `integrations/`: separate integration packages |
| 16 | +- `internal/golden/`: compatibility and golden projects |
| 17 | +- `docs/`: supporting docs |
| 18 | + |
| 19 | +Important code areas in `py/src/braintrust/`: |
| 20 | + |
| 21 | +- core SDK modules: top-level package files |
| 22 | +- wrappers/integrations: `wrappers/` |
| 23 | +- temporal: `contrib/temporal/` |
| 24 | +- CLI/devserver: `cli/`, `devserver/` |
| 25 | +- tests: colocated `test_*.py` |
| 26 | + |
| 27 | +## Setup |
| 28 | + |
| 29 | +Preferred repo bootstrap: |
| 30 | + |
| 31 | +```bash |
| 32 | +mise install |
| 33 | +make develop |
| 34 | +``` |
| 35 | + |
| 36 | +Package-focused setup: |
| 37 | + |
| 38 | +```bash |
| 39 | +cd py |
| 40 | +make install-dev |
| 41 | +``` |
| 42 | + |
| 43 | +Install optional provider dependencies only if needed: |
| 44 | + |
| 45 | +```bash |
| 46 | +cd py |
| 47 | +make install-optional |
| 48 | +``` |
| 49 | + |
| 50 | +## Commands |
| 51 | + |
| 52 | +Preferred SDK workflow: |
| 53 | + |
| 54 | +```bash |
| 55 | +cd py |
| 56 | +make lint |
| 57 | +make test-core |
| 58 | +nox -l |
| 59 | +``` |
| 60 | + |
| 61 | +Targeted wrapper/session runs: |
| 62 | + |
| 63 | +```bash |
| 64 | +cd py |
| 65 | +nox -s "test_openai(latest)" |
| 66 | +nox -s "test_openai(latest)" -- -k "test_chat_metrics" |
| 67 | +``` |
| 68 | + |
| 69 | +Root `Makefile` exists as a convenience wrapper. The authoritative SDK workflow is in `py/Makefile` and `py/noxfile.py`. |
| 70 | + |
| 71 | +## Tests |
| 72 | + |
| 73 | +`py/noxfile.py` is the source of truth for compatibility coverage. |
| 74 | + |
| 75 | +Key facts: |
| 76 | + |
| 77 | +- `test_core` runs without optional vendor packages. |
| 78 | +- wrapper coverage is split across dedicated nox sessions by provider/version. |
| 79 | +- `pylint` installs the broad dependency surface before checking files. |
| 80 | +- `test-wheel` is a wheel sanity check and requires a built wheel first. |
| 81 | + |
| 82 | +When changing behavior, run the narrowest affected session first, then expand only if needed. |
| 83 | + |
| 84 | +## VCR |
| 85 | + |
| 86 | +VCR cassette directories: |
| 87 | + |
| 88 | +- `py/src/braintrust/cassettes/` |
| 89 | +- `py/src/braintrust/wrappers/cassettes/` |
| 90 | +- `py/src/braintrust/devserver/cassettes/` |
| 91 | + |
| 92 | +Behavior from `py/src/braintrust/conftest.py`: |
| 93 | + |
| 94 | +- local default: `record_mode="once"` |
| 95 | +- CI default: `record_mode="none"` |
| 96 | +- wheel-mode skips VCR-marked tests |
| 97 | +- test fixtures inject dummy API keys and reset global state |
| 98 | + |
| 99 | +Common commands: |
| 100 | + |
| 101 | +```bash |
| 102 | +cd py |
| 103 | +nox -s "test_openai(latest)" |
| 104 | +nox -s "test_openai(latest)" -- --disable-vcr |
| 105 | +nox -s "test_openai(latest)" -- --vcr-record=all -k "test_openai_chat_metrics" |
| 106 | +``` |
| 107 | + |
| 108 | +Only re-record cassettes when the behavior change is intentional. If in doubt, ask the user. |
| 109 | + |
| 110 | +## Build Notes |
| 111 | + |
| 112 | +Build from `py/`: |
| 113 | + |
| 114 | +```bash |
| 115 | +cd py |
| 116 | +make build |
| 117 | +``` |
| 118 | + |
| 119 | +Important caveat: |
| 120 | + |
| 121 | +- `py/scripts/template-version.sh` rewrites `py/src/braintrust/version.py` during build. |
| 122 | +- `py/Makefile` restores that file afterward with `git checkout`. |
| 123 | + |
| 124 | +Avoid editing `py/src/braintrust/version.py` while also running build commands. |
| 125 | + |
| 126 | +## Editing Guidance |
| 127 | + |
| 128 | +- Keep tests near the code they cover. |
| 129 | +- Reuse existing fixtures and cassette patterns. |
| 130 | +- If a change affects examples or integrations, update the nearest example or focused test. |
| 131 | +- For CLI/devserver changes, consider whether wheel-mode behavior also needs coverage. |
0 commit comments