Skip to content

Issues: UKGovernmentBEIS/inspect_ai

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

45 Open 251 Closed
45 Open 251 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Custom Displays
#1267 opened Feb 7, 2025 by Baidicoot
Run cleanup and finalizers
#1265 opened Feb 7, 2025 by sjawhar
Score events recorded in the transcript
#1264 opened Feb 7, 2025 by sjawhar
Pause events recorded in the transcript
#1263 opened Feb 7, 2025 by sjawhar
Unique ID for each run of each sample
#1262 opened Feb 7, 2025 by sjawhar
Create clustered standard error metric
#1251 opened Feb 5, 2025 by alexandraabbas
1
record default values in log file
#1245 opened Feb 5, 2025 by jjallaire-aisi
@jjallaire
add sandbox parameter to bash and python tools
#1237 opened Feb 4, 2025 by jjallaire
Log viewer: scroll bar resets when using the scorer drop-down
#1220 opened Jan 30, 2025 by tadamcz
@dragonstyle
[Feature request] Support for LiteLLM and LiteLLM proxy
#1213 opened Jan 30, 2025 by danrr
Execute --runslow tests in CI
#1212 opened Jan 30, 2025 by jjallaire-aisi
@art-dsit
1
validate that there is no container_name in docker compose files
#1203 opened Jan 28, 2025 by jjallaire
verify that log_dir is writeable before running eval
#1202 opened Jan 28, 2025 by jjallaire
inspect-ai-vscode adds unnecessary entries to .gitignore.
#1194 opened Jan 27, 2025 by tobiasraabe
1
Tutorial: Mathematics dataset not available in HF anymore
#1183 opened Jan 24, 2025 by guiem
Broader notion of retryable exception
#1174 opened Jan 22, 2025 by tadamcz
10
Feature Request: Grouped output metrics & exportable result files for benchmarks
#1172 opened Jan 22, 2025 by owenparsons
@dragonstyle
3
Enable sandboxes to connect to existing Docker or k8s instances
#1153 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
Enforcing cost limits
#1152 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
2
Broader notions of "model" for evals
#1149 opened Jan 19, 2025 by jjallaire
@jjallaire
Bridge to convert from compose.yaml to HELM values
#1148 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
1
Improve task scheduler to not require model as an axis
#1146 opened Jan 19, 2025 by jjallaire
@jjallaire
Direct linking to samples and transcript entries in Inspect View
#1145 opened Jan 19, 2025 by jjallaire
@dragonstyle
Checkpointing and restarting from checkpoints
#1144 opened Jan 19, 2025 by jjallaire
@jjallaire
Log annotation UI
#1143 opened Jan 19, 2025 by jjallaire
@dragonstyle
ProTip! Updated in the last three days: updated:>2025-02-07.