Repository navigation
feat: Add a docs eval for storage bucket fundamentals - #531
Open
jeremenichelli wants to merge 1 commit into
Open
jeremenichelli wants to merge 1 commit into
jeremenichelli wants to merge 1 commit into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
czenko
approved these changes
Nov 20, 2026
Contributor
There was a problem hiding this comment.
Approving. I checked the prompt against the README's vocabulary list. I also checked the lifecycle-first ordering, the accepted-upload gate on the refusal checks, the server-key read-back control on a link alone does not open one, and the run-scoped markers. They all hold up. Two notes:
the buckets declared in config.toml existpasses when the scorer can't parseconfig.toml, or when the bucket query throws.seedDeclaredBucketsreturnsdeclared: []withseedFailedset, and the empty branch reports a pass. The probes still red, but this check should fail when it couldn't run.- The four refusal checks are named for a whole class but probe one sample each. A 2.5 MB photos limit passes
press-photos: a file over 2 MB is turned away, and a list that also allowstext/plainpassespress-photos: a file that isn't an image is turned away. The README already scopes the boundary and the full type list out, so this only needs new names, such aspress-photos: a 3 MB image is turned away. Please rename them before the first baseline, since results history is keyed on check names.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Feature: a new docs eval,
build-docs-023-storage-bucket-fundamentals.What is the current behavior?
Nothing scores the Storage Buckets guide. Eval 008 covers policies on a different page, and no eval checks bucket size or type limits.
What is the new behavior?
An agent sets up two buckets: press photos that anyone with the link can see, limited to images under 2 MB, and invoices that a link alone can't open, limited to PDFs under 10 MB. The scorer uploads files within and beyond each limit and fetches the public URLs.
Not run against a real stack yet: the probes only ran against a fake Storage. Needs the
run-evals-changedlabel for the baseline.