Skip to content

Don't pass test file contents that fail to fetch to be used in the coverage evaluation prompts. #5833

@DanielRyanSmith

Description

@DanielRyanSmith

Currently, the prompts contain some guardrail text to ignore bad data, but now that we're fetching both the test file contents and their dependency file contents, we can simply remove the text about failures from the prompts and not generate the prompts with the error text.

Optionally, it may be useful to add a caveat to the final report that "Failed to fetch the contents of X files", along with listing their file path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    WPT-AIRelating to AI-powered WPT coverage evaluation reports.bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions