Skip to content

feat(tools): include structured output in exported .prompt files#5506

Open
chrisraygill wants to merge 3 commits into
mainfrom
feat/export-output-schema
Open

feat(tools): include structured output in exported .prompt files#5506
chrisraygill wants to merge 3 commits into
mainfrom
feat/export-output-schema

Conversation

@chrisraygill

Copy link
Copy Markdown
Contributor

Summary

This is the server-side counterpart to the editable output config added to the model runner in genkit-ai/genkit-ui#1919. That PR lets you set structured output in the Dev UI model runner; this PR makes the Export to .prompt action actually include it.

Linked PR: https://github.com/genkit-ai/genkit-ui/pull/1919

Previously createPrompt dropped the generation's output config entirely, so a flow that used structured output lost its schema when exported. The exported front matter now includes the output, with the JSON Schema written as Picoschema.

Before

---
model: googleai/gemini-flash-latest
config:
  temperature: 0.7
---

After

---
model: googleai/gemini-flash-latest
config:
  temperature: 0.7
output:
  format: json
  schema:
    title: string
    servings: integer, number of servings
    ingredients(array):
      name: string
      onSale: boolean
    steps(array): string
---

What changed

  • CreatePromptRequestSchema now accepts output (it was being stripped by tRPC input validation before reaching the frontmatter).
  • createPrompt maps the output config onto the frontmatter, reading the schema from jsonSchema (generate action shape) or schema (model request shape), and mapping the JSON-producing formats (json, jsonl, array, enum) onto json.
  • jsonSchemaToPicoschema converts an object JSON Schema into the compact Picoschema form: required/optional (?), descriptions, enums, scalar and object arrays, nested objects, and additionalProperties wildcards. Non-object top-level schemas (a bare array or scalar) pass through as raw JSON Schema, which Dotprompt also accepts.

Testing

  • Unit tests in genkit-tools/common/tests/utils/prompt_test.ts cover the converter and the frontmatter mapping (15 cases).
  • Verified end to end against a running Dev UI: exporting a structured-output prompt from the model runner now produces the Picoschema output block above.

Notes

  • The frontmatter format field is limited to json/text/media, so jsonl/array/enum export as format: json with the schema preserved. Exact format fidelity could be a follow-up.

Checklist

The Dev UI prompt export dropped the generation's output config, so a flow that
used structured output lost its schema when exported. createPrompt now carries
the output config through and writes it to the frontmatter, converting the JSON
Schema to Picoschema.

- Accept output in CreatePromptRequestSchema.
- Map the output config onto the frontmatter, reading the schema from jsonSchema
  (generate action) or schema (model request), and mapping the JSON-producing
  formats onto json.
- Add jsonSchemaToPicoschema to convert an object JSON Schema into the compact
  Picoschema form (required/optional, descriptions, enums, scalar and object
  arrays, nested objects, additionalProperties wildcards). Non-object top-level
  schemas pass through as raw JSON Schema, which Dotprompt also accepts.

Server-side counterpart to the editable output config in the model runner:
genkit-ai/genkit-ui#1919

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for mapping a generate request's output configuration onto .prompt frontmatter, converting JSON Schema to Picoschema. This includes schema updates, router integration, and helper functions to handle the conversion. Feedback on the changes suggests making the scalarType helper function more robust by safely handling null or undefined schemas to prevent potential runtime crashes.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread genkit-tools/common/src/utils/prompt.ts Outdated
Safely access the type in scalarType so a null or malformed property schema in
`properties` falls back to `any` instead of throwing. Adds a regression test.

Addresses review feedback on the picoschema converter.
@chrisraygill chrisraygill marked this pull request as ready for review June 10, 2026 04:31

@MichaelDoyle MichaelDoyle left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really cool.

Is the conversion code useful for other applications, besides dev ui generating dotprompts? I wonder if that code should live somewhere else. e.g. in the dotprompt library, and we just depend on it here.

Comment thread genkit-tools/common/tests/utils/prompt_test.ts Outdated
Comment thread genkit-tools/common/tests/utils/prompt_test.ts Outdated
Comment thread genkit-tools/common/tests/utils/prompt_test.ts Outdated
/**
* Maps a generate request's output config onto `.prompt` frontmatter, converting
* the JSON Schema to Picoschema. The frontmatter format is limited to
* json/text/media, so the JSON-producing formats (json, jsonl, array, enum) map

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is interesting -- so there is a mismatch between what can be done in code, and what dotprompt supports?

If so, whenever we do such a conversion, let's leave a comment in the produced dotprompt that explains.
e.g.) # converted from <type>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSON Schema to Picoschema conversion is lossy... Picoschema has a very limited subset of features.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, how do we think output format set to jsonl should work when converted to picoschema? What if the JSON schema includes unsupported features in picoschema?

Address review feedback: break the combined encode test into per-feature
tests (enum, array of scalars, array of objects, nested object), split
the text/media format test in two, and rename the null-property test to
describe what it returns.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants