Skip to content

Make System One the base protocol for decision models - #2145

Open
jimador wants to merge 17 commits into
feature/decision-model-providersfrom
feature/systemone-protocol
Open

jimador wants to merge 17 commits into
feature/decision-model-providersfrom
feature/systemone-protocol

Conversation

@jimador

@jimador jimador commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #2139. The diff is against feature/decision-model-providers.

System One is the JSON protocol decision models speak. One request asks several typed questions about one piece of state, and each answer comes back as a probability, a distribution or a confidence. This PR makes it the base for every decision model endpoint. TypeSafe becomes a preset over it, and any other System One endpoint, such as CLEF in a local Ollama, works through the same client.

flowchart LR
    S[DecisionService] --> F[SystemOneModelFactory]
    F --> C[SystemOneCodec<br/>embabel-agent-ai]
    F --> T[SystemOneTransport]
    T --> E[(System One endpoint)]
    TS[TypeSafeModelFactory] -- extends --> F
Loading

Surfaces

Type Module What it does
SystemOneCodec embabel-agent-ai Writes requests and reads responses. Questions go out keyed q1 to qN, so caller question names never leave the process. Reads model listings as {"models": [{"name"}]} or the OpenAI shape {"data": [{"id"}]}.
SystemOneEndpoint embabel-agent-systemone Provider name, base address, timeouts, response size limit and request id header. Rejects addresses that could carry credentials.
SystemOneModelFactory embabel-agent-systemone Builds a DecisionService per model and implements ByokFactory. Reads the key on every call.
TypeSafeModelFactory embabel-agent-typesafe Extends SystemOneModelFactory with TypeSafe's address, provider name, x-typesafe-request-id and a required key. Constructors, build, buildValidated, PROVIDER and DEFAULT_MODEL keep their signatures.
ClassificationResult.Selected embabel-agent-ai Carries the per-category distribution a model reported. It must cover exactly the question's options and sum to 1.

Running a model locally needs no key:

var ollama = new SystemOneEndpoint("Ollama", URI.create("http://localhost:11434"));
DecisionService clefFlash = new SystemOneModelFactory(ollama, "clef-flash", () -> null).build();

The System One reference page covers pulling clef or clef-flash into Ollama, a curl check, plain Java and Spring wiring, and the smoke test.

Transport

  • An error status fails without the body being parsed or logged. A success body is read up to the endpoint's limit.
  • No redirects on the factory's own client, and no retries.
  • An application RestClient.Builder is cloned, so its interceptors and timeouts apply. The key is set by the last interceptor and replaces any Authorization header the application adds.
  • Each call is observed as embabel.systemone.request with a provider tag.

Failure behaviour

  • A failed call returns failure outcomes with one WARN line: the call kind, the reason, then service, provider, operation, cause, status, attempts, elapsed time and request id. The line never holds the key or the request content.
  • A missing or unusable key fails as UNAVAILABLE with cause credential before anything is sent.
  • A response that can't be read as a whole fails the request as INVALID_RESPONSE. A malformed answer fails only its own question.
  • An interrupted caller is cancelled before anything is sent and keeps its interrupt flag, including during buildValidated().

Behaviour changes

  • Single classify and assess calls go out keyed q1, like every other question set.
  • The observation is embabel.systemone.request. It was embabel.typesafe.request.
  • A missing or unusable TypeSafe key fails as UNAVAILABLE with cause credential.
  • Selected is a plain class like RatingResult.Answered, with an unmodifiable copy of the distribution. It loses copy and destructuring. It has not been released.
  • The typesafe-java-sdk dependency is gone.

@igordayen

@igordayen igordayen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jimador solid design/code. first round of manual review. coding agent review to follow. thanks

@igordayen

Copy link
Copy Markdown
Contributor

A few non-blocking points from CODEX

  • Rating legend validation is limited. SystemOneAnswers.kt:157 checks the legend’s indexes, but not its values. A response with reversed descriptions
    would still be interpreted using the requested level order. This behavior existed in the previous adapter; consider validating it or explicitly
    documenting that indexes alone are authoritative.

  • Exception traversal should be cycle-safe. SystemOneTransport.kt:323 walks causes without tracking visited exceptions. An application-supplied client or
    credential supplier returning a cyclic cause chain could hang error handling. Low-frequency hardening, not a normal-path blocker.

  • Add an interrupted-validation edge-case test. buildValidated() resolves credentials before reaching the transport’s interrupt check. Test an already-
    interrupted caller with a missing credential: currently it can receive InvalidApiKeyException rather than cancellation.

Thanks

A choice result can now carry the per-category probabilities a model reported.
The distribution must cover exactly the question's options, and the assembler
reports a selection whose distribution does not fit as a bad distribution.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
One encoder and decoder for the System One wire format, shared by every
endpoint that speaks it. Questions go out keyed q1 to qN, so caller question
names stay in the process. A response that can't be read as a whole fails the
request. Each answer is checked on its own, so one bad answer fails only its
question.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
SystemOneModelFactory builds decision services for any endpoint that speaks
System One. It reads the key on every call, never reads error bodies, caps the
response size, observes each call and logs one line per failure with fixed
fields. buildValidated checks a key by listing the endpoint's models.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
A System One service reports every question kind and answers single
proposition and rating questions through the per-question hooks, each as a
set of one.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
TypeSafeModelFactory is now a SystemOneModelFactory with TypeSafe's address,
provider name and request id header, so the community SDK and its transport
guard go away. Its constructors, build methods and BYOK validation keep their
signatures. Single classifications and propositions now go out keyed q1 like
every other question set.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
A new System One page covers the wire format, the client, transport,
failures, logging and observations. The TypeSafe page now covers only what
TypeSafe adds, and the decision pages name System One as a provider family.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
- Turn off response buffering on the client, so the size cap and the
  never-read error body hold with an application client too.
- Drop an Authorization header the application client adds when the key
  is blank.
- Cancel a call whose key supplier was interrupted.
- Validate a BYOK key by listing models with the key that was read.
- Keep a failing observation handler from changing a result or leaving a
  scope open.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
The key now replaces an Authorization header from the application's
initializers and interceptors too. A key supplier that throws
InterruptedException directly cancels the call. A handler that fails while an
observation starts or opens its scope no longer leaves the earlier handlers'
state open.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Ollama and OpenRouter list models as {"data": [{"id"}]}, so the codec now
reads that shape beside the System One {"models": [{"name"}]} listing.
SystemOneLocalSmokeTest runs a triage spec, a classification and an
assessment against a local System One endpoint such as CLEF Flash in Ollama
when SYSTEMONE_SMOKE_URL is set.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
The System One page gains a section on running decision models locally:
pulling clef or clef-flash, checking the endpoint with curl, using it from
plain Java and Spring, and running the smoke test. The examples are compiled
and tested.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Resolves the five java:S5778 issues SonarCloud reports on the TypeSafe
factory tests.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
buildValidated caught IllegalStateException around the model listing.
CancellationException extends it, so an interrupt during the listing
was reported as an invalid key. Rethrow cancellation first, so the
caller sees the interrupt and keeps its flag.

The read timeout test now runs against the fallback client with the
endpoint's own readTimeout on a separate slow context, and the slow
handler waits on a latch released after each test. The response limit
message states the accepted range.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Add short KDoc and Javadoc blocks to the private and internal helpers
in the System One transport, service, observation, failure, answer and
codec code, and fold the why comments into them. Rename the TypeSafe
key supplier parameter to apiKey to match SystemOneModelFactory. Share
one distribution tolerance constant between ClassificationResult and
SystemOneAnswers. Fix the validate throws text, the example test logger
name and the smoke test command.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
List every field of the System One WARN line, say which calls nest in
a framework observation, and add INFO to the logging table row. Drop
third party names that are not dependencies, the performance claim and
the history framing. Fix the zero confidence rule on the TypeSafe page
and add the System One module to the Dokka aggregate.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Selected compared confidence with ==, which treats -0.0 and 0.0 as equal,
while its hash code tells them apart. Objects.equals keeps equal
selections hashing alike.

Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
@jimador
jimador force-pushed the feature/systemone-protocol branch from 5e7863d to cee6d83 Compare October 16, 2026 02:53
@jimador

jimador commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator Author

on these three points

kept the mapping positional and documented it. added a reversed-label test so it's clear that returned labels don't change the requested level order

fixed, cause traversal now tracks exception identity so cycles terminate. added tests for client and credential failures, including a timeout inside a cycle

good catch, validation now checks for interruption before touching credentials. added coverage for missing and blank keys too

@igordayen igordayen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jimador - looks good, thanks for addressing feedback.

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants