gliner_decide.md

GLiNER2.5-Decide on colibri

GLiNER2.5-Decide, by fastino (Apache-2.0), is a classifier for operational decisions: intent, routing, sentiment, urgency, moderation, yes/no questions over a passage, ordinal scores. It is a GLiNER2 checkpoint, a DeBERTa-v3-large encoder with GLiNER2's heads, and it never generates text.

c/gliner_decide runs it from the official checkpoint with no conversion, and coli serve exposes it on POST /v1/systemone, the same request and reply as TypeSafe's Jev API and as every other model colibri serves (systemone.md).

Download and run

The checkpoint is the root of the repository (1.9 GB, F32):

hf download fastino/GLiNER2.5-Decide model.safetensors config.json \
   "encoder_config/*" tokenizer.json tokenizer_config.json special_tokens_map.json \
   --local-dir ~/Models/GLiNER2.5-Decide
make -C c gliner_decide
./c/coli serve --model ~/Models/GLiNER2.5-Decide

coli recognises the checkpoint by its config.json (model_type extractor, GLiNER2's) and the encoder's encoder_config/config.json; the registry keys it gliner2_span_deberta-v2. coli info shows the encoder and the token budget, coli plan the RAM, coli doctor what is missing.

curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "jev-latest",
  "state": "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan.",
  "questions": {
    "department": {"type": "choice", "instructions": "Which department should handle this?",
                   "criteria": {"billing": "invoices, payments, refunds",
                                "technical": "bugs, outages, system errors", "other": "everything else"}},
    "urgency": {"type": "score", "instructions": "How urgent is this?",
                "criteria": ["not urgent", "soon", "blocking"]},
    "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"}}}'

{"id": "req_...", "model": "gliner2.5-decide", "provider": "colibri",
 "answers": {
   "department": {"type": "choice", "choice": "billing",
                  "probabilities": {"billing": 0.751237, "technical": 0.092047, "other": 0.156716},
                  "confidence": 0.626855},
   "urgency": {"type": "score", "score": 1.218665,
               "legend": {"0": "not urgent", "1": "soon", "2": "blocking"},
               "probabilities": {"0": 0.167397, "1": 0.446541, "2": 0.386062}, "confidence": 0.169812},
   "churn_risk": {"type": "noul", "noul": 0.995977}},
 "usage": {"input_tokens": 118, "output_tokens": 0, "cost": 0}}

That is the reply coli serve gave on the release (the id shortened). input_tokens is the length of the one sequence the encoder read. Chat, completions and messages answer 400 with a pointer to /v1/systemone, and coli chat says the same before starting anything.

From System One to the model's task

GLiNER2 has no notion of choice, score or noul: it classifies a text against named tasks, each with its labels. The engine turns a System One request into the call the model card makes, classify_text(state, tasks), one task per question, in the order given:

System One	GLiNER2 task
question id	the task name (department, urgency)
instructions	the task's prompt (department: Which department should handle this?)
choice criteria	the labels, in order; a non-empty description becomes a label description
score criteria	the labels "0" .. "n-1", the way the model card writes an ordinal scale; each level's text becomes its description
noul	the labels "yes" then "no", the order of every yes/no example of the model card; criteria.true / criteria.false describe them

Every task is single-label, so its probabilities are the softmax over its labels, which is what classify_text returns for a single-label task. The reply follows from them as on every engine: choice is the argmax, score the expected level, noul the probability of yes.

Two consequences of reading the request as one call:

• All questions share one sequence. The encoder reads every question's schema and the state together, so a question's probabilities depend on the other questions of the same request, as they do in the package when one call scores several tasks (the model card's email triage). One request is one forward pass, whatever the number of questions.
• Missing instructions are filled by the gateway with the text the language-model path asks with (Which of the following applies?, Rate this on the scale below., Is this true?). For GLiNER2 that text becomes the task's prompt. Measured on eleven single-task examples of the model card, asked with no prompt and with the default one (Is this true? for the yes/no labels, Rate this on the scale below. for the 0-5 and 0-10 scales, Which of the following applies? for the rest), the default prompt changed no decision and moved the top probability by less than 0.001 up to 0.12 (refund_request 0.997 -> 0.969, seat_change 0.501 -> 0.622). Send instructions when you have them.

What the engine computes

Each step follows the gliner2 package (2.0.0) and the transformers DeBERTa-v2 code it runs, which are the reference:

step	reference	in c/gliner_decide.c
the text	processor._collate_batch	a . appended when the state does not end in ., ! or ?; an empty state is .
words	processing/word_splitter.WhitespaceTokenSplitter	Python's regex (URLs, e-mails, @handles, \w+(?:[-_]\w+)*, \S) with Python's own \w, \s and re.IGNORECASE, each word lower-cased with str.lower() (final sigma and dotted capital I included); the tables come from Python by tools/gen_gliner_unicode.py
schema	Schema.classification, processor._transform_schema	per task ( [P] name[: prompt][ [DESCRIPTION] label: text ...] ( [L] label ... ) ), tasks joined by [SEP_STRUCT], then [SEP_TEXT] and the words
tokens	processor._format_input_with_mapping	every element above tokenized on its own, no [CLS] or [SEP]; the classifier reads the first token of each [L]
tokenizer	tokenizer.json (DebertaV2TokenizerFast)	added tokens split out of the raw text, Replace(\s{2,}|[\n\r\t] -> " "), NFC, right strip, [UNK] matched after normalization, Metaspace (prepend always, split), Unigram (best-scoring segmentation, unknown characters at min score - 10, adjacent unknowns fused into one [UNK])
encoder	DebertaV2Model	word embeddings and LayerNorm (no absolute positions); 24 post-norm layers of disentangled attention: (q.k + q.pos_k[r] + k.pos_q[r]) / sqrt(3 * 64) with r the relative position in log buckets (256 buckets, maximum 512, clamped to the 512-row table), pos_q and pos_k the LayerNorm'd relative embeddings through the layer's own query and key projections; GELU FFN
head	runtime._extract_classification_result	the classifier (Linear 1024 -> 2048, ReLU, Linear 2048 -> 1) at each [L], temperature 1, softmax over the task's labels

The span, count and relation heads (span_rep, count_pred, count_embed) serve extraction and are not loaded. The relative embeddings are the same for every request, so their projections are computed once at load. Weights are kept in f32 as released.

Checked against the reference

• Tiny oracle (make -C c gliner-decide-tiny-check, in CI): a random checkpoint in the release's layout (tools/make_gliner_decide_tiny.py, numpy, a Unigram tokenizer counted from a small corpus, a head size of 24, 16 position buckets with a maximum of 64 so every sequence reaches the log buckets and the clamp) and what gliner2 answers on it (gliner_decide_tiny/ref.json, written by tools/make_gliner_decide_ref.py): 13 requests with 26 questions plus one whose questions do not fit, covering every type, 2 to 40 options, labels with and without descriptions, missing instructions, long states and a conversation cut at max_len, JSON states and instructions, the GLiNER2 marker tokens inside the text, URLs, e-mails, handles, unicode and an empty state; plus 30 tokenizer strings and 15 word-splitter strings. Token ids and markers identical, probabilities within 2.0e-6 (tolerance 2e-5), every decision the same; then the same fixture through coli serve.
• Tokenizer and word splitter fuzzing (during development, not in CI): 20,013 random strings per seed mixing ASCII, whitespace of every kind, Greek (final sigma), Cyrillic, CJK, combining marks, emoji, the Kelvin sign, long s, dotted and dotless i, URLs, e-mails and the added tokens. Four seeds against the release's tokenizer and the word splitter, two against the tiny tokenizer: 0 differences in token ids, 0 in words.
• The real checkpoint (tools/compare_gliner_decide.py): 39 requests, 79 questions: every example of the model card written as System One questions (the multi-label ones as one noul per label), the Jev examples of systemone.md, a contract of about 960 tokens, a conversation, JSON states and instructions, 40 banking intents, ten questions on one ticket, unicode, emoji, URLs, the marker tokens in the text, an empty state. Token ids identical on 39/39, decisions 79/79 the same, max |dp| 1.7e-6.

Speed

Measured on an i7-1355U laptop (10 cores, 12 threads), 6 threads for both sides, on a machine shared with other jobs: the load average stayed between 10 and 13 during the runs below, so absolute numbers move from run to run. Engine and package were run in turn, request by request, so both saw the same load (median of 5 each):

request	tokens	coli serve, at the client	of which engine	gliner2 package (torch, CPU, f32)
one question (model card, spam or not)	37	294 ms	290 ms	718 ms
the README request, 3 questions	118	897 ms	888 ms	1217 ms
ten questions on one ticket	286	1955 ms	1950 ms	2231 ms
a contract, 3 questions	965	8401 ms	8397 ms	9457 ms

The gateway's share is 4 to 10 ms. Over the 39 requests of the comparison the engine's median was 502 ms per request (load average 5 to 8 when sampled). A request is one forward pass whatever the number of questions, so its time follows the length of the sequence: the ten questions above are one pass over 286 tokens, most of them the questions' own schema.

Longer sequences cost more than in proportion, the attention growing with the square of the length. A three-label question over a repeated support text, same machine: 0.54 s at 81 tokens and 2.8 s at 494 with the machine otherwise idle; with a load average of 7 to 15, 0.69 s at 81, 4.0 s at 494, 8.7 s at 970, 25.7 s at 1923 and 93 s at 3829. COLI_GLINER_MAX_LEN bounds it.

RAM: 1.81 GB resident for the engine serving the release (peak 1.88 GB): the f32 encoder and classifier, 436M parameters of the 486M in the file.

Knobs

variable	default	meaning
COLI_GLINER_MAX_LEN	4096	tokens of the one sequence a request becomes: the questions first, then as many whole words of the state as fit
OMP_NUM_THREADS	physical cores	as for the other engines

Questions whose schema alone exceeds COLI_GLINER_MAX_LEN are refused with a 422 that says how many tokens they take.

Limits

• English. The model card measures this checkpoint on English; fastino points to GLiNER2.5-multi-Decide for other languages, which this engine has not been run on.
• No calibration ships with the checkpoint. The probabilities are the softmax of the classifier at temperature 1, as the package computes them; temperature in the DECISION frame is null.
• Single-label only. System One has no multi-label question, so the package's multi_label mode (a sigmoid per label and a threshold) is not reachable; ask one noul per label instead, as the comparison above does.
• Long states. The package reads a whole text by default; the engine stops at COLI_GLINER_MAX_LEN tokens so that one request cannot take the machine, and state_dropped in the DECISION frame says how many state tokens were left out. Below the limit the two read the same tokens. The cost grows with the square of the length (see Speed).
• The noul label order. The engine asks yes before no; the order moves the probabilities a little (handoff example of the model card: p(yes) 0.325 asked yes-first, 0.297 asked no-first), never the decision on the examples measured.
• Question ids that extend each other. The package finds a task's answer by the prefix of its schema text, so with two question ids such as risk (with instructions score it) and risk: score, classify_text returns only one of them (checked on the tiny fixture). The engine answers every question by position.
• Other GLiNER2 checkpoints. The registry accepts any span GLiNER2 checkpoint with a DeBERTa-v2/v3 encoder, and a 24 x 1024 encoder is named GLiNER2.5-Decide. Only fastino/GLiNER2.5-Decide was compared with the package; the boundary architecture is refused.