refactor: consolidate on_schema_mismatch into reprompt config (#493)

* refactor(config): move on_schema_mismatch + strict_schema into RepromptConfig

Remove on_schema_mismatch and strict_schema from ActionConfig and DefaultsConfig.
Add on_schema_mismatch: Literal['reject', 'reprompt'] | None to RepromptConfig.
Schema conformance checking is now declared inside reprompt: block only.

* refactor(processing): _resolve_schema_mismatch_mode reads reprompt config

Read on_schema_mismatch from reprompt dict instead of top-level key.
Remove ON_SCHEMA_MISMATCH_KEY and STRICT_SCHEMA_KEY imports from helpers.py
and factory.py. Update warning messages to reference new config location.

* refactor(validation): granularity validator uses reprompt.on_schema_mismatch

Remove old top-level on_schema_mismatch check. Add check that
reprompt.on_schema_mismatch: reprompt/reject requires a schema defined.

* chore: remove ON_SCHEMA_MISMATCH_KEY + STRICT_SCHEMA_KEY constants

Both constants are dead — no production code references them after
the config model and processing changes.

* test: migrate on_schema_mismatch fixtures to reprompt config

Update all test fixtures from top-level on_schema_mismatch/strict_schema
to reprompt: {on_schema_mismatch: reject/reprompt}. Remove tests for
warn mode and strict_schema alias. 189 tests pass.

* docs: update output-validation + reprompt-patterns for new config shape

Move on_schema_mismatch inside reprompt: block in all examples.
Remove warn mode and strict_schema references.

* refactor(test): extract shared _make_schema_config helper

De-duplicate _make_config which was copied verbatim across
TestValidateSchemaSkip and TestRepromptFallbackWarning.

* chore: add changie breaking change entry for reprompt consolidation

* fix: changie timestamp to valid ISO-8601

* fix: add isinstance guard for non-dict reprompt + extra=forbid on RepromptConfig

Granularity validator now uses isinstance(reprompt_raw, dict) instead of
truthy-or to prevent AttributeError when reprompt is a non-dict value.
RepromptConfig gains extra='forbid' to catch typos like on_schema_mistmatch.

* fix: improve reprompt CLI output clarity

- Client parse warnings downgraded to debug (reprompt loop owns the messaging)
- Remove redundant 'Validation failed' after 'Invalid JSON' (one message per event)
- Add 'Retrying with feedback (attempt N/M)' before each retry
- Only log 'Reprompt passed on attempt N/M' when it took >1 attempt
- 'Schema validation failed' only shown for schema errors, not parse errors

* fix: include expected schema fields in JSON parse error feedback

When the model returns empty/invalid JSON, the reprompt feedback now
includes the expected field names from the schema. This guides the model
on WHAT to produce, not just 'return JSON'.

Example feedback:
  Your previous response was not valid JSON. Return only a JSON object...
  Expected JSON fields: summary, exam_density, key_topics

* fix: forceful JSON parse feedback with expected field names

When the model returns empty/invalid JSON, the reprompt feedback is now
assertive and includes the exact fields expected from the schema.

Example:
  CRITICAL: Your previous response was empty or not valid JSON.
  This is a strict JSON pipeline — every response MUST be a single
  JSON object. No markdown. No code fences. No explanation.

  You MUST return a JSON object with these fields: summary, exam_density
  Example structure:
  {
    "summary": "...",
    "exam_density": "...",
  }

* docs: fix dangling reprompt blocks missing on_schema_mismatch

Every reprompt: example now includes on_schema_mismatch: reprompt or
validation: to match actual framework behavior. A bare reprompt block
without either does not create a RepromptService.

Also added on_schema_mismatch, use_self_reflection, use_llm_critique,
and critique_after_attempt to the reprompt-patterns field reference table.

Author: Muizzkolapo

.changes/unreleased/Breaking Change-20260504-493000.yaml

@@ -0,0 +1,3 @@
+kind: Breaking Change
+body: "Remove top-level on_schema_mismatch and strict_schema config keys. Schema conformance is now declared inside reprompt: {on_schema_mismatch: reject/reprompt}."
+time: 2026-05-04T14:33:00.000000Z

agent_actions/config/schema.py

@@ -89,11 +89,17 @@ class RetryConfig(BaseModel):
 class RepromptConfig(BaseModel):
     """Configuration for reprompt behavior on validation failures.
 
-    ``validation`` is optional when an external validator is provided
-    (e.g. via ``on_schema_mismatch: reprompt``).
+    ``validation`` is optional when ``on_schema_mismatch`` is set — the schema
+    itself serves as the validator.
     """
 
+    model_config = ConfigDict(extra="forbid")
+
     validation: str | None = Field(default=None, description="Name of validation UDF function")
+    on_schema_mismatch: Literal["reject", "reprompt"] | None = Field(
+        default=None,
+        description="Schema conformance check: reject (hard fail) or reprompt (retry on mismatch)",
+    )
     max_attempts: int = Field(
         default=2,
         ge=1,
@@ -187,12 +193,6 @@ class ActionConfig(BaseModel):
     reprompt: RepromptConfig | None = Field(
         default=None, description="Reprompt configuration for validation failures"
     )
-    strict_schema: bool | None = Field(
-        default=None, description="Enable strict schema validation (reject on mismatch)"
-    )
-    on_schema_mismatch: Literal["warn", "reprompt", "reject"] | None = Field(
-        default=None, description="Schema mismatch mode: warn, reprompt, or reject"
-    )
     idempotency_key: str | None = Field(default=None, description="Idempotency key template")
     prompt: str | None = Field(default=None, description="Prompt template or reference")
     dependencies: list[str] = Field(
@@ -361,10 +361,6 @@ class DefaultsConfig(BaseModel):
     )
     constraints: Any | None = Field(default=None, description="Default constraints")
     retry: RetryConfig | None = Field(default=None, description="Default retry configuration")
-    strict_schema: bool | None = Field(default=None, description="Default strict schema flag")
-    on_schema_mismatch: Literal["warn", "reprompt", "reject"] | None = Field(
-        default=None, description="Default schema mismatch mode"
-    )
 
     # --- Expander-consumed keys ---
     context_scope: dict[str, Any] | None = Field(default=None, description="Default ctx scope")

agent_actions/llm/providers/ollama/client.py

@@ -170,7 +170,7 @@ def _call_ollama_json(
             parsed = json.loads(content)
             return [parsed] if isinstance(parsed, dict) else [{"response": parsed}]
         except json.JSONDecodeError as e:
-            logger.warning(
+            logger.debug(
                 "%s/%s returned invalid JSON: %s",
                 vendor_slug,
                 model,

agent_actions/llm/providers/openai/client.py

@@ -127,15 +127,15 @@ def call_json(
                     request_id=request_id,
                 )
             )
-            logger.warning(
+            logger.debug(
                 "Empty response content from OpenAI API, model=%s",
                 model_name,
             )
             return [{"raw_response": "", "_parse_error": "Empty response from API"}]
         try:
             response_data: dict[str, Any] | list[dict[str, Any]] = json.loads(response_content)
         except json.JSONDecodeError as e:
-            logger.warning(
+            logger.debug(
                 "Failed to parse JSON from OpenAI response: %s (snippet: %.200s)",
                 e,
                 response_content,

agent_actions/processing/helpers.py

@@ -6,7 +6,7 @@
 from typing import Any
 
 from agent_actions.errors import SchemaValidationError
-from agent_actions.utils.constants import ON_SCHEMA_MISMATCH_KEY, SCHEMA_KEY, STRICT_SCHEMA_KEY
+from agent_actions.utils.constants import SCHEMA_KEY
 from agent_actions.utils.transformation import PassthroughTransformer
 
 logger = logging.getLogger(__name__)
@@ -70,20 +70,14 @@ def run_dynamic_agent(
 
 
 def _resolve_schema_mismatch_mode(agent_config: dict[str, Any]) -> str:
-    """Resolve on_schema_mismatch to 'warn', 'reprompt', or 'reject'."""
-    explicit = agent_config.get(ON_SCHEMA_MISMATCH_KEY)
-    if explicit in ("warn", "reprompt", "reject"):
-        return str(explicit)
-
-    if explicit is not None:
-        logger.warning(
-            "Unrecognized on_schema_mismatch value '%s', defaulting to 'warn'",
-            explicit,
-        )
-
-    if agent_config.get(STRICT_SCHEMA_KEY, False):
-        return "reject"
+    """Resolve schema mismatch mode from reprompt.on_schema_mismatch.
 
+    Returns ``"reject"``, ``"reprompt"``, or ``"warn"`` (internal signal for
+    no enforcement).
+    """
+    reprompt = agent_config.get("reprompt")
+    if isinstance(reprompt, dict) and reprompt.get("on_schema_mismatch"):
+        return str(reprompt["on_schema_mismatch"])
     return "warn"
 
 
@@ -96,9 +90,9 @@ def _validate_llm_output_schema(
 ) -> Any:
     """Validate LLM output against expected schema if defined.
 
-    Returns the response unchanged. When ``on_schema_mismatch`` is "reprompt"
-    and ``skip_schema_validation`` is True, validation is deferred to the
-    outer reprompt loop.
+    Returns the response unchanged. When ``reprompt.on_schema_mismatch`` is
+    "reprompt" and ``skip_schema_validation`` is True, validation is deferred
+    to the outer reprompt loop.
 
     Raises:
         SchemaValidationError: If on_schema_mismatch="reject" and validation fails.
@@ -108,9 +102,8 @@ def _validate_llm_output_schema(
         mismatch_mode = _resolve_schema_mismatch_mode(agent_config)
         if mismatch_mode in ("reject", "reprompt"):
             logger.warning(
-                "Action '%s': on_schema_mismatch is '%s' but no schema is defined — "
-                "schema validation will be skipped. Define a schema or set "
-                "on_schema_mismatch to 'warn'.",
+                "Action '%s': reprompt.on_schema_mismatch is '%s' but no schema is "
+                "defined — schema validation will be skipped.",
                 agent_name,
                 mismatch_mode,
             )
@@ -141,8 +134,8 @@ def _validate_llm_output_schema(
         if not report.is_compliant:
             if strict_mode:
                 hint = (
-                    "Enable strict_schema: false to allow schema mismatches, "
-                    "or update the prompt to match expected schema"
+                    "Remove reprompt.on_schema_mismatch: reject to allow schema "
+                    "mismatches, or update the prompt to match expected schema"
                 )
                 if report.namespace_hint:
                     hint = f"{hint}. {report.namespace_hint}"

agent_actions/processing/invocation/factory.py

@@ -77,7 +77,7 @@ def _build_validator(agent_config: dict[str, Any]) -> ResponseValidator | None:
             SchemaValidator,
             UdfValidator,
         )
-        from agent_actions.utils.constants import SCHEMA_KEY, STRICT_SCHEMA_KEY
+        from agent_actions.utils.constants import SCHEMA_KEY
 
         validators: list[ResponseValidator] = []
 
@@ -92,8 +92,7 @@ def _build_validator(agent_config: dict[str, Any]) -> ResponseValidator | None:
             mode = _resolve_schema_mismatch_mode(agent_config)
             if mode == "reprompt":
                 action_name = agent_config.get("name", "unknown")
-                strict = agent_config.get(STRICT_SCHEMA_KEY, False)
-                validators.append(SchemaValidator(schema, action_name, strict_mode=strict))
+                validators.append(SchemaValidator(schema, action_name))
 
         if not validators:
             return None

agent_actions/processing/recovery/reprompt.py

@@ -19,10 +19,46 @@
 )
 from .retry import RetryExhaustedException
 
-_JSON_PARSE_FEEDBACK = (
-    "Your previous response was not valid JSON. "
-    "Return only a JSON object — no markdown, no explanation, no code fences."
-)
+
+def _extract_field_names(schema: dict) -> list[str]:
+    """Extract field names from any schema format."""
+    # fields-style (agent-actions format)
+    fields = schema.get("fields")
+    if isinstance(fields, list):
+        return [f.get("id") or f.get("name", "") for f in fields if isinstance(f, dict)]
+    # properties-style (JSON Schema format)
+    props = schema.get("properties")
+    if isinstance(props, dict):
+        return list(props.keys())
+    # Inline schema: keys are field names directly
+    if all(isinstance(v, str) for v in schema.values()):
+        return list(schema.keys())
+    return []
+
+
+def _build_json_parse_feedback(validator: Any) -> str:
+    """Build forceful JSON parse error feedback with expected schema fields."""
+    schema = getattr(validator, "_schema", None)
+    field_names = _extract_field_names(schema) if isinstance(schema, dict) else []
+
+    if field_names:
+        fields_str = ", ".join(field_names)
+        return (
+            "CRITICAL: Your previous response was empty or not valid JSON. "
+            "This is a strict JSON pipeline — every response MUST be a single "
+            "JSON object. No markdown. No code fences. No explanation. No "
+            "preamble. Just the raw JSON object.\n\n"
+            f"You MUST return a JSON object with these fields: {fields_str}\n\n"
+            "Example structure:\n"
+            "{\n" + "".join(f'  "{name}": "...",\n' for name in field_names) + "}"
+        )
+
+    return (
+        "CRITICAL: Your previous response was empty or not valid JSON. "
+        "This is a strict JSON pipeline — every response MUST be a single "
+        "JSON object. No markdown. No code fences. No explanation. No "
+        "preamble. Just the raw JSON object."
+    )
 
 
 def _get_parse_error(response: Any) -> str | None:
@@ -206,7 +242,7 @@ def execute(
             if parse_error is not None:
                 is_valid = False
                 logger.warning(
-                    "[%s] Provider returned invalid JSON on attempt %d/%d: %s",
+                    "[%s] Invalid JSON on attempt %d/%d — %s",
                     context,
                     attempts,
                     self.max_attempts,
@@ -216,12 +252,13 @@ def execute(
                 is_valid = safe_validate(self._validator.validate, response, context=context)
 
             if is_valid:
-                logger.info(
-                    "[%s] Validation passed on attempt %d/%d",
-                    context,
-                    attempts,
-                    self.max_attempts,
-                )
+                if attempts > 1:
+                    logger.info(
+                        "[%s] Reprompt passed on attempt %d/%d",
+                        context,
+                        attempts,
+                        self.max_attempts,
+                    )
                 return RepromptResult(
                     response=response,
                     executed=True,
@@ -231,18 +268,26 @@ def execute(
                     exhausted=False,
                 )
 
-            logger.warning(
-                "[%s] Validation failed on attempt %d/%d",
-                context,
-                attempts,
-                self.max_attempts,
-            )
+            if parse_error is None:
+                logger.warning(
+                    "[%s] Schema validation failed on attempt %d/%d",
+                    context,
+                    attempts,
+                    self.max_attempts,
+                )
 
             if attempts >= self.max_attempts:
                 break
 
+            logger.info(
+                "[%s] Retrying with feedback (attempt %d/%d)",
+                context,
+                attempts + 1,
+                self.max_attempts,
+            )
+
             if parse_error is not None:
-                feedback = _JSON_PARSE_FEEDBACK
+                feedback = _build_json_parse_feedback(self._validator)
             else:
                 feedback = build_validation_feedback(
                     response, self._validator.feedback_message, strategies=self._strategies

agent_actions/skills/agac-agent-skills/references/reprompt-patterns.md

@@ -28,24 +28,28 @@ How to configure automatic retry with feedback when LLM output fails validation.
 
 | Field | Type | Default | Description |
 |-------|------|---------|-------------|
+| `on_schema_mismatch` | string | None | `"reprompt"` (retry on schema fail) or `"reject"` (hard fail) |
 | `validation` | string | None | Name of `@reprompt_validation` function |
 | `max_attempts` | int | 2 | Total attempts including first try (1-10) |
 | `on_exhausted` | string | `"return_last"` | What to do when all attempts fail |
+| `use_self_reflection` | bool | false | Add self-analysis prompt before retry |
+| `use_llm_critique` | bool | false | Use a second LLM call to critique failures |
+| `critique_after_attempt` | int | 2 | Critique starts on this attempt number |
 
 **`on_exhausted` options:**
 - `"return_last"` — Accept the last response even though it failed validation. Downstream actions receive potentially invalid data.
 - `"raise"` — Raise a `RuntimeError`, failing the action. Use this when invalid data is worse than no data.
 
 ## Schema-Based Reprompt (No Custom UDF)
 
-For simple schema validation without writing Python, use `on_schema_mismatch`:
+For simple schema validation without writing Python, set `on_schema_mismatch` inside the `reprompt` block:
 
 ```yaml
 - name: classify_issue
   schema: issue_classification
   json_mode: true
-  on_schema_mismatch: reprompt         # "warn" | "reprompt" | "reject"
   reprompt:
+    on_schema_mismatch: reprompt       # "reprompt" | "reject"
     max_attempts: 3
 ```
 
@@ -101,13 +105,13 @@ tools/
 
 ## Composed Validators
 
-When both `validation` (custom UDF) and `on_schema_mismatch: reprompt` are configured, validators are composed — the schema check runs first, then the custom UDF. Fails on the first failure.
+When both `validation` (custom UDF) and `on_schema_mismatch: reprompt` are configured in the same `reprompt` block, validators are composed — the schema check runs first, then the custom UDF. Fails on the first failure.
 
 ```yaml
 - name: generate_catalog_entry
   schema: catalog_entry
-  on_schema_mismatch: reprompt         # Layer 1: schema check
   reprompt:
+    on_schema_mismatch: reprompt       # Layer 1: schema check
     validation: "check_valid_bisac"    # Layer 2: custom business logic
     max_attempts: 3
 ```
@@ -189,8 +193,8 @@ def check_genre_classification(response: dict) -> bool:
     entities: array[string]!           # Required array of strings
     confidence: number!                # Required number
   json_mode: true
-  on_schema_mismatch: reprompt
   reprompt:
+    on_schema_mismatch: reprompt
     max_attempts: 2
     on_exhausted: raise                # Fail if schema still violated
 ```

agent_actions/utils/constants.py

@@ -12,8 +12,6 @@
 PROMPT_KEY = "prompt"
 SCHEMA_NAME_KEY = "schema_name"
 SCHEMA_KEY = "schema"
-STRICT_SCHEMA_KEY = "strict_schema"
-ON_SCHEMA_MISMATCH_KEY = "on_schema_mismatch"
 CHUNK_CONFIG_KEY = "chunk_config"
 
 # Reserved agent/action names that cannot be used in workflows.

agent_actions/validation/action_validators/granularity_output_field_validator.py

@@ -4,7 +4,6 @@
 from agent_actions.utils.constants import (
     HITL_FILE_GRANULARITY_ERROR,
     JSON_MODE_KEY,
-    ON_SCHEMA_MISMATCH_KEY,
     SCHEMA_KEY,
     SCHEMA_NAME_KEY,
 )
@@ -56,32 +55,19 @@ def validate(self, context) -> ActionEntryValidationResult:
             if json_mode:
                 errors.append(f"{desc} 'output_field' can only be used when 'json_mode' is false.")
 
-        on_mismatch_raw = normalized_entry.get(ON_SCHEMA_MISMATCH_KEY)
-        if isinstance(on_mismatch_raw, str):
-            on_mismatch = on_mismatch_raw.lower()
-        else:
-            on_mismatch = None
-
-        if on_mismatch in ("reject", "reprompt"):
+        reprompt_raw = normalized_entry.get("reprompt")
+        reprompt_cfg = reprompt_raw if isinstance(reprompt_raw, dict) else {}
+        schema_mismatch_mode = reprompt_cfg.get("on_schema_mismatch")
+        if schema_mismatch_mode in ("reprompt", "reject"):
             has_schema = bool(
                 normalized_entry.get(SCHEMA_KEY) or normalized_entry.get(SCHEMA_NAME_KEY)
             )
             if not has_schema:
                 errors.append(
-                    f"{desc} 'on_schema_mismatch: {on_mismatch}' requires a schema "
-                    "to validate against. Define 'schema' or 'schema_name', "
-                    "or change on_schema_mismatch to 'warn'."
+                    f"{desc} reprompt.on_schema_mismatch: {schema_mismatch_mode} requires "
+                    "a schema to validate against. Define 'schema' or 'schema_name'."
                 )
 
-            if on_mismatch == "reprompt":
-                reprompt = normalized_entry.get("reprompt")
-                if not reprompt:
-                    errors.append(
-                        f"{desc} 'on_schema_mismatch: reprompt' requires a 'reprompt' "
-                        "configuration block. Add reprompt: {{validation: your_udf_name}} "
-                        "or change on_schema_mismatch to 'warn' or 'reject'."
-                    )
-
         if errors:
             return ActionEntryValidationResult.with_errors(errors)