|
1 | | -# Recovery Postmortem Generator |
| 1 | +# Recovery Postmortem |
2 | 2 |
|
3 | | -Converge includes an optional recovery postmortem workflow that turns benchmark, chaos, and convergence evidence into a structured incident-style summary. |
| 3 | +The postmortem generator is evidence-driven. It uses benchmark artifacts, chaos artifacts, and live snapshots to generate a bounded summary instead of pretending the model has extra context. |
4 | 4 |
|
5 | | -## Architecture |
| 5 | +## Inputs |
6 | 6 |
|
7 | | -- Evidence collection gathers benchmark artifacts, chaos artifacts, workflow snapshots, convergence snapshots, retry counts, DLQ counts, worker health, and recovery timing. |
8 | | -- LangGraph orchestrates the workflow: |
9 | | - - collect recovery evidence |
10 | | - - summarize the recovery timeline |
11 | | - - assess convergence |
12 | | - - identify operational risks |
13 | | - - generate the postmortem |
14 | | - - validate the schema |
15 | | - - verify evidence grounding |
16 | | -- LangChain structured-output parsing keeps the response aligned with the Pydantic schema. |
17 | | -- Ollama is supported as the optional local model backend. |
18 | | -- A fake provider powers tests, CI, and offline validation. |
| 7 | +- benchmark JSON/Markdown under `benchmarks/` |
| 8 | +- chaos JSON/Markdown under `benchmarks/` |
| 9 | +- optional workflow/convergence snapshots from the API |
19 | 10 |
|
20 | | -## Runtime Defaults |
21 | | - |
22 | | -- `AI_PROVIDER=disabled` |
23 | | -- `OLLAMA_BASE_URL=http://localhost:11434` |
24 | | -- `AI_MODEL=llama3.1:8b` |
25 | | -- `AI_FALLBACK_MODEL=qwen2.5-coder:7b` |
26 | | -- `AI_TIMEOUT_SECONDS=20` |
27 | | - |
28 | | -The service does not require Ollama at startup. If Ollama is unavailable, the generator falls back to deterministic evidence-based output. |
29 | | - |
30 | | -## How To Generate |
| 11 | +## Usage |
31 | 12 |
|
32 | 13 | ```bash |
33 | 14 | python scripts/generate_postmortem.py --artifact benchmarks/benchmark_replay_*.json --workflow-id <workflow_id> |
34 | | -``` |
35 | | - |
36 | | -You can also point the CLI at a running API: |
37 | | - |
38 | | -```bash |
39 | | -python scripts/generate_postmortem.py --artifact benchmarks/benchmark_replay_*.json --base-url http://127.0.0.1:18000 --workflow-id <workflow_id> |
40 | | -``` |
41 | | - |
42 | | -## Evaluation |
43 | | - |
44 | | -```bash |
45 | 15 | python scripts/evaluate_postmortem.py |
46 | 16 | ``` |
47 | 17 |
|
48 | | -The evaluation harness checks: |
49 | | - |
50 | | -- schema-valid output rate |
51 | | -- recovery classification accuracy |
52 | | -- evidence coverage |
53 | | -- average latency |
54 | | -- insufficient-evidence handling |
55 | | - |
56 | | -## Evidence Rules |
57 | | - |
58 | | -- The generator must cite actual evidence collected from artifacts or live recovery state. |
59 | | -- If the evidence is too thin, it returns `insufficient_evidence`. |
60 | | -- The output should not claim 100K-event chaos runs or 3,000+ msg/sec throughput unless those numbers are actually measured in this repository. |
| 18 | +## Guardrails |
61 | 19 |
|
62 | | -## CI Behavior |
| 20 | +- Returns `insufficient_evidence` when the source material is too thin |
| 21 | +- Uses local deterministic mode by default |
| 22 | +- Optional external providers are only used when configured |
| 23 | +- Does not replace the recovery engine or the operator console |
63 | 24 |
|
64 | | -- CI and tests use the fake provider. |
65 | | -- No paid API keys are required. |
66 | | -- Ollama is optional and not part of the default startup path. |
0 commit comments