Skip to content

Commit e2c66ad

Browse files
rokoss21rokoss21
committed
docs: add production scenario, benchmark report, and integration playbook
1 parent d953310 commit e2c66ad

7 files changed

Lines changed: 382 additions & 4 deletions

‎docs/05-examples-guide.md‎

Lines changed: 45 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@ title: Examples
55

66
# 05. FACET Examples Guide
77
**Reading Time:** 45-60 minutes | **Difficulty:** Beginner → Advanced | **Previous:** [04-type-system.md](04-type-system.html) | **Next:** [06-cli.md](06-cli.html)
8-
**Version:** 1.0
9-
**Last Updated:** 2025-12-09
10-
**Status:** Production Ready
8+
**Version:** 0.1.2
9+
**Last Updated:** 2026-04-02
10+
**Status:** Spec-aligned examples
1111

1212
## Table of Contents
1313

@@ -20,6 +20,7 @@ title: Examples
2020
- [Example 5: RAG Pipeline (`rag_pipeline.facet`)](#example-5-rag-pipeline-rag_pipelinefacet)
2121
- [Example 6: Advanced Features (`advanced_features.facet`)](#example-6-advanced-features-advanced_featuresfacet)
2222
- [Example 7: Testing (`test_example.facet`)](#example-7-testing-test_examplefacet)
23+
- [Example 8: Production Failure Flow (`production_support_flow.facet`)](#example-8-production-failure-flow-production_support_flowfacet)
2324
- [Common Patterns](#common-patterns)
2425
- [Creating Your Own Examples](#creating-your-own-examples)
2526
- [Troubleshooting](#troubleshooting)
@@ -28,7 +29,7 @@ title: Examples
2829

2930
## Overview
3031

31-
FACET ships with 8 comprehensive examples demonstrating different aspects of the language. Each example is runnable and shows real-world usage patterns.
32+
FACET ships with comprehensive examples demonstrating different aspects of the language. Each example is runnable and shows real-world usage patterns.
3233

3334
### Examples Directory Structure
3435

@@ -44,6 +45,8 @@ examples/
4445
└── README.md # Quick reference
4546
```
4647

48+
Additional documentation-only examples are in `docs/examples/`.
49+
4750
### Running Examples
4851

4952
All examples can be run with:
@@ -754,6 +757,44 @@ fct test --input examples/test_example.facet --output verbose
754757

755758
---
756759

760+
## Example 8: Production Failure Flow (`production_support_flow.facet`)
761+
762+
**File:** `docs/examples/production_support_flow.facet`
763+
**Purpose:** Show contract execution boundaries in a production-style support pipeline
764+
**Concepts:** `@input`, normalization pipeline, deterministic fail-fast behavior
765+
766+
### Source Code
767+
768+
```facet
769+
@meta
770+
name: "Support Escalation Contract"
771+
version: "1.0"
772+
773+
@context
774+
budget: 32000
775+
776+
@vars
777+
tenant: @input(type="string", default="default")
778+
query: @input(type="string")
779+
normalized_query: $query |> trim() |> lowercase()
780+
781+
@system
782+
content: "Classify support intent. Return JSON with keys: action, confidence, reason."
783+
784+
@user
785+
content: $normalized_query
786+
```
787+
788+
### Why It Matters
789+
790+
- Demonstrates strict request-side contract control.
791+
- Supports bounded retry/fallback strategy in host code.
792+
- Works with explicit rule: execution never continues after contract violation.
793+
794+
See the full walkthrough in [Production Scenario](16-production-scenario.html).
795+
796+
---
797+
757798
## Common Patterns
758799

759800
### Variable Processing Pipeline

‎docs/16-production-scenario.md‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
---
2+
permalink: /16-production-scenario.html
3+
title: Production Scenario
4+
---
5+
6+
# 16. Production Failure Scenario
7+
**Reading Time:** 10-12 minutes | **Difficulty:** Advanced
8+
**Compiler Version:** 0.1.2 | **Spec Version:** 2.1.3
9+
10+
This page shows a realistic multi-step path with failure, retry, and fallback behavior.
11+
12+
## 1. Contract
13+
14+
**Example contract:** [production_support_flow.facet](examples/production_support_flow.facet)
15+
16+
```facet
17+
@meta
18+
name: "Support Escalation Contract"
19+
version: "1.0"
20+
21+
@context
22+
budget: 32000
23+
24+
@vars
25+
tenant: @input(type="string", default="default")
26+
query: @input(type="string")
27+
normalized_query: $query |> trim() |> lowercase()
28+
29+
@system
30+
content: "Classify support intent. Return JSON with keys: action, confidence, reason."
31+
32+
@user
33+
content: $normalized_query
34+
```
35+
36+
## 2. Runtime Chain
37+
38+
1. Host calls `fct run` and gets canonical JSON.
39+
2. Host sends canonical JSON to model provider.
40+
3. Host validates model response against strict app schema.
41+
4. If schema fails, host applies bounded retry policy.
42+
5. If retries fail, host routes to deterministic fallback.
43+
44+
## 3. Failure Semantics
45+
46+
### Case A: Contract violation (FACET layer)
47+
48+
Examples:
49+
- missing required `@input` value (`F453`)
50+
- denied operation by policy (`F454`)
51+
- guard undecidable (`F455`)
52+
53+
Behavior:
54+
- Execution stops immediately.
55+
- No next transition is executed.
56+
- Error code is emitted as the contract outcome.
57+
58+
**Rule:** execution never continues after contract violation.
59+
60+
### Case B: Invalid model payload (application layer)
61+
62+
Examples:
63+
- model returns malformed JSON
64+
- JSON shape does not match expected app schema
65+
66+
Behavior:
67+
- FACET run itself is considered successful (request contract executed).
68+
- Host response validator fails the response path.
69+
- Host decides retry/fallback (outside FACET core).
70+
71+
## 4. Deterministic Retry Policy (Host)
72+
73+
A practical policy:
74+
75+
- max attempts: `2`
76+
- retry only for schema/parse failure
77+
- fixed backoff: `250ms`
78+
- no unbounded loops
79+
80+
Pseudo-flow:
81+
82+
```text
83+
compile+run -> provider call -> schema validate
84+
success -> continue
85+
fail -> retry(1)
86+
fail -> retry(2)
87+
fail -> fallback("manual-review")
88+
```
89+
90+
## 5. Why This Is Production-Safe
91+
92+
- FACET constrains request construction and guarded operations.
93+
- Host constrains response acceptance and recovery policy.
94+
- Both sides expose explicit failure surfaces.
95+
96+
## 6. Related References
97+
98+
- [Execution Model](15-execution-model.html)
99+
- [Error Codes](12-errors.html)
100+
- [Security](11-security.html)

‎docs/17-benchmark-report.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
---
2+
permalink: /17-benchmark-report.html
3+
title: Benchmark Report
4+
---
5+
6+
# 17. Mini Benchmark Report
7+
**Reading Time:** 8-10 minutes | **Difficulty:** Advanced
8+
**Compiler Version:** 0.1.2 | **Spec Version:** 2.1.3
9+
10+
This benchmark is a quick engineering signal, not a formal performance paper.
11+
12+
## 1. Goal
13+
14+
Show operational overhead of FACET execution compared with naive JSON assembly.
15+
16+
- **Baseline:** direct JSON construction with `jq` (no type checks, no policy checks, no R-DAG).
17+
- **FACET:** full `facet-fct run` pipeline.
18+
19+
## 2. Environment
20+
21+
- Host: Apple M2 Pro
22+
- OS: macOS
23+
- Binary: `target/release/facet-fct` (`fct 0.1.2`)
24+
25+
## 3. Commands Used
26+
27+
### Baseline (basic payload, 1000 iterations)
28+
29+
```bash
30+
/usr/bin/time -p /tmp/bench_baseline_basic.sh
31+
```
32+
33+
Script body:
34+
35+
```bash
36+
jq -c . examples/basic_prompt.input.json >/dev/null
37+
jq -n '{"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Hello"}]}' >/dev/null
38+
```
39+
40+
### FACET run (basic contract, 200 iterations)
41+
42+
```bash
43+
/usr/bin/time -p sh -c 'for i in $(seq 1 200); do target/release/facet-fct run --input examples/basic_prompt.facet --format json >/dev/null; done'
44+
```
45+
46+
### Baseline (RAG-like payload, 500 iterations)
47+
48+
```bash
49+
/usr/bin/time -p /tmp/bench_baseline_rag.sh
50+
```
51+
52+
### FACET run (RAG contract, 100 iterations)
53+
54+
```bash
55+
/usr/bin/time -p sh -c 'for i in $(seq 1 100); do target/release/facet-fct run --input examples/rag_pipeline.facet --format json >/dev/null; done'
56+
```
57+
58+
### FACET build-only (300 iterations each)
59+
60+
```bash
61+
/usr/bin/time -p sh -c 'for i in $(seq 1 300); do target/release/facet-fct build --input examples/basic_prompt.facet >/dev/null; done'
62+
/usr/bin/time -p sh -c 'for i in $(seq 1 300); do target/release/facet-fct build --input examples/rag_pipeline.facet >/dev/null; done'
63+
```
64+
65+
## 4. Results
66+
67+
| Scenario | Total Time | Iterations | Avg per Iteration |
68+
|---|---:|---:|---:|
69+
| Baseline basic JSON assembly | 5.57s | 1000 | 5.57ms |
70+
| FACET run (`basic_prompt.facet`) | 40.30s | 200 | 201.50ms |
71+
| Baseline RAG-like JSON assembly | 2.51s | 500 | 5.02ms |
72+
| FACET run (`rag_pipeline.facet`) | 20.37s | 100 | 203.70ms |
73+
| FACET build (`basic_prompt.facet`) | 50.07s | 300 | 166.90ms |
74+
| FACET build (`rag_pipeline.facet`) | 49.46s | 300 | 164.87ms |
75+
76+
## 5. Interpretation
77+
78+
- Naive JSON assembly is faster, but it does not provide contract/type/policy guarantees.
79+
- FACET overhead is the cost of deterministic validation and bounded execution semantics.
80+
- In exchange, failures are explicit (`F*`), and system behavior is controlled rather than best-effort.
81+
82+
## 6. Limits of This Benchmark
83+
84+
- Synthetic setup, single machine.
85+
- No provider network latency included.
86+
- Not a throughput benchmark under concurrent load.
87+
88+
For release gating, use CI matrix + workload-specific perf tests.
89+
90+
## 7. Related
91+
92+
- [Performance Guide](10-performance.html)
93+
- [Execution Model](15-execution-model.html)
94+
- [Production Scenario](16-production-scenario.html)

‎docs/18-integration-guide.md‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
---
2+
permalink: /18-integration-guide.html
3+
title: Integration Guide
4+
---
5+
6+
# 18. Integration Guide (Brownfield)
7+
**Reading Time:** 10-12 minutes | **Difficulty:** Intermediate
8+
**Compiler Version:** 0.1.2 | **Spec Version:** 2.1.3
9+
10+
This guide focuses on integrating FACET into existing systems without a full rewrite.
11+
12+
## 1. Minimal Insertion Points
13+
14+
### Pattern A: Sidecar Compiler (lowest risk)
15+
16+
- Keep current application flow.
17+
- Replace prompt assembly function with `fct run` call.
18+
- Continue using your existing provider client and response validator.
19+
20+
### Pattern B: CI Contract Gate
21+
22+
- Keep runtime unchanged initially.
23+
- Add `fct build` in CI for all `.facet` contracts.
24+
- Block merges on contract/type/policy failures.
25+
26+
### Pattern C: Runtime Guard Rollout
27+
28+
- Start in `core` profile for static validation.
29+
- Move selected flows to `hypervisor` with policy/guard events.
30+
- Enable strict fail-closed behavior for high-risk operations.
31+
32+
## 2. Migration Plan (Incremental)
33+
34+
1. Inventory 3 high-value prompt flows.
35+
2. Port them to `.facet` contracts.
36+
3. Add CI gate: `fct build --input <file>`.
37+
4. Add runtime call path: `fct run` -> provider.
38+
5. Enforce schema validation on provider response.
39+
6. Add bounded retry/fallback policy.
40+
41+
## 3. Cost of Adoption
42+
43+
### Engineering Cost
44+
45+
- Initial contract modeling effort.
46+
- Host-side response validation and retry policy wiring.
47+
- Policy/guard setup for Hypervisor flows.
48+
49+
### Runtime Cost
50+
51+
- Additional per-request compile/run overhead.
52+
- Artifact/telemetry storage if guard provenance is enabled.
53+
54+
### Risk Reduction Value
55+
56+
- Contract errors move left (compile-time).
57+
- Runtime failures become explicit and classifiable.
58+
- Guarded operations become auditable.
59+
60+
## 4. Compatibility Strategy
61+
62+
- Keep provider SDKs unchanged.
63+
- Keep business logic unchanged.
64+
- Replace only request construction and policy gating layer first.
65+
66+
## 5. Anti-Patterns
67+
68+
- Treating FACET as response truth validator by itself.
69+
- Skipping host response schema checks.
70+
- Enabling retries without strict bounds.
71+
- Using nondeterministic host defaults for policy decisions.
72+
73+
## 6. Recommended “First Week” Checklist
74+
75+
- [ ] Add `fct build` in CI on changed contracts.
76+
- [ ] Add one production flow via sidecar integration.
77+
- [ ] Log `document_hash` and `policy_hash` per request.
78+
- [ ] Add deterministic retry/fallback policy.
79+
- [ ] Validate post-model response schema in host code.
80+
81+
## 7. Related
82+
83+
- [Execution Model](15-execution-model.html)
84+
- [Production Scenario](16-production-scenario.html)
85+
- [Mini Benchmark Report](17-benchmark-report.html)

0 commit comments

Comments
 (0)