Skip to content

Commit 52c4751

Browse files
InfoSecHackEric Conklin
InfoSecHack
and
Eric Conklin
authored
Add reviewer demo package (#78)
Co-authored-by: Eric Conklin <your-email@example.com>
1 parent 51fcef9 commit 52c4751

5 files changed

Lines changed: 379 additions & 0 deletions

File tree

‎docs/demo/README.md‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# IAMScope Reviewer Demo Package
2+
3+
## What This Demo Is
4+
5+
This demo package gives reviewers a short, bounded walkthrough of IAMScope’s differentiated value: evidence-grade IAM findings with explicit verdicts, required checks, blockers, `collection_context`, capability boundaries, replay, human labels, and an owner-confirmation trail.
6+
7+
The strongest current milestone is the frozen tag:
8+
9+
- `real-pilot-dev-001-final-calibrated-reviewed`
10+
11+
The main case study is:
12+
13+
- [`docs/case-studies/real-pilot-dev-001-human-review-summary.md`](../case-studies/real-pilot-dev-001-human-review-summary.md)
14+
15+
The capability boundary reference is:
16+
17+
- [`docs/reference/capability-honesty-matrix.md`](../reference/capability-honesty-matrix.md)
18+
19+
## What This Demo Proves In The Narrow Sense
20+
21+
The final calibrated real-pilot summary records:
22+
23+
- 18 findings.
24+
- 18 validated.
25+
- 15 `cross_account_trust` findings.
26+
- 3 `admin_reachability` findings.
27+
- 18 labeled findings.
28+
- 14 `valid_path` labels.
29+
- 3 `expected_benign` labels.
30+
- 1 `needs_more_evidence` label.
31+
- 5 `owner_confirmed` labels.
32+
- complete `collection_context`.
33+
- sanitized output hygiene clean.
34+
35+
This proves only a bounded workflow point: IAMScope can turn a collected IAM graph into reviewable, evidence-grade findings with explicit capability boundaries, human-review labels, and owner-confirmation metadata for the documented pilot. It does not prove broad correctness or safety.
36+
37+
## What This Demo Does Not Prove
38+
39+
No findings does not mean safe. Validated does not mean exploited. The demo evidence is bounded.
40+
41+
This package does not claim:
42+
43+
- production readiness.
44+
- exploitability proof.
45+
- full IAM safety.
46+
- full AWS authorization semantics.
47+
- complete IAM privilege-escalation coverage.
48+
- broad IAMScope correctness.
49+
- a composite score.
50+
- a pass/fail benchmark label.
51+
52+
## What To Open First
53+
54+
1. [`demo-narrative-one-pager.md`](demo-narrative-one-pager.md) — shortest public-facing story.
55+
2. [`iamscope-vs-pacu-pmapper.md`](iamscope-vs-pacu-pmapper.md) — positioning against Pacu and PMapper without disparaging either tool.
56+
3. [`recorded-demo-script.md`](recorded-demo-script.md) — 7-10 minute recording script.
57+
4. [`live-demo-runbook.md`](live-demo-runbook.md) — safe no-AWS and authorized-AWS demo modes.
58+
5. [`../case-studies/real-pilot-dev-001-human-review-summary.md`](../case-studies/real-pilot-dev-001-human-review-summary.md) — final calibrated real-pilot evidence.
59+
60+
## How To Use The Final Real-Pilot Case Study
61+
62+
Use the case study as the evidence anchor, not as a broad score. The useful review path is:
63+
64+
- confirm the final calibrated replay counts;
65+
- inspect how `cross_account_trust` and `admin_reachability` findings are separated;
66+
- explain how `collection_context` and non-claims keep the evidence bounded;
67+
- show how human labels and owner-confirmation add reviewer accountability without claiming exploitability.
68+
69+
## Related Demo Files
70+
71+
- [`recorded-demo-script.md`](recorded-demo-script.md)
72+
- [`live-demo-runbook.md`](live-demo-runbook.md)
73+
- [`iamscope-vs-pacu-pmapper.md`](iamscope-vs-pacu-pmapper.md)
74+
- [`demo-narrative-one-pager.md`](demo-narrative-one-pager.md)

‎docs/demo/demo-narrative-one-pager.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# IAMScope Demo Narrative
2+
3+
IAMScope is not trying to be Pacu.
4+
5+
IAMScope is not trying to be a full replacement for PMapper.
6+
7+
IAMScope is trying to answer a narrower reviewer question: what IAM paths can we support with evidence, what remains uncertain, and what should a human review first?
8+
9+
## The Short Version
10+
11+
Many IAM tools help you exploit, query, or graph an AWS environment. Those are useful jobs. IAMScope is focused on a different job: evidence-grade review.
12+
13+
In the current final calibrated real-pilot milestone, IAMScope produced:
14+
15+
- 18 findings.
16+
- 18 validated.
17+
- 15 `cross_account_trust`.
18+
- 3 `admin_reachability`.
19+
- 14 `valid_path`.
20+
- 3 `expected_benign`.
21+
- 1 `needs_more_evidence`.
22+
- 5 `owner_confirmed`.
23+
- complete `collection_context`.
24+
25+
The point is not that these numbers are a score. They are not. The point is that each finding can be discussed as a review artifact with checks, evidence boundaries, labels, and non-claims.
26+
27+
## Why This Is Different
28+
29+
IAMScope keeps the reviewer workflow visible:
30+
31+
- collect or replay the IAM graph;
32+
- run bounded reasoners;
33+
- emit findings with verdicts and required checks;
34+
- show blockers and uncertainty;
35+
- preserve `collection_context`;
36+
- map findings to human labels;
37+
- add owner-confirmation when available;
38+
- keep raw artifacts local unless sanitized.
39+
40+
This is why the demo uses phrases like evidence-grade, bounded, reviewable, and owner-confirmed instead of “exploited” or “safe.”
41+
42+
## The Lines We Do Not Cross
43+
44+
No findings does not mean safe.
45+
46+
Validated does not mean exploited.
47+
48+
The demo evidence is bounded.
49+
50+
This demo does not claim:
51+
52+
- production readiness.
53+
- exploitability proof.
54+
- full IAM safety.
55+
- full AWS authorization semantics.
56+
- complete IAM privilege-escalation coverage.
57+
- broad IAMScope correctness.
58+
- a composite score.
59+
- a pass/fail benchmark label.
60+
61+
## Reviewer Takeaway
62+
63+
IAMScope is for the moment after a graph looks scary but before a reviewer can act. It helps separate “the evidence supports this,” “this looks expected-benign,” “this needs more evidence,” and “this is outside current modeled capability.”

‎docs/demo/iamscope-vs-pacu-pmapper.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# IAMScope vs Pacu vs PMapper
2+
3+
Pacu and PMapper are useful tools with different goals. This comparison is positioning, not a ranking.
4+
5+
| Tool | Primary purpose | What it is good at | What IAMScope is not trying to replace | IAMScope difference |
6+
| --- | --- | --- | --- | --- |
7+
| Pacu | Offensive AWS exploitation / attack module framework. | Running AWS attack modules in authorized security testing, demonstrating exploitation workflows, and exploring attacker tradecraft. | IAMScope is not trying to replace Pacu as an exploitation framework or attack-module runner. | IAMScope does not attempt exploitation. It produces evidence-grade findings with verdicts, required checks, blockers, `collection_context`, capability boundaries, replay, human labels, and owner-confirmation trail. |
8+
| PMapper | IAM graph/query/local authorization simulation / privilege-escalation path mapping. | Building IAM relationship graphs, querying policies, local authorization reasoning, and mapping privilege-escalation paths. | IAMScope is not trying to be a full replacement for PMapper’s graph/query and simulation workflows. | IAMScope focuses on reviewer-facing findings: what is validated, blocked, inconclusive, expected-benign, unsupported, or needs more evidence. It emphasizes capability honesty and bounded non-claims. |
9+
| IAMScope | Evidence-grade IAM finding workflow. | Collecting or replaying IAM graph artifacts, running reasoners, emitting verdicts and required checks, preserving blockers and `collection_context`, supporting human labels, owner-confirmation, and sanitized review summaries. | IAMScope is not trying to replace offensive testing tools or general IAM graph/query tools. | IAMScope’s value is the review workflow: it says what the evidence supports, what remains uncertain, and what a human should review first. |
10+
11+
## Positioning Summary
12+
13+
- Pacu helps authorized testers exercise offensive AWS techniques.
14+
- PMapper helps users inspect IAM graph and authorization relationships.
15+
- IAMScope helps reviewers handle evidence, verdicts, blockers, capability boundaries, `collection_context`, human labels, and owner-confirmation without claiming exploitation.
16+
17+
## Non-Claims
18+
19+
IAMScope’s demo evidence does not claim:
20+
21+
- production readiness.
22+
- exploitability proof.
23+
- full IAM safety.
24+
- full AWS authorization semantics.
25+
- complete IAM privilege-escalation coverage.
26+
- broad IAMScope correctness.
27+
- a composite score.
28+
- a pass/fail benchmark label.
29+
30+
No findings does not mean safe. Validated does not mean exploited.

‎docs/demo/live-demo-runbook.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Live Demo Runbook
2+
3+
This runbook supports two demo modes: a no-AWS walkthrough and an explicitly authorized AWS walkthrough. The default and safest path is the no-AWS demo.
4+
5+
## Safety Boundary
6+
7+
- Do not run live AWS by default.
8+
- Do not run Terraform by default.
9+
- Do not show raw account IDs or raw IAM/STS ARNs on screen unless explicitly authorized.
10+
- Do not commit raw `scenario.json`, `findings.json`, labels, logs, or generated review artifacts.
11+
- Do not present any result as production readiness, exploitability proof, full IAM safety, a composite score, or a pass/fail benchmark label.
12+
13+
## Mode A — No-AWS Demo
14+
15+
Use this mode for recorded demos, public walkthroughs, and first-pass reviewer conversations.
16+
17+
Steps:
18+
19+
1. Open [`README.md`](README.md).
20+
2. Open [`demo-narrative-one-pager.md`](demo-narrative-one-pager.md).
21+
3. Open [`iamscope-vs-pacu-pmapper.md`](iamscope-vs-pacu-pmapper.md).
22+
4. Open [`../reference/capability-honesty-matrix.md`](../reference/capability-honesty-matrix.md).
23+
5. Open [`../case-studies/real-pilot-dev-001-human-review-summary.md`](../case-studies/real-pilot-dev-001-human-review-summary.md).
24+
6. If a sanitized review table is present locally, show only sanitized columns and avoid raw account IDs or raw IAM/STS ARNs.
25+
26+
Facts to state:
27+
28+
- 18 findings.
29+
- 18 validated.
30+
- 15 `cross_account_trust`.
31+
- 3 `admin_reachability`.
32+
- 14 `valid_path`.
33+
- 3 `expected_benign`.
34+
- 1 `needs_more_evidence`.
35+
- 5 `owner_confirmed`.
36+
- complete `collection_context`.
37+
- sanitized output hygiene clean.
38+
39+
Explain that this is evidence-grade review material, not live exploitation and not a safety certificate.
40+
41+
## Mode B — Authorized AWS Demo
42+
43+
Use this mode only in a sandbox, non-production account, or explicitly authorized environment controlled by the reviewer.
44+
45+
Preconditions:
46+
47+
- written or clearly recorded authorization;
48+
- scoped AWS profile;
49+
- expected account checked before collection;
50+
- no production admin access requested;
51+
- output path outside the repository, preferably under `/tmp`;
52+
- redaction plan agreed before screen sharing or publication.
53+
54+
Suggested flow:
55+
56+
1. Confirm the profile and expected account out of band with the environment owner.
57+
2. Run collection or replay only if authorized.
58+
3. Produce sanitized reviewer output under `/tmp` or another non-repo path.
59+
4. Run hygiene grep against generated demo output before showing or sharing it.
60+
5. Show findings, required checks, blockers, `collection_context`, labels, and owner-confirmation status.
61+
6. Keep raw account IDs, raw IAM/STS ARNs, raw findings, and logs off-screen unless explicitly authorized.
62+
7. Do not commit generated artifacts.
63+
64+
Stop conditions:
65+
66+
- profile/account does not match the expected environment;
67+
- collection emits unexpected raw artifacts into the repo tree;
68+
- raw account IDs or raw IAM/STS ARNs would be exposed without authorization;
69+
- reviewer asks to stop;
70+
- any command would mutate resources without explicit approval;
71+
- any result is being framed as exploitability proof, production readiness, or a pass/fail benchmark label.
72+
73+
## Hygiene Checks For Demo Outputs
74+
75+
Before sharing sanitized output, run equivalent local checks against the output directory:
76+
77+
- no raw 12-digit account IDs unless explicitly authorized;
78+
- no raw IAM/STS ARNs unless explicitly authorized;
79+
- no Terraform state, plan, lock file, or output JSON;
80+
- no raw AWS logs;
81+
- no generated artifacts committed to git.
82+
83+
## Demo Close
84+
85+
End with:
86+
87+
> “IAMScope is not an exploitation framework. It is an evidence-grade IAM finding workflow. No findings does not mean safe, and validated does not mean exploited.”

‎docs/demo/recorded-demo-script.md‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Recorded Demo Script
2+
3+
Target length: 7-10 minutes.
4+
5+
## 0:00-0:45 — Problem
6+
7+
IAM trust is messy. A reviewer does not only need a scary graph edge or an exploit module; they need to know what is supported by evidence, what is blocked, what is uncertain, and what should be reviewed first.
8+
9+
Say:
10+
11+
> “IAMScope is built for evidence-grade IAM review. It does not try to prove the account is safe, and it does not claim exploitability.”
12+
13+
## 0:45-1:45 — Positioning: Pacu vs PMapper vs IAMScope
14+
15+
Show [`iamscope-vs-pacu-pmapper.md`](iamscope-vs-pacu-pmapper.md).
16+
17+
Talk track:
18+
19+
- Pacu is useful as an offensive AWS exploitation and attack-module framework.
20+
- PMapper is useful for IAM graph/query work, local authorization simulation, and privilege-escalation path mapping.
21+
- IAMScope is narrower: it turns collection or replay into findings with verdicts, required checks, blockers, `collection_context`, capability honesty, human labels, and owner-confirmation trail.
22+
23+
Do not disparage Pacu or PMapper. They solve different jobs.
24+
25+
## 1:45-3:00 — Capability-Honesty Matrix
26+
27+
Open [`../reference/capability-honesty-matrix.md`](../reference/capability-honesty-matrix.md).
28+
29+
Call out:
30+
31+
- modeled areas;
32+
- unsupported or static-only areas;
33+
- places where IAMScope refuses to turn missing evidence into a stronger claim;
34+
- why no composite score and no pass/fail benchmark label are used.
35+
36+
Say:
37+
38+
> “The matrix is part of the product. It tells reviewers what not to believe.”
39+
40+
## 3:00-4:30 — Real-Pilot Case Study And Final Calibrated Replay
41+
42+
Open [`../case-studies/real-pilot-dev-001-human-review-summary.md`](../case-studies/real-pilot-dev-001-human-review-summary.md).
43+
44+
Show the final calibrated replay:
45+
46+
- 18 findings.
47+
- 18 validated.
48+
- 15 `cross_account_trust`.
49+
- 3 `admin_reachability`.
50+
- 18 labeled.
51+
- 14 `valid_path`.
52+
- 3 `expected_benign`.
53+
- 1 `needs_more_evidence`.
54+
- 5 `owner_confirmed`.
55+
- complete `collection_context`.
56+
57+
Say:
58+
59+
> “This is bounded real-pilot evidence. It is not production readiness, exploitability proof, or full IAM safety.”
60+
61+
## 4:30-6:00 — Cross-Account Trust Finding
62+
63+
Walk through one `cross_account_trust` row from the sanitized local review material if it is present locally. If the raw or sanitized table is not available, use the case-study summary instead.
64+
65+
Explain:
66+
67+
- the finding is a trust-structure review row;
68+
- wildcard-principal trust findings were repeatedly classified as `valid_path`;
69+
- expected-benign rows are still useful because they represent real structures owners may need to confirm;
70+
- owner-confirmed rows strengthen reviewability but do not create a full owner-confirmed truth set.
71+
72+
Avoid showing raw account IDs or raw IAM/STS ARNs unless the demo owner explicitly authorizes it.
73+
74+
## 6:00-7:30 — Admin Reachability Finding
75+
76+
Walk through one `admin_reachability` row from the sanitized local review material if present.
77+
78+
Explain the calibrated evidence chain:
79+
80+
- source role has `sts:AssumeRole` to `ProdDBAdminRole`;
81+
- the target trust is conditioned account-root trust narrowed by `aws:PrincipalArn`;
82+
- the admin witness is AWS-managed `AdministratorAccess`;
83+
- `clean_witness_check` is pass;
84+
- `source_has_assume_role`, `reaches_at_least_one_admin`, and `walk_terminated_within_depth_limit` are pass.
85+
86+
Say:
87+
88+
> “Validated does not mean exploited. It means IAMScope’s modeled checks for this finding passed under the current bounded evidence.”
89+
90+
## 7:30-8:30 — Collection Context And Non-Claims
91+
92+
Show that `collection_context` is complete:
93+
94+
- `graph_collection_complete`: true.
95+
- `has_collection_failures`: false.
96+
- `has_policy_parse_failures`: false.
97+
98+
Then read the non-claims:
99+
100+
- no production readiness.
101+
- no exploitability proof.
102+
- no full IAM safety.
103+
- no full AWS authorization semantics.
104+
- no complete IAM privilege-escalation coverage.
105+
- no composite score.
106+
- no pass/fail benchmark label.
107+
108+
## 8:30-9:30 — Owner-Confirmation Layer
109+
110+
Explain why owner-confirmation matters:
111+
112+
- labels are human review, not an automatic truth oracle;
113+
- five priority trust findings were owner-confirmed;
114+
- owner confirmation is bounded to those findings only;
115+
- this creates a review trail without claiming broad IAMScope correctness.
116+
117+
## 9:30-10:00 — Close
118+
119+
Close with:
120+
121+
> “IAMScope does not prove the account is safe. It gives a reviewer evidence they can act on.”
122+
123+
Then repeat:
124+
125+
> “No findings does not mean safe. Validated does not mean exploited. The demo evidence is bounded.”

0 commit comments

Comments
 (0)