Skip to content

Commit b40b3a1

Browse files
Johnclaude
John
and
claude
committed
Prepare for OpenIPC publication: drop MVP framing, add CLAUDE.md
Doc cleanup ahead of pushing under github.com/OpenIPC: - README: drop "0.1.0 / alpha" and "MVP slice" intro; replace with a Beta-honest "What's inside" section that lists actual coverage (Device / Network / Auth / WS-Discovery / Media v10 / Media2 / Events / PTZ / Imaging, with the read/write split). CI badge now points at the GitHub Actions workflow. - pyproject: version 0.1.0 → 0.2.0, Development Status 3 - Alpha → 4 - Beta, add Programming Language classifiers for 3.10–3.13, add [project.urls] block with the OpenIPC/onvif-tt GitHub URLs, authors attribute to "OpenIPC contributors", add openipc keyword. - src/onvif_tt/__init__.py: __version__ = "0.2.0". - LICENSE: copyright line attributes both OpenIPC and contributors. New docs: - CLAUDE.md — cold-start brief for any AI agent collaborating on the repo. Conventions that bit us during development are documented inline (look up real IDs before @register, python-onvif-zeep takes positional args, void responses are None, PullPoint needs two bindings, LOCAL-* prefix, --allow-writes, xfail_on, corpus determinism). Also documents the repo layout and a 6-step recipe for adding a test. - CONTRIBUTING.md — quick setup + PR expectations + bug-report template for vendor non-conformances. - docs/adding-a-test.md — focused walk-through that the original plan named but I never wrote. Used by CONTRIBUTING.md and CLAUDE.md. Local verification before publication: pytest tests/ → 9/9 PASS onvif-tt corpus stats → 1121 entries list --implemented validation → 48/48 catalog IDs present onvif-tt run … (full) → 43 passed / 15 skipped / 8 xfailed / 0 failed / exit 0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 27e6978 commit b40b3a1

7 files changed

Lines changed: 474 additions & 23 deletions

File tree

‎CLAUDE.md‎

Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
# Working on `onvif-tt` with Claude (or any AI agent)
2+
3+
This file is a cold-start brief for AI agents collaborating on this
4+
repo. It explains what the project is, how its pieces fit together,
5+
and the conventions that — if you get them wrong — produce subtle
6+
bugs that look right but aren't.
7+
8+
## What this is
9+
10+
`onvif-tt` is a Linux-native, headless ONVIF conformance test tool —
11+
an open-source alternative to the closed (members-only, Windows-only)
12+
ONVIF Device Test Tool. It:
13+
14+
1. Parses the public ONVIF Test Specification HTML corpus into a
15+
structured catalog (`corpus/parsed.json`, 1,121 test cases).
16+
2. Lets contributors register Python implementations of individual
17+
spec IDs via `@register("DEVICE-1-1-2")` decorators.
18+
3. Runs the registered tests against a Device Under Test via pytest,
19+
emitting JUnit XML for CI and a structured JSON report whose schema
20+
is published in `docs/schemas/`.
21+
22+
## Repo layout
23+
24+
```
25+
onvif-tt/
26+
├── corpus/
27+
│ ├── html/ # 22 ONVIF Test Specification files (verbatim, redistributable)
28+
│ └── parsed.json # generated cache; deterministic-parse-tested
29+
├── src/onvif_tt/
30+
│ ├── specs/ # lxml DocBook parser + dataclasses
31+
│ ├── registry.py # @register decorator + REGISTRY dict + xfail_on matcher
32+
│ ├── runtime/
33+
│ │ ├── dut.py # ONVIFCamera wrapper, lazy service binding,
34+
│ │ │ # PullPointHandle, NotifyHandle, SOAP trace
35+
│ │ ├── features.py # cached GetServices + GetDeviceInformation
36+
│ │ ├── discovery.py # multicast WS-Discovery Probe helper
37+
│ │ └── soap_trace.py # zeep plugin that captures envelopes
38+
│ ├── runner/
39+
│ │ ├── dispatch.py # pytest parametrise — one node per registry entry
40+
│ │ └── plugin.py # CLI options + JSON reporter (xdist-safe)
41+
│ ├── cases/ # one .py per profile area
42+
│ │ ├── base.py # Device + capabilities + GetServices flavours
43+
│ │ ├── auth.py # LOCAL-AUTH-* (no AUTH-* in catalog)
44+
│ │ ├── discovery.py # WS-Discovery
45+
│ │ ├── ipconfig.py # LOCAL-NETWORK-* read-only (catalog IPCONFIG-* are writes)
46+
│ │ ├── media.py # Media v10 + Media2 (adaptive) + consistency tests
47+
│ │ ├── event.py # GetEventProperties + PullPoint + Basic Notification
48+
│ │ ├── ptz.py # GetNodes + Move/Stop write ops
49+
│ │ └── imaging.py # GetImagingSettings + Move/Stop write ops
50+
│ └── cli.py # argparse: `onvif-tt list|show|corpus|run`
51+
├── tests/ # Tests for the tool itself (parser + registry)
52+
├── docs/
53+
│ ├── ai-readme.md # How to drive the tool from an LLM tool-use loop
54+
│ ├── adding-a-test.md # Contributor guide
55+
│ └── schemas/ # JSON Schema for results.json + corpus/parsed.json
56+
└── scripts/ # Shell helpers (subscription-cleanup verifier, …)
57+
```
58+
59+
## Conventions that matter
60+
61+
These are mistakes that bit us during development. If you reproduce
62+
them, the tests will pass in spurious ways.
63+
64+
### 1. Look up real test IDs via the catalog before `@register`-ing
65+
66+
The spec ID `DEVICE-1-1-2` is "ALL CAPABILITIES", not "GetDeviceInformation"
67+
(that's `DEVICE-3-1-9`). The names are not as obvious as they look.
68+
69+
```bash
70+
onvif-tt show DEVICE-3-1-9 # prints the actual procedure
71+
onvif-tt list --id-glob "MEDIA2-2-*" --missing # what's left to implement
72+
```
73+
74+
If you invent an ID that isn't in the catalog, the CI check
75+
"every registered spec ID exists in the catalog" will fail.
76+
77+
### 2. `python-onvif-zeep` takes **positional** WSDL parameters
78+
79+
```python
80+
dut.devicemgmt.GetServices(False) # ✅
81+
dut.devicemgmt.GetServices(IncludeCapability=False) # ❌ raises
82+
dut.devicemgmt.GetCapabilities("All") # ✅
83+
dut.devicemgmt.GetCapabilities(Category="All") # ❌
84+
```
85+
86+
For operations with structured arguments, build with `create_type`:
87+
88+
```python
89+
req = dut.media.create_type("GetStreamUri")
90+
req.StreamSetup = {"Stream": "RTP-Unicast", "Transport": {"Protocol": "RTSP"}}
91+
req.ProfileToken = profile.token
92+
resp = dut.media.GetStreamUri(req)
93+
```
94+
95+
### 3. ONVIF "void" responses come back as Python `None`
96+
97+
Operations like `Move`, `Stop`, `AbsoluteMove`, `RelativeMove`,
98+
`Unsubscribe`, `SetSynchronizationPoint`, … have empty response bodies
99+
in the WSDL — zeep returns `None`. Never assert `resp is not None` on
100+
these; the assertion is "no SOAP Fault was raised".
101+
102+
```python
103+
dut.imaging.Move(req) # ✅ just the call
104+
resp = dut.imaging.Move(req)
105+
assert resp is not None # ❌ will FAIL spuriously on a perfectly conformant device
106+
```
107+
108+
### 4. PullPoint subscriptions need **two** WSDL bindings
109+
110+
The PullPoint subscription URL is queried via different bindings for
111+
different operations:
112+
113+
* `PullPointSubscriptionBinding` → `PullMessages`, `SetSynchronizationPoint`
114+
* `SubscriptionManagerBinding` → `Renew`, `Unsubscribe`
115+
116+
`PullPointHandle` (and `NotifyHandle` for Basic Notification) in
117+
`runtime/dut.py` already wraps both. Use them; don't roll your own.
118+
119+
```python
120+
with dut.create_pullpoint("PT60S") as pp:
121+
pp.pull_messages(timeout="PT3S", limit=5)
122+
# auto-unsubscribe on __exit__
123+
```
124+
125+
### 5. `LOCAL-*` prefix for tool-author IDs
126+
127+
When the spec catalog has no clean counterpart for the conformance
128+
intent you want to test (e.g. a basic "GetStreamUri returns a valid
129+
RTSP URL" smoke), use a `LOCAL-*` ID:
130+
131+
```python
132+
@register("LOCAL-MEDIA-S-STREAM-URI", profiles={"S"}, mandatory=True,
133+
requires_services={"devicemgmt", "media"},
134+
tags={"local"})
135+
```
136+
137+
The CI check skips `LOCAL-*` IDs when validating "registered ID is in
138+
catalog". Don't put real-looking IDs there; if it's in `corpus/parsed.json`,
139+
use the real ID.
140+
141+
### 6. `--allow-writes` for hardware-actuating tests
142+
143+
Anything that moves a focus motor, pans a head, mutates network
144+
config, sets the system clock, or reboots needs `requires_writes=True`:
145+
146+
```python
147+
@register("PTZ-3-1-1", ..., requires_writes=True)
148+
def test_ptz_absolute_move(dut, spec):
149+
...
150+
```
151+
152+
Without `--allow-writes` on the run command, these skip with a clear
153+
reason. Default runs are read-only.
154+
155+
### 7. `xfail_on` for known-buggy firmware, not for "test is hard"
156+
157+
If a vendor's firmware reliably violates the spec, document it inline:
158+
159+
```python
160+
@register("DEVICE-1-1-9", ...,
161+
xfail_on=[{
162+
"Manufacturer": "H264",
163+
"reason": "Xiongmai stock closes the TCP connection on "
164+
"invalid GetCapabilities Category instead of "
165+
"returning a SOAP 1.2 Fault.",
166+
}])
167+
def test_soap_fault_on_invalid_capability(dut, spec): ...
168+
```
169+
170+
Matchers compare against `GetDeviceInformation` fields (literal or
171+
callable). Multiple matchers OR together. If the bug ever gets fixed,
172+
the test becomes `xpassed` and a Python warning surfaces it.
173+
174+
**Do not** use `xfail_on` to silence flaky tests. If a test is flaky,
175+
fix the test.
176+
177+
### 8. Don't break corpus determinism
178+
179+
`corpus/parsed.json` is committed AND regenerable. A CI test asserts
180+
that re-parsing produces byte-identical output. If you touch the
181+
parser, run:
182+
183+
```bash
184+
onvif-tt corpus refresh
185+
pytest tests/test_parser.py
186+
```
187+
188+
Both must succeed.
189+
190+
## Adding a new test — quick recipe
191+
192+
```bash
193+
# 1. Find the right spec ID (don't invent — look it up):
194+
onvif-tt list --id-glob "EVENT-3-*" --missing --format json | jq
195+
196+
# 2. Read the spec procedure:
197+
onvif-tt show EVENT-3-1-32
198+
199+
# 3. Add to the appropriate cases/*.py file. Mirror the spec's
200+
# Pre-Requisite as requires_services, mark mandatory/optional honestly.
201+
202+
# 4. Verify against the reference camera:
203+
onvif-tt run --target 10.216.128.71:8899 --user admin --password admin \
204+
--id-glob "EVENT-3-1-32" --json-report /tmp/r.json
205+
206+
# 5. Tool's own unit tests still pass:
207+
pytest tests/
208+
```
209+
210+
Add `xfail_on` only if you observe a real device behaviour you want
211+
the catalog to remember. Don't speculate.
212+
213+
## When to ask before acting
214+
215+
- **Anything that mutates the DUT** (writes, reboot, config change) —
216+
only with explicit `--allow-writes` user intent.
217+
- **Pushing branches / opening PRs** — CI runs on push; ask before
218+
triggering it on the live repo.
219+
- **Adding dependencies to `pyproject.toml`** — every dep widens the
220+
install surface; prefer stdlib or the existing zeep/lxml/pytest
221+
stack.
222+
223+
## Pointers
224+
225+
- Detailed AI tool-use shapes: `docs/ai-readme.md`
226+
- JSON Schemas: `docs/schemas/`
227+
- Contributor docs: `CONTRIBUTING.md`, `docs/adding-a-test.md`
228+
- Project home: <https://github.com/OpenIPC/onvif-tt>

‎CONTRIBUTING.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Contributing to onvif-tt
2+
3+
Thanks for considering a contribution! Here's the quick orientation.
4+
5+
## Quick setup
6+
7+
```bash
8+
git clone https://github.com/OpenIPC/onvif-tt
9+
cd onvif-tt
10+
python -m venv .venv && . .venv/bin/activate
11+
pip install -e .
12+
pytest tests/ # parser + registry unit tests
13+
onvif-tt corpus stats # catalog sanity
14+
```
15+
16+
You'll need network access to a real ONVIF device (or a Happytime
17+
virtual ONVIF server) to exercise the `run` command. Read-only tests
18+
work against any compliant device.
19+
20+
## Adding a test
21+
22+
See [`docs/adding-a-test.md`](docs/adding-a-test.md) for a focused
23+
walk-through and [`CLAUDE.md`](CLAUDE.md) for the conventions —
24+
especially around calling-convention quirks, void responses, and the
25+
`LOCAL-*` prefix for IDs not in the public spec corpus.
26+
27+
## Pull request expectations
28+
29+
* CI must be green. `.github/workflows/ci.yml` runs the parser and
30+
registry unit tests across Python 3.10–3.13 plus catalog sanity
31+
checks.
32+
* If you change the parser, regenerate the cache:
33+
`onvif-tt corpus refresh` (the deterministic-parse test will catch
34+
drift).
35+
* New spec implementations should mark `mandatory` honestly per the
36+
ONVIF spec's requirement level, declare `requires_services`, and use
37+
`xfail_on` only when you have observed a real vendor non-conformance.
38+
* New `LOCAL-*` IDs are fine — they're how we cover gaps in the v20.12
39+
spec corpus.
40+
41+
## Reporting bugs / new device non-conformances
42+
43+
When a test surfaces a new vendor bug, include in the issue:
44+
45+
* The full `GetDeviceInformation` (`Manufacturer`, `Model`,
46+
`FirmwareVersion`, `SerialNumber`, `HardwareId`).
47+
* The failing test ID and `results.json` snippet (especially
48+
`last_request` and `last_response`).
49+
* What the spec calls for (link to the relevant `corpus/html/*.html`
50+
section).
51+
52+
If it's reproducible on a known firmware family, an `xfail_on` matcher
53+
plus the bug report is the highest-value contribution.
54+
55+
## Code style
56+
57+
* Python ≥ 3.10. Type hints encouraged, not mandatory.
58+
* Standard library + the existing `zeep` / `lxml` / `pytest` stack
59+
preferred over new deps.
60+
* Tests for the tool itself live under `tests/`; ONVIF test
61+
implementations live under `src/onvif_tt/cases/`.
62+
63+
## License
64+
65+
MIT. By contributing you agree your code is released under the same
66+
terms (see `LICENSE`).

‎LICENSE‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
MIT License
22

3-
Copyright (c) 2026 onvif-tt contributors
3+
Copyright (c) 2026 OpenIPC and onvif-tt contributors
44

55
Permission is hereby granted, free of charge, to any person obtaining a copy
66
of this software and associated documentation files (the "Software"), to deal

‎README.md‎

Lines changed: 35 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -2,31 +2,49 @@
22

33
> Open-source, headless, **CI- and AI-friendly** ONVIF conformance test tool.
44
5-
![ci](https://img.shields.io/badge/ci-github_actions-blue) ![license](https://img.shields.io/badge/license-MIT-green) ![tests](https://img.shields.io/badge/implementations-66-informational)
5+
[![ci](https://github.com/OpenIPC/onvif-tt/actions/workflows/ci.yml/badge.svg)](https://github.com/OpenIPC/onvif-tt/actions/workflows/ci.yml)
6+
![license](https://img.shields.io/badge/license-MIT-green)
7+
![python](https://img.shields.io/badge/python-3.10%E2%80%933.13-blue)
68

79
`onvif-tt` is a Linux-native, MIT-licensed alternative to the closed
810
ONVIF Device Test Tool. It parses the public ONVIF Test Specification
911
HTML corpus into a machine-readable catalog and executes registered
1012
test implementations against a Device Under Test, emitting both
1113
**JUnit XML** (CI) and **structured JSON** (LLM agents).
1214

13-
This is **0.1.0 / alpha**. Today:
14-
15-
* 22 ONVIF test specification files → **1,121 test cases** parsed and
16-
cached as JSON.
17-
* **66 test implementations** across Device, Network, Auth, WS-Discovery,
18-
Media v10, Media2 (incl. cross-endpoint consistency), Events
19-
(PullPoint + Basic Notification), PTZ (read + write), Imaging
20-
(read + write).
21-
* CLI: `list`, `show`, `corpus refresh|stats`, `run`.
22-
* `run` outputs JUnit XML + JSON + plain pytest stdout.
23-
* Device-fingerprint xfail surfaces known-buggy firmware without
24-
alarming CI; `xpassed` warning signals when a vendor fixes it.
25-
* `.github/workflows/ci.yml` runs parser/registry tests + catalog
26-
sanity on every push (no DUT needed). An optional integration job
15+
It's built and maintained inside [OpenIPC](https://openipc.org) to
16+
provide an automated conformance signal for the OpenIPC camera-side
17+
ONVIF library; the tool itself is vendor-neutral and works against any
18+
ONVIF-compliant device.
19+
20+
## What's inside
21+
22+
* 22 ONVIF Test Specification files → **1,121 test cases** parsed into
23+
`corpus/parsed.json` with a deterministic-parse guard.
24+
* **66 test implementations** spanning:
25+
* Device service (capabilities, GetServices, system commands, SOAP-fault
26+
handling)
27+
* Network read-only (interfaces, DNS, NTP, gateway, protocols)
28+
* Authentication (valid creds, wrong password, anonymous-rejected)
29+
* WS-Discovery (multicast Probe, ProbeMatch validation)
30+
* Media v10 + Media2 (profiles, encoders, stream URI, snapshot,
31+
end-to-end RTSP decode via `ffprobe`, cross-endpoint consistency)
32+
* Events — PullPoint and Basic Notification lifecycles, with a
33+
subscription tracker that auto-unsubscribes on teardown
34+
* PTZ (nodes, AbsoluteMove / RelativeMove / ContinuousMove / Stop)
35+
* Imaging (settings, options, MoveOptions, Status, Absolute /
36+
Relative / Continuous focus moves)
37+
* Read-only by default. Tests that actuate hardware (motors, recording,
38+
factory reset) opt in via `--allow-writes`.
39+
* Device-fingerprint **`xfail_on`** surfaces known-buggy firmware
40+
without alarming CI; an `xpassed` warning signals when a vendor
41+
fixes it.
42+
* `pytest-xdist` parallel execution measured at ~2.7× speedup vs
43+
sequential against a typical LAN camera.
44+
* `.github/workflows/ci.yml` runs parser + registry + catalog sanity
45+
on every push across Python 3.10–3.13. Optional integration job
2746
exercises onvif-tt against a Happytime virtual ONVIF device.
28-
* `--allow-writes` opt-in for tests that actuate hardware (focus motor,
29-
PTZ head, recording, factory reset). Default: read-only.
47+
* CLI: `list`, `show`, `corpus refresh|stats`, `run`.
3048

3149
## Why this exists
3250

0 commit comments

Comments
 (0)