Skip to content

Add edgeimpulseocr element for text recognition#56

Open
ferjm wants to merge 1 commit into
mainfrom
ocr-element
Open

Add edgeimpulseocr element for text recognition#56
ferjm wants to merge 1 commit into
mainfrom
ocr-element

Conversation

@ferjm

@ferjm ferjm commented Jul 10, 2026

Copy link
Copy Markdown
Member

Add a new edgeimpulseocr GStreamer element that reads text from video frames (optical character recognition) and attaches each recognized line to the buffer as inference metadata, bringing the plugin to seven elements.

For every line of text found, the element attaches a VideoRegionOfInterestMeta — so downstream elements such as edgeimpulseoverlay render the text like any other detection — and, optionally, posts an ocr message on the bus.

Highlights:

  • Recognition runs on a background worker thread, fully decoupled from the streaming thread, so a slow model never stalls the pipeline: frames pass through immediately and results attach to a slightly later frame.
  • Pluggable backend design via the OcrBackend trait:
    • ocrs (default): pure-Rust text detection and recognition using the ocrs crate with rten models embedded in the plugin, so no external files are required. Gated behind the default-on ocrs cargo feature.
    • edge-impulse (planned): will decode text from an upstream Edge Impulse recognition model; currently a no-op placeholder.
  • Configurable properties: backend, detection-model, recognition-model, min-confidence, max-text-length, post-message, and interval (run recognition on one frame out of every N).
  • ocr bus message carrying the recognized text, confidence, full-frame bounding box, and the source-frame PTS (ms).
  • Embedded detection/recognition models are verified against a pinned SHA-256 before being vendored into the build.

Also adds:

  • docs/edgeimpulseocr.md plus README updates (element table, architecture diagram, and metadata reference).
  • examples/ocr_inference.rs end-to-end example and examples/download-ocr-models.sh helper.
  • tests/ocr.rs integration tests.

Add a new `edgeimpulseocr` GStreamer element that reads text from video
frames (optical character recognition) and attaches each recognized line to
the buffer as inference metadata, bringing the plugin to seven elements.

For every line of text found, the element attaches a
`VideoRegionOfInterestMeta` — so downstream elements such as
`edgeimpulseoverlay` render the text like any other detection — and,
optionally, posts an `ocr` message on the bus.

Highlights:
- Recognition runs on a background worker thread, fully decoupled from the
  streaming thread, so a slow model never stalls the pipeline: frames pass
  through immediately and results attach to a slightly later frame.
- Pluggable backend design via the `OcrBackend` trait:
  - `ocrs` (default): pure-Rust text detection and recognition using the ocrs
    crate with rten models embedded in the plugin, so no external files are
    required. Gated behind the default-on `ocrs` cargo feature.
  - `edge-impulse` (planned): will decode text from an upstream Edge Impulse
    recognition model; currently a no-op placeholder.
- Configurable properties: `backend`, `detection-model`, `recognition-model`,
  `min-confidence`, `max-text-length`, `post-message`, and `interval`
  (run recognition on one frame out of every N).
- `ocr` bus message carrying the recognized text, confidence, full-frame
  bounding box, and the source-frame PTS (ms).
- Embedded detection/recognition models are verified against a pinned
  SHA-256 before being vendored into the build.

Also adds:
- docs/edgeimpulseocr.md plus README updates (element table, architecture
  diagram, and metadata reference).
- examples/ocr_inference.rs end-to-end example and
  examples/download-ocr-models.sh helper.
- tests/ocr.rs integration tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant