Add edgeimpulseocr element for text recognition#56
Open
ferjm wants to merge 1 commit into
Open
Conversation
Add a new `edgeimpulseocr` GStreamer element that reads text from video
frames (optical character recognition) and attaches each recognized line to
the buffer as inference metadata, bringing the plugin to seven elements.
For every line of text found, the element attaches a
`VideoRegionOfInterestMeta` — so downstream elements such as
`edgeimpulseoverlay` render the text like any other detection — and,
optionally, posts an `ocr` message on the bus.
Highlights:
- Recognition runs on a background worker thread, fully decoupled from the
streaming thread, so a slow model never stalls the pipeline: frames pass
through immediately and results attach to a slightly later frame.
- Pluggable backend design via the `OcrBackend` trait:
- `ocrs` (default): pure-Rust text detection and recognition using the ocrs
crate with rten models embedded in the plugin, so no external files are
required. Gated behind the default-on `ocrs` cargo feature.
- `edge-impulse` (planned): will decode text from an upstream Edge Impulse
recognition model; currently a no-op placeholder.
- Configurable properties: `backend`, `detection-model`, `recognition-model`,
`min-confidence`, `max-text-length`, `post-message`, and `interval`
(run recognition on one frame out of every N).
- `ocr` bus message carrying the recognized text, confidence, full-frame
bounding box, and the source-frame PTS (ms).
- Embedded detection/recognition models are verified against a pinned
SHA-256 before being vendored into the build.
Also adds:
- docs/edgeimpulseocr.md plus README updates (element table, architecture
diagram, and metadata reference).
- examples/ocr_inference.rs end-to-end example and
examples/download-ocr-models.sh helper.
- tests/ocr.rs integration tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a new
edgeimpulseocrGStreamer element that reads text from video frames (optical character recognition) and attaches each recognized line to the buffer as inference metadata, bringing the plugin to seven elements.For every line of text found, the element attaches a
VideoRegionOfInterestMeta— so downstream elements such asedgeimpulseoverlayrender the text like any other detection — and, optionally, posts anocrmessage on the bus.Highlights:
OcrBackendtrait:ocrs(default): pure-Rust text detection and recognition using the ocrs crate with rten models embedded in the plugin, so no external files are required. Gated behind the default-onocrscargo feature.edge-impulse(planned): will decode text from an upstream Edge Impulse recognition model; currently a no-op placeholder.backend,detection-model,recognition-model,min-confidence,max-text-length,post-message, andinterval(run recognition on one frame out of every N).ocrbus message carrying the recognized text, confidence, full-frame bounding box, and the source-frame PTS (ms).Also adds: