Merge pull request #101 from ezmsg-org/dev

Bulk update dev -> main for next release

Author: cboulay

.github/workflows/python-publish-ezmsg-sigproc.yml

@@ -17,8 +17,8 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
-      - name: Install uv
-        uses: astral-sh/setup-uv@v2
+      - name: Install the latest version of uv
+        uses: astral-sh/setup-uv@v6
 
       - name: Build Package
         run: uv build

.github/workflows/python-tests.yml

@@ -13,7 +13,7 @@ jobs:
   build:
     strategy:
       matrix:
-        python-version: ["3.10", "3.11", "3.12"]
+        python-version: ["3.10.15", "3.11", "3.12", "3.13"]
         os:
           - "ubuntu-latest"
           - "windows-latest"
@@ -23,17 +23,11 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
-      - name: Install uv
-        uses: astral-sh/setup-uv@v2
-        with:
-          enable-cache: true
-          cache-dependency-glob: "uv.lock"
-
-      - name: Set up Python ${{ matrix.python-version }}
-        run: uv python install ${{ matrix.python-version }}
+      - name: Install the latest version of uv
+        uses: astral-sh/setup-uv@v6
 
       - name: Install the project
-        run: uv sync --all-extras --dev
+        run: uv sync --python ${{ matrix.python-version }}
 
       - name: Lint
         run:

.gitignore

@@ -142,4 +142,5 @@ cython_debug/
 # JetBrains
 .idea/
 
-src/ezmsg/sigproc/__version__.py
+src/ezmsg/sigproc/__version__.py
+uv.lock

README.md

@@ -1,33 +1,48 @@
 # ezmsg.sigproc
 
-Timeseries signal processing implementations for ezmsg
+## Overview
 
-## Dependencies
-
-* `ezmsg`
-* `numpy`
-* `scipy`
-* `pywavelets`
+ezmsg-sigproc offers timeseries signal‑processing primitives built atop the ezmsg message‑passing framework. Core dependencies include ezmsg, numpy, scipy, pywavelets, and sparse; the project itself is managed through hatchling and uses VCS hooks to populate __version__.py.
 
 ## Installation
 
-### Release
-
 Install the latest release from pypi with: `pip install ezmsg-sigproc` (or `uv add ...` or `poetry add ...`).
 
-### Development Version
+You can install pre-release versions directly from GitHub:
 
-You can add the development version of `ezmsg-sigproc` to your project's dependencies in one of several ways.
+* Using `pip`: `pip install git+https://github.com/ezmsg-org/ezmsg-sigproc.git@dev`
+* Using `uv`: `uv add git+https://github.com/ezmsg-org/ezmsg-sigproc --branch dev`
+* Using `poetry`: `poetry add "git+https://github.com/ezmsg-org/ezmsg-sigproc.git@dev"`
 
-You can clone it and add its path to your project dependencies. You may wish to do this if you intend to edit `ezmsg-sigproc`. If so, please refer to the [Developers](#developers) section below.
+> See the [Development](#development) section below for installing with the intention of developing.
 
-You can also add it directly from GitHub:
+## Source layout & key modules
+* All source resides under src/ezmsg/sigproc, which contains a suite of processors (for example, filter.py, spectrogram.py, spectrum.py, sampler.py) and math and util subpackages.
+* The framework’s backbone is base.py, defining standard protocols—Processor, Producer, Consumer, and Transformer—that enable both stateless and stateful processing chains.
+* Filtering is implemented in filter.py, providing settings dataclasses and a stateful transformer that applies supplied coefficients to incoming data.
+* Spectral analysis uses a composite spectrogram transformer chaining windowing, spectrum computation, and axis adjustments.
 
-* Using `pip`: `pip install git+https://github.com/ezmsg-org/ezmsg-sigproc.git@dev`
-* Using `poetry`: `poetry add "git+https://github.com/ezmsg-org/ezmsg-sigproc.git@dev"`
-* Using `uv`: `uv add git+https://github.com/ezmsg-org/ezmsg-sigproc --branch dev`
+## Operating styles: Standalone processors vs. ezmsg pipelines
+While each processor is designed to be assembled into an ezmsg pipeline, the components are also well‑suited for offline, ad‑hoc analysis. You can instantiate processors directly in scripts or notebooks for quick prototyping or to validate results from other code. The companion Unit wrappers, however, are meant for assembling processors into a full ezmsg pipeline.
+
+A fully defined ezmsg pipeline shines in online streaming scenarios where message routing, scheduling, and latency handling are crucial. Nevertheless, you can run the same pipeline offline—say, within a Jupyter notebook—if your analysis benefits from ezmsg’s structured execution model. Deciding between a standalone processor and a full pipeline comes down to the trade‑off between simplicity and the operational overhead of the pipeline:
+
+* Standalone processors: Low overhead, ideal for one‑off or exploratory offline tasks.
+* Pipeline + Unit wrappers: Additional setup cost but bring concurrency, standardized interfaces, and automatic message flow—useful when your offline experiment mirrors a live system or when you require fine‑grained pipeline behavior.
+
+## Documentation & tests
+* `docs/ProcessorsBase.md` details the processor hierarchy and generic type patterns, providing a solid foundation for custom components.
+* Unit tests (e.g., `tests/unit/test_sampler.py`) offer concrete examples of usage, showcasing sampler generation, windowing, and message handling.
+
+## Where to learn next
+* Study docs/ProcessorsBase.md to master the processor architecture.
+* Explore unit tests for hands‑on examples of composing processors and Units.
+* Review the ezmsg framework in pyproject.toml to understand the surrounding ecosystem.
+* Experiment with the code—try running processors standalone and then integrate them into a small pipeline to observe the trade‑offs firsthand.
+
+This approach equips newcomers to choose the right level of abstraction—raw processor, Unit wrapper, or full pipeline—based on the demands of their analysis or streaming application.
 
-## Developers
+## Development
 
 We use [`uv`](https://docs.astral.sh/uv/getting-started/installation/) for development. It is not strictly required, but if you intend to contribute to ezmsg-sigproc then using `uv` will lead to the smoothest collaboration.
 
@@ -36,4 +51,5 @@ We use [`uv`](https://docs.astral.sh/uv/getting-started/installation/) for devel
 3. Open a terminal and `cd` to the cloned folder.
 4. `uv sync` to create a .venv and install dependencies.
 5. `uv run pre-commit install` to install pre-commit hooks to do linting and formatting.
-6. After editing code and making commits, Run the test suite before making a PR: `uv run pytest tests`
+6. Run the test suite before finalizing your edits: `uv run pytest tests`
+7. Make a PR against the `dev` branch of the main repo.

pyproject.toml

@@ -20,9 +20,18 @@ dependencies = [
   "sparse>=0.15.4",
 ]
 
-[project.optional-dependencies]
+[dependency-groups]
+dev = [
+  "typer>=0.12.5",
+  "pre-commit>=4.2.0",
+  "jupyter>=1.1.1",
+  {include-group = "lint"},
+  {include-group = "test"},
+]
+lint = [
+  "ruff"
+]
 test = [
-  "flake8>=7.1.1",
   "frozendict>=2.4.4",
   "pytest-asyncio>=0.24.0",
   "pytest-cov>=5.0.0",
@@ -42,9 +51,6 @@ version-file = "src/ezmsg/sigproc/__version__.py"
 [tool.hatch.build.targets.wheel]
 packages = ["src/ezmsg"]
 
-[tool.uv]
-dev-dependencies = ["pre-commit>=3.8.0", "ruff>=0.6.7"]
-
 [tool.pytest.ini_options]
 norecursedirs = "tests/helpers"
 addopts = "-p no:warnings"

src/ezmsg/sigproc/filter.py

@@ -32,17 +32,19 @@ class FilterCoefficients:
 
 
 def _normalize_coefs(
-    coefs: FilterCoefficients | tuple[npt.NDArray, npt.NDArray] | npt.NDArray,
-) -> tuple[str, tuple[npt.NDArray, ...]]:
+    coefs: FilterCoefficients | tuple[npt.NDArray, npt.NDArray] | npt.NDArray | None,
+) -> tuple[str, tuple[npt.NDArray, ...] | None]:
     coef_type = "ba"
     if coefs is not None:
         # scipy.signal functions called with first arg `*coefs`.
         # Make sure we have a tuple of coefficients.
-        if isinstance(coefs, npt.NDArray):
+        if isinstance(coefs, np.ndarray):
             coef_type = "sos"
             coefs = (coefs,)  # sos funcs just want a single ndarray.
         elif isinstance(coefs, FilterCoefficients):
-            coefs = (FilterCoefficients.b, FilterCoefficients.a)
+            coefs = (coefs.b, coefs.a)
+        elif not isinstance(coefs, tuple):
+            coefs = (coefs,)
     return coef_type, coefs
 
 
@@ -91,16 +93,20 @@ def _reset_state(self, message: AxisArray) -> None:
         axis = message.dims[0] if self.settings.axis is None else self.settings.axis
         axis_idx = message.get_axis_idx(axis)
         n_tail = message.data.ndim - axis_idx - 1
-        coefs = (
-            (self.settings.coefs,)
-            if self.settings.coefs is not None
-            and not isinstance(self.settings.coefs, tuple)
-            else self.settings.coefs
-        )
-        zi_func = {"ba": scipy.signal.lfilter_zi, "sos": scipy.signal.sosfilt_zi}[
-            self.settings.coef_type
-        ]
-        zi = zi_func(*coefs)
+        _, coefs = _normalize_coefs(self.settings.coefs)
+
+        if self.settings.coef_type == "ba":
+            b, a = coefs
+            if len(a) == 1 or np.allclose(a[1:], 0):
+                # For FIR filters, use lfiltic with zero initial conditions
+                zi = scipy.signal.lfiltic(b, a, [])
+            else:
+                # For IIR filters...
+                zi = scipy.signal.lfilter_zi(b, a)
+        else:
+            # For second-order sections (SOS) filters, use sosfilt_zi
+            zi = scipy.signal.sosfilt_zi(*coefs)
+
         zi_expand = (None,) * axis_idx + (slice(None),) + (None,) * n_tail
         n_tile = (
             message.data.shape[:axis_idx] + (1,) + message.data.shape[axis_idx + 1 :]
@@ -166,12 +172,7 @@ def _process(self, message: AxisArray) -> AxisArray:
         if message.data.size > 0:
             axis = message.dims[0] if self.settings.axis is None else self.settings.axis
             axis_idx = message.get_axis_idx(axis)
-            coefs = (
-                (self.settings.coefs,)
-                if self.settings.coefs is not None
-                and not isinstance(self.settings.coefs, tuple)
-                else self.settings.coefs
-            )
+            _, coefs = _normalize_coefs(self.settings.coefs)
             filt_func = {"ba": scipy.signal.lfilter, "sos": scipy.signal.sosfilt}[
                 self.settings.coef_type
             ]

src/ezmsg/sigproc/gaussiansmoothing.py

@@ -0,0 +1,93 @@
+from typing import Callable
+import warnings
+
+import numpy as np
+
+from .filter import (
+    FilterBaseSettings,
+    BACoeffs,
+    FilterByDesignTransformer,
+    BaseFilterByDesignTransformerUnit,
+)
+
+
+class GaussianSmoothingSettings(FilterBaseSettings):
+    sigma: float | None = 1.0
+    """
+    sigma : float
+        Standard deviation of the Gaussian kernel.
+    """
+
+    width: int | None = 4
+    """
+    width : int
+        Number of standard deviations covered by the kernel window if kernel_size is not provided.
+    """
+
+    kernel_size: int | None = None
+    """
+    kernel_size : int | None
+        Length of the kernel in samples. If provided, overrides automatic calculation.
+    """
+
+
+def gaussian_smoothing_filter_design(
+    sigma: float = 1.0,
+    width: int = 4,
+    kernel_size: int | None = None,
+) -> BACoeffs | None:
+    # Parameter checks
+    if sigma <= 0:
+        raise ValueError(f"sigma must be positive. Received: {sigma}")
+
+    if width <= 0:
+        raise ValueError(f"width must be positive. Received: {width}")
+
+    if kernel_size is not None:
+        if kernel_size < 1:
+            raise ValueError(f"kernel_size must be >= 1. Received: {kernel_size}")
+    else:
+        kernel_size = int(2 * width * sigma + 1)
+
+    # Warn if kernel_size is smaller than recommended but don't fail
+    expected_kernel_size = int(2 * width * sigma + 1)
+    if kernel_size < expected_kernel_size:
+        ## TODO: Either add a warning or determine appropriate kernel size and raise an error
+        warnings.warn(
+            f"Provided kernel_size {kernel_size} is smaller than recommended "
+            f"size {expected_kernel_size} for sigma={sigma} and width={width}. "
+            "The kernel may be truncated."
+        )
+
+    from scipy.signal.windows import gaussian
+
+    b = gaussian(kernel_size, std=sigma)
+    b /= np.sum(b)  # Ensure normalization
+    a = np.array([1.0])
+
+    return b, a
+
+
+class GaussianSmoothingFilterTransformer(
+    FilterByDesignTransformer[GaussianSmoothingSettings, BACoeffs]
+):
+    def get_design_function(
+        self,
+    ) -> Callable[[float], BACoeffs]:
+        # Create a wrapper function that ignores fs parameter since gaussian smoothing doesn't need it
+        def design_wrapper(fs: float) -> BACoeffs:
+            return gaussian_smoothing_filter_design(
+                sigma=self.settings.sigma,
+                width=self.settings.width,
+                kernel_size=self.settings.kernel_size,
+            )
+
+        return design_wrapper
+
+
+class GaussianSmoothingFilter(
+    BaseFilterByDesignTransformerUnit[
+        GaussianSmoothingSettings, GaussianSmoothingFilterTransformer
+    ]
+):
+    SETTINGS = GaussianSmoothingSettings