diff --git a/README.md b/README.md index b80bdc35..a5ef3c3a 100644 --- a/README.md +++ b/README.md @@ -10,8 +10,9 @@ Signal processing primitives for the [ezmsg](https://www.ezmsg.org) message-pass * **Windowing** - Sliding windows and buffering utilities * **Math operations** - Arithmetic, log, abs, difference, and more * **Signal generation** - Synthetic signal generators +* More! Brows the API documentation for more details. -All modules use `AxisArray` as the primary data structure for passing signals between components. +All modules use [`AxisArray`](https://www.ezmsg.org/ezmsg/reference/API/axisarray.html) as the primary data structure for passing signals between components. The default data backend is NumPy, but other backends are supported via the Array API such as CuPy and MLX. ## Installation diff --git a/docs/source/_templates/autosummary/module.rst b/docs/source/_templates/autosummary/module.rst index fd632b42..d69f3151 100644 --- a/docs/source/_templates/autosummary/module.rst +++ b/docs/source/_templates/autosummary/module.rst @@ -1,6 +1,7 @@ {{ fullname | escape | underline}} .. automodule:: {{ fullname }} + :no-members: {% block attributes %} {% if attributes %} diff --git a/docs/source/conf.py b/docs/source/conf.py index d437505e..f2756271 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -30,6 +30,7 @@ "sphinx.ext.intersphinx", "sphinx.ext.viewcode", "sphinx.ext.duration", + "sphinx.ext.graphviz", # "sphinx_autodoc_typehints", # Disabled due to compatibility issue "sphinx_copybutton", "myst_parser", # For markdown files @@ -70,7 +71,7 @@ intersphinx_mapping = { "python": ("https://docs.python.org/3/", None), "numpy": ("https://numpy.org/doc/stable/", None), - "scipy": ("https://scipy.org/doc/scipy/", None), + "scipy": ("https://docs.scipy.org/doc/scipy/", None), "ezmsg": ("https://www.ezmsg.org/ezmsg/", None), "ezmsg.baseproc": ("https://www.ezmsg.org/ezmsg-baseproc/", None), "ezmsg.learn": ("https://www.ezmsg.org/ezmsg-learn/", None), diff --git a/docs/source/guides/HybridBuffer.md b/docs/source/guides/HybridBuffer.md index 2352a559..eb46329d 100644 --- a/docs/source/guides/HybridBuffer.md +++ b/docs/source/guides/HybridBuffer.md @@ -1,10 +1,10 @@ -## HybridBuffer +# HybridBuffer The HybridBuffer is a stateful, FIFO buffer that combines a deque for fast appends with a contiguous circular buffer for efficient, advancing reads. The synchronization between the deque and the circular buffer can be immediate, upon threshold reaching, or on demand, allowing for flexible data management strategies. This buffer is designed to be agnostic to the array library used (e.g., NumPy, CuPy, PyTorch) via the Python Array API standard. -### Basic Reading and Writing Behaviour +## Basic Reading and Writing Behaviour The following diagram illustrates the states of the HybridBuffer across data writes and reads when `update_strategy="on_demand"`: @@ -39,7 +39,7 @@ H. We then `read(4)` again. This time, a `flush()` is not triggered because we h Note: `peek(n)` and `seek(n)`, where `n` > `n_available` will raise an error. However, `peek(None)` will return all available samples without error, and `seek(None)` will advance the tail to the end of the available data. -### Overflow Behaviour +## Overflow Behaviour The criteria to trigger an overflow are as follows: * the deque has more data than there is space in the circular buffer, where space is the combination of previously read samples and unwritten samples in the circular buffer. @@ -70,7 +70,7 @@ There are a few mitigations to defer flushing to help prevent overflows: * Be cautious relying on repeated calls to `peek_at(k, allow_flush=False)` as it scans over the items in the deque which can be slow. * When calling `read(n)`, if a flush is necessary, and it will cause an overflow, and the overflow could be prevented with a pre-emptive read up to `n`, then it will do the read in 2 parts. First it will call `peek(n_unread_in_buffer)` and `seek(n_unread_in_buffer)` to read the unread samples in the circular buffer. Second, it will call `peek(n_remaining)` and `seek(n_remaining)` to trigger a flush -- which should no longer cause an overflow -- then read the remaining requested samples and stitch them together. -### Advanced Pointer Manipulation +## Advanced Pointer Manipulation The previous section describes how `read`, `peek`, `seek`, and `peek_at` function in normal use cases. It is also possible to call `seek` with a negative value, which will attempt to move the tail pointer backwards over previously-read (or previously sought-over) data by that many samples. `seek` returns the number of samples that were actually moved, which may be less than the requested value if there was insufficient room. Negative seeks can only rewind into previously read data, and positive seeks can only advance into unread data, possibly including data that gets flushed from the deque. diff --git a/docs/source/guides/explanations/sigproc.rst b/docs/source/guides/explanations/sigproc.rst index f052e874..0af9fd47 100644 --- a/docs/source/guides/explanations/sigproc.rst +++ b/docs/source/guides/explanations/sigproc.rst @@ -9,7 +9,7 @@ It also comes with a collection of pre-built signal processing classes and relev A list of available signal processors and ezmsg Units can be found in the (TBD) `ezmsg-sigproc reference `_. -|ezmsg_logo_small| Rationale For Implementation +Rationale For Implementation ******************************************************** Providing a flexible and extensible framework for signal processing tasks makes it @@ -20,7 +20,7 @@ Providing a flexible and extensible framework for signal processing tasks makes - allows standalone use outside of an ezmsg context -|ezmsg_logo_small| How to decide which processor template to use? +How to decide which processor template to use? ****************************************************************** We use the term "processor" to refer to any class that processes signals. We then separate processors into types based on whether or not they receive input messages (typically signal data), send output messages, or both: @@ -94,7 +94,7 @@ The decision tree for this classification is as follows: The leaf nodes in yellow are abstract base classes provided in `ezmsg.sigproc.base` for implementing standalone processors. The table below summarizes these base classes. -|ezmsg_logo_small| Abstract implementations (Base Classes) for standalone processors +Abstract implementations (Base Classes) for standalone processors *************************************************************************************** @@ -242,7 +242,7 @@ NOTES: do not inherit from ``BaseStatefulProcessor`` and ``BaseStatefulProducer``. They accomplish statefulness by inheriting from the mixin abstract base class ``CompositeStateful``, which implements the state related methods: ``get_state_type``, ``state.setter``, ``state.getter``, ``_hash_message``, ``_reset_state``, and ``stateful_op`` (as well as composite processor chain related methods). However, ``BaseStatefulProcessor``, ``BaseStatefulProducer`` implement ``stateful_op`` method for a single processor in an incompatible way to what is required for composite chains of processors. -|ezmsg_logo_small| Implementing a custom standalone processor +Implementing a custom standalone processor **************************************************************** 1. Create a new settings dataclass: ``class MySettings(ez.Settings):`` @@ -268,7 +268,7 @@ do not inherit from ``BaseStatefulProcessor`` and ``BaseStatefulProducer``. They * ``ClockProducer`` overrides ``__call__`` in order to provide a synchronous call bypassing the default async behaviour. -|ezmsg_logo_small| Abstract implementations (Base Classes) for ezmsg Units using processors +Abstract implementations (Base Classes) for ezmsg Units using processors ********************************************************************************************** Generic TypeVars for ezmsg Units @@ -314,7 +314,7 @@ Base Classes for ezmsg processor Units: Note, it is strongly recommended to use `BaseConsumerUnit`, `BaseTransformerUnit`, or `BaseAdaptiveTransformerUnit` for implementing concrete subclasses rather than `BaseProcessorUnit`. -|ezmsg_logo_small| How to implement a custom ezmsg Unit from a standalone processor +How to implement a custom ezmsg Unit from a standalone processor ===================================================================================== 1. Create and test custom standalone processor as above. @@ -353,13 +353,9 @@ Often, all that is required is the following (e.g., for a custom transformer): .. note:: The type of ProcessorUnit is based on the internal processor and not the input or output of the unit. Input streams are allowed in ProducerUnits and output streams in ConsumerUnits. For an example of such a use case, see ``BaseCounterFirstProducerUnit`` and its subclasses. ``BaseCounterFirstProducerUnit`` has an input stream that receives a flag signal from a clock that triggers a call to the internal producer. -|ezmsg_logo_small| See Also +See Also ******************************** -1. `Signal Processor Documentation `_ -#. `Signal Processing Tutorial <../../tutorials/signalprocessing.html>`_ -#. `Signal Processing HOW TOs <../../how-tos/signalprocessing/main.html>`_ - -.. |ezmsg_logo_small| image:: ../_static/_images/ezmsg_logo.png - :width: 40 - :alt: ezmsg logo \ No newline at end of file +#. :doc:`Signal Processing Tutorial <../tutorials/signalprocessing>` — end-to-end walkthrough building a processor and Unit +#. `ezmsg-baseproc documentation `_ — base class API reference and implementation guide +#. :doc:`ezmsg-sigproc API reference <../../api/index>` — autogenerated reference for all sigproc modules \ No newline at end of file diff --git a/docs/source/guides/how-tos/signalprocessing/adaptive.rst b/docs/source/guides/how-tos/signalprocessing/adaptive.rst deleted file mode 100644 index 3dbaf7d3..00000000 --- a/docs/source/guides/how-tos/signalprocessing/adaptive.rst +++ /dev/null @@ -1,4 +0,0 @@ -How to implement adaptive signal processing in ezmsg? -####################################################### - -(under construction) \ No newline at end of file diff --git a/docs/source/guides/how-tos/signalprocessing/checkpoint.rst b/docs/source/guides/how-tos/signalprocessing/checkpoint.rst deleted file mode 100644 index 79fcdc46..00000000 --- a/docs/source/guides/how-tos/signalprocessing/checkpoint.rst +++ /dev/null @@ -1,4 +0,0 @@ -How to use checkpoints for ezmsg signal processing Units that leverage ML models? -###################################################################################### - -(under construction) \ No newline at end of file diff --git a/docs/source/guides/how-tos/signalprocessing/composite.rst b/docs/source/guides/how-tos/signalprocessing/composite.rst deleted file mode 100644 index 18cc5bd3..00000000 --- a/docs/source/guides/how-tos/signalprocessing/composite.rst +++ /dev/null @@ -1,6 +0,0 @@ -How to efficiently chain multiple signal processors in ezmsg? -############################################################# - -For general composite processor implementation guidance, see the `ezmsg-baseproc documentation `_. - -(under construction) diff --git a/docs/source/guides/how-tos/signalprocessing/content-signalprocessing.rst b/docs/source/guides/how-tos/signalprocessing/content-signalprocessing.rst deleted file mode 100644 index 4aac0758..00000000 --- a/docs/source/guides/how-tos/signalprocessing/content-signalprocessing.rst +++ /dev/null @@ -1,13 +0,0 @@ -Signal Processing HOW TOs -########################## - -.. toctree:: - :maxdepth: 1 - - processor - stateful - standalone - adaptive - composite - unit - checkpoint diff --git a/docs/source/guides/how-tos/signalprocessing/processor.rst b/docs/source/guides/how-tos/signalprocessing/processor.rst deleted file mode 100644 index 3c38f8b9..00000000 --- a/docs/source/guides/how-tos/signalprocessing/processor.rst +++ /dev/null @@ -1,8 +0,0 @@ -How to write a signal processor in ezmsg? -######################################### - -For general processor implementation guidance, see the `ezmsg-baseproc documentation `_. - -For signal processing specific patterns, see the existing processors in ``ezmsg.sigproc`` as examples. - -(under construction) diff --git a/docs/source/guides/how-tos/signalprocessing/standalone.rst b/docs/source/guides/how-tos/signalprocessing/standalone.rst deleted file mode 100644 index 2e8eefa8..00000000 --- a/docs/source/guides/how-tos/signalprocessing/standalone.rst +++ /dev/null @@ -1,4 +0,0 @@ -How to use ezmsg-sigproc signal processors outside of an ezmsg context? -############################################################################### - -(under construction) \ No newline at end of file diff --git a/docs/source/guides/how-tos/signalprocessing/stateful.rst b/docs/source/guides/how-tos/signalprocessing/stateful.rst deleted file mode 100644 index 2754d37c..00000000 --- a/docs/source/guides/how-tos/signalprocessing/stateful.rst +++ /dev/null @@ -1,8 +0,0 @@ -How to implement a stateful signal processor in ezmsg? -###################################################### - -For general stateful processor implementation guidance, see the `ezmsg-baseproc documentation `_. - -For signal processing specific patterns using stateful processors, see implementations like ``ChebyshevFilterTransformer`` and ``WindowTransformer`` as examples. - -(under construction) diff --git a/docs/source/guides/how-tos/signalprocessing/unit.rst b/docs/source/guides/how-tos/signalprocessing/unit.rst deleted file mode 100644 index ee3a4124..00000000 --- a/docs/source/guides/how-tos/signalprocessing/unit.rst +++ /dev/null @@ -1,41 +0,0 @@ -How to turn a signal processor into an ``ezmsg`` Unit? -###################################################### - -For general guidance on converting processors to ezmsg Units, see the `ezmsg-baseproc documentation `_. - -Example with Signal Processing ------------------------------- - -To convert a signal processor to an ``ezmsg`` Unit: - -1. **Define the Processor**: Create a class that inherits from the appropriate base class (e.g., ``BaseTransformer``, ``BaseStatefulTransformer``). -2. **Implement the Processing Logic**: Override the ``_process`` method to implement the signal processing logic. -3. **Create the Unit**: Inherit from the appropriate Unit base class (e.g., ``BaseTransformerUnit``). - -.. code-block:: python - - import ezmsg.core as ez - from ezmsg.util.messages.axisarray import AxisArray - from ezmsg.baseproc import BaseTransformer, BaseTransformerUnit - - - class MyProcessorSettings(ez.Settings): - # Your settings here - pass - - - class MyProcessor(BaseTransformer[MyProcessorSettings, AxisArray, AxisArray]): - def _process(self, message: AxisArray) -> AxisArray: - # Your signal processing logic here - return message - - - class MyUnit(BaseTransformerUnit[ - MyProcessorSettings, - AxisArray, - AxisArray, - MyProcessor, - ]): - SETTINGS = MyProcessorSettings - -(under construction) diff --git a/docs/source/guides/sigproc/architecture.rst b/docs/source/guides/sigproc/architecture.rst index ba3ea5ba..64d72812 100644 --- a/docs/source/guides/sigproc/architecture.rst +++ b/docs/source/guides/sigproc/architecture.rst @@ -33,9 +33,10 @@ Key Modules Where to Learn Next ------------------- -* Study the :doc:`base` page to master the processor architecture. +* Read the :doc:`../explanations/sigproc` explainer to master the processor architecture. +* Follow the :doc:`../tutorials/signalprocessing` to build a processor and Unit from scratch. +* See the `ezmsg-baseproc documentation `_ for the base class API and implementation details. * Explore unit tests in the repository for hands-on examples of composing processors and Units. * Review the `ezmsg framework `_ to understand the surrounding ecosystem. -* Experiment with the code—try running processors standalone and then integrate them into a small pipeline to observe the trade-offs firsthand. This approach equips newcomers to choose the right level of abstraction—raw processor, Unit wrapper, or full pipeline—based on the demands of their analysis or streaming application. diff --git a/docs/source/guides/sigproc/base.rst b/docs/source/guides/sigproc/base.rst deleted file mode 100644 index bc4639fd..00000000 --- a/docs/source/guides/sigproc/base.rst +++ /dev/null @@ -1,19 +0,0 @@ -Base Processors -=============== - -The base processor classes are provided by the ``ezmsg-baseproc`` package and re-exported from ``ezmsg.sigproc.base`` for backwards compatibility. - -For detailed documentation on the base processor architecture, see the `ezmsg-baseproc documentation `_. - -.. note:: - New code should import directly from ``ezmsg.baseproc`` instead of ``ezmsg.sigproc.base``. - -API Reference -------------- - -The following classes are re-exported from ``ezmsg.baseproc``: - -.. automodule:: ezmsg.sigproc.base - :members: - :show-inheritance: - :imported-members: diff --git a/docs/source/guides/sigproc/content-sigproc.rst b/docs/source/guides/sigproc/content-sigproc.rst index be77ff9b..334c14f3 100644 --- a/docs/source/guides/sigproc/content-sigproc.rst +++ b/docs/source/guides/sigproc/content-sigproc.rst @@ -1,25 +1,9 @@ ezmsg-sigproc =============== -Timeseries signal processing implementations in ezmsg, leveraging numpy and scipy. -Most of the methods and classes in this extension are intended to be used in building signal processing pipelines. -They use :class:`ezmsg.util.messages.axisarray.AxisArray` as the primary data structure for passing signals between components. -The message's data are typically NumPy arrays, though many transformers support the -:doc:`Array API standard <../explanations/array_api>` for use with CuPy, PyTorch, and other backends. - -.. note:: Some generators might yield valid :class:`AxisArray` messages with ``.data`` size of 0. -This may occur when the generator receives inadequate data to produce a valid output, such as when windowing or buffering. - -`ezmsg-sigproc` contains two types of modules: - -- base processors and units that provide fundamental building blocks for signal processing pipelines -- in-built signal processing modules that implement common signal processing techniques - .. toctree:: :maxdepth: 1 architecture - base - units - processors + ../explanations/sigproc ../explanations/array_api diff --git a/docs/source/guides/sigproc/processors.rst b/docs/source/guides/sigproc/processors.rst deleted file mode 100644 index 8662d8ea..00000000 --- a/docs/source/guides/sigproc/processors.rst +++ /dev/null @@ -1,149 +0,0 @@ -In-Built Signal Processing Modules -====================================================== - -Here is the API reference for the in-built signal processing modules included in the `ezmsg-sigproc` extension. - -ezmsg.sigproc.activation --------------------------- - -.. automodule:: ezmsg.sigproc.activation - :members: - - -ezmsg.sigproc.affinetransform -------------------------------- - -.. automodule:: ezmsg.sigproc.affinetransform - :members: - - -ezmsg.sigproc.aggregate -------------------------- - -.. automodule:: ezmsg.sigproc.aggregate - :members: - :undoc-members: - - -ezmsg.sigproc.bandpower -------------------------- - -.. automodule:: ezmsg.sigproc.bandpower - :members: - - -ezmsg.sigproc.filter ----------------------- - -.. automodule:: ezmsg.sigproc.filter - :members: - -ezmsg.sigproc.butterworthfilter ---------------------------------- - -.. automodule:: ezmsg.sigproc.butterworthfilter - :members: - - -ezmsg.sigproc.decimate ------------------------- - -.. automodule:: ezmsg.sigproc.decimate - :members: - - -ezmsg.sigproc.denormalize ------------------------- - -.. automodule:: ezmsg.sigproc.denormalize - :members: - - -ezmsg.sigproc.downsample --------------------------- - -.. automodule:: ezmsg.sigproc.downsample - :members: - - -ezmsg.sigproc.ewmfilter -------------------------- - -.. automodule:: ezmsg.sigproc.ewmfilter - :members: - - -ezmsg.sigproc.math ------------------------ - -.. automodule:: ezmsg.sigproc.math.clip - :members: - -.. automodule:: ezmsg.sigproc.math.difference - :members: - -.. automodule:: ezmsg.sigproc.math.invert - :members: - -.. automodule:: ezmsg.sigproc.math.log - :members: - -.. automodule:: ezmsg.sigproc.math.scale - :members: - - -ezmsg.sigproc.sampler ------------------------ - -.. automodule:: ezmsg.sigproc.sampler - :members: - - -ezmsg.sigproc.scaler ----------------------- - -.. automodule:: ezmsg.sigproc.scaler - :members: - - -ezmsg.sigproc.signalinjector ------------------------------- - -.. automodule:: ezmsg.sigproc.signalinjector - :members: - - -ezmsg.sigproc.slicer ------------------------ - -.. automodule:: ezmsg.sigproc.slicer - :members: - - -ezmsg.sigproc.spectrum ------------------------- - -.. automodule:: ezmsg.sigproc.spectrum - :members: - :undoc-members: - - -ezmsg.sigproc.spectrogram ---------------------------- - -.. automodule:: ezmsg.sigproc.spectrogram - :members: - - -ezmsg.sigproc.synth ---------------------- - -.. automodule:: ezmsg.sigproc.synth - :members: - - -ezmsg.sigproc.window ----------------------- - -.. automodule:: ezmsg.sigproc.window - :members: \ No newline at end of file diff --git a/docs/source/guides/sigproc/units.rst b/docs/source/guides/sigproc/units.rst deleted file mode 100644 index 322e8927..00000000 --- a/docs/source/guides/sigproc/units.rst +++ /dev/null @@ -1,29 +0,0 @@ -Base Processor Units -============================= - -Here is the API for the base processor ezmsg ``Unit``\ s included in the `ezmsg-sigproc` extension. For more detailed information on the design decisions behind these base units, please refer to the :doc:`ezmsg-sigproc explainer <../../explanations/sigproc>`. - -.. autoclass:: ezmsg.sigproc.base.BaseProducerUnit - :members: - :show-inheritance: - :inherited-members: - -.. autoclass:: ezmsg.sigproc.base.BaseProcessorUnit - :members: - :show-inheritance: - :inherited-members: - -.. autoclass:: ezmsg.sigproc.base.BaseConsumerUnit - :members: - :show-inheritance: - :inherited-members: - -.. autoclass:: ezmsg.sigproc.base.BaseTransformerUnit - :members: - :show-inheritance: - :inherited-members: - -.. autoclass:: ezmsg.sigproc.base.BaseAdaptiveTransformerUnit - :members: - :show-inheritance: - :inherited-members: diff --git a/docs/source/guides/tutorials/signalprocessing.rst b/docs/source/guides/tutorials/signalprocessing.rst index 96f0a4ce..acec2237 100644 --- a/docs/source/guides/tutorials/signalprocessing.rst +++ b/docs/source/guides/tutorials/signalprocessing.rst @@ -8,7 +8,7 @@ We will explore how to do this by recreating the `Downsample` signal processor u .. tip:: Downsampling is a common signal processing operation that reduces the sampling rate of a signal by keeping only every nth sample. This is useful for reducing the amount of data to be processed, especially in real-time applications. -|ezmsg_logo_small| Choosing your signal processing class +Choosing your signal processing class ********************************************************** We make use of the following decision tree to choose the appropriate signal processing class: @@ -112,7 +112,7 @@ First, we need to install the `ezmsg-sigproc` package if we haven't already. Thi pip install "ezmsg-sigproc" -|ezmsg_logo_small| Creating the `Downsample` signal processor +Creating the `Downsample` signal processor ************************************************************* We begin by identifying the components needed to create the `Downsample` signal processor. This includes defining the settings, state, and the main processing class itself. @@ -199,7 +199,7 @@ Again, our class seems to be missing an ``__init__`` method, but this is because .. note:: Finally, our transformer is **not async first** as we do not need to prioritise asynchronous processing, which is usually more relevant for processors that interface with IO operations whose timing is unpredictable. -|ezmsg_logo_small| DownsampleTransformer Class +DownsampleTransformer Class ******************************************************* We have already identified that we will be using a stateful transformer, so we will inherit from the ``BaseStatefulTransformer`` class. Create the class definition as follows: @@ -307,7 +307,7 @@ Finally, we reset the index of the next message's first sample to 0. .. _processing_data_tutorial: -|ezmsg_logo_small| Processing the Data +Processing the Data *********************************************** To finish the `DownsampleTransformer` class, we need to actually process the data by downsampling. @@ -442,7 +442,7 @@ The final implementation of the ``_process`` method looks like this: return msg_out -|ezmsg_logo_small| Final DownsampleTransformer Class +Final DownsampleTransformer Class ******************************************************* Confirm that your final `DownsampleTransformer` class looks like this: @@ -515,7 +515,7 @@ Confirm that your final `DownsampleTransformer` class looks like this: return msg_out -|ezmsg_logo_small| Using the DownsampleTransformer +Using the DownsampleTransformer ********************************************************** The `Downsample` class is now fully implemented and ready for use in signal processing pipelines. @@ -558,7 +558,7 @@ Doing the above is very handy for unit testing your processor as well as for off Of course, the real power of `ezmsg` comes from integrating your processor into an `ezmsg` Unit and using it in a processing pipeline. We will see how to do this next. -|ezmsg_logo_small| Creating the `Downsample ezmsg` Unit +Creating the `Downsample ezmsg` Unit *********************************************************** `ezmsg-sigproc` provides convenient ezmsg `Unit` wrappers for all the signal processor base classes. To do this inherit from the appropriate `ezmsg-sigproc` unit class. These are: @@ -584,13 +584,9 @@ A lot of the behind-the-scenes work is done for you by the `BaseTransformerUnit` Connecting it to other `Component`\ s and initialising the transformer are accomplished in the same way that we did in the `Pipeline Tutorial `_ -|ezmsg_logo_small| See Also +See Also ************************************ - `Further examples `_ can be found in the examples directory in `ezmsg`. These are examples of creating and using `ezmsg` Units and pipelines. - `ezmsg-sigproc` has a large number of already implemented signal processors. More information can be found at the :doc:`ezmsg-sigproc reference <../sigproc/content-sigproc>`. -- :doc:`Downsample class reference <../../api/generated/ezmsg.sigproc.downsample>` - -.. |ezmsg_logo_small| image:: ../_static/_images/ezmsg_logo.png - :width: 40 - :alt: ezmsg logo \ No newline at end of file +- :doc:`Downsample class reference <../../api/generated/ezmsg.sigproc.downsample>` \ No newline at end of file diff --git a/docs/source/index.md b/docs/source/index.md index c29dd1fb..25ef007b 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -1,13 +1,10 @@ ```{include} ../../README.md ``` -## Documentation - ```{toctree} :maxdepth: 2 :caption: Contents: -guides/how-tos/signalprocessing/content-signalprocessing guides/sigproc/content-sigproc guides/tutorials/signalprocessing guides/HybridBuffer diff --git a/pyproject.toml b/pyproject.toml index 0e6ab006..5826da5a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -13,8 +13,8 @@ requires-python = ">=3.10" dynamic = ["version"] dependencies = [ "array-api-compat>=1.11.1", - "ezmsg[axisarray]>=3.7.2", - "ezmsg-baseproc>=1.5.0", + "ezmsg[axisarray]>=3.7.3", + "ezmsg-baseproc>=1.5.1", "mlx>=0.18.0; sys_platform == 'darwin' and platform_machine == 'arm64'", "numba>=0.61.0", "numpy>=1.26.0", diff --git a/src/ezmsg/sigproc/activation.py b/src/ezmsg/sigproc/activation.py index 21ef1f20..12dd137c 100644 --- a/src/ezmsg/sigproc/activation.py +++ b/src/ezmsg/sigproc/activation.py @@ -1,3 +1,5 @@ +"""Activation functions (ReLU, sigmoid, logit, etc.) applied element-wise.""" + import ezmsg.core as ez import scipy.special from ezmsg.baseproc import BaseTransformer, BaseTransformerUnit diff --git a/src/ezmsg/sigproc/adaptive_lattice_notch.py b/src/ezmsg/sigproc/adaptive_lattice_notch.py index dae3640e..666a5d09 100644 --- a/src/ezmsg/sigproc/adaptive_lattice_notch.py +++ b/src/ezmsg/sigproc/adaptive_lattice_notch.py @@ -1,3 +1,5 @@ +"""Adaptive lattice notch filter for narrow-band interference removal.""" + import ezmsg.core as ez import numpy as np import numpy.typing as npt diff --git a/src/ezmsg/sigproc/bandpower.py b/src/ezmsg/sigproc/bandpower.py index 5151b66e..b6dc540f 100644 --- a/src/ezmsg/sigproc/bandpower.py +++ b/src/ezmsg/sigproc/bandpower.py @@ -1,3 +1,5 @@ +"""Spectral band power estimation via windowed FFT and aggregation.""" + from dataclasses import field import ezmsg.core as ez diff --git a/src/ezmsg/sigproc/butterworthfilter.py b/src/ezmsg/sigproc/butterworthfilter.py index 4d53cbe3..942f5dc9 100644 --- a/src/ezmsg/sigproc/butterworthfilter.py +++ b/src/ezmsg/sigproc/butterworthfilter.py @@ -1,3 +1,5 @@ +"""Butterworth IIR filter design and application.""" + import functools import typing diff --git a/src/ezmsg/sigproc/cheby.py b/src/ezmsg/sigproc/cheby.py index 6faf9004..67178c25 100644 --- a/src/ezmsg/sigproc/cheby.py +++ b/src/ezmsg/sigproc/cheby.py @@ -1,3 +1,5 @@ +"""Chebyshev Type I IIR filter design and application.""" + import functools import typing diff --git a/src/ezmsg/sigproc/combfilter.py b/src/ezmsg/sigproc/combfilter.py index afe04c5d..7f968d6c 100644 --- a/src/ezmsg/sigproc/combfilter.py +++ b/src/ezmsg/sigproc/combfilter.py @@ -1,3 +1,5 @@ +"""Comb filter for suppressing a fundamental frequency and its harmonics.""" + import functools import typing diff --git a/src/ezmsg/sigproc/decimate.py b/src/ezmsg/sigproc/decimate.py index 8a037ab4..3c59d660 100644 --- a/src/ezmsg/sigproc/decimate.py +++ b/src/ezmsg/sigproc/decimate.py @@ -1,3 +1,5 @@ +"""Decimation (downsample with anti-alias filtering).""" + import typing import ezmsg.core as ez diff --git a/src/ezmsg/sigproc/denormalize.py b/src/ezmsg/sigproc/denormalize.py index 9e7050c5..2bc943df 100644 --- a/src/ezmsg/sigproc/denormalize.py +++ b/src/ezmsg/sigproc/denormalize.py @@ -1,3 +1,5 @@ +"""Denormalize signals by applying random gains and offsets.""" + import ezmsg.core as ez import numpy as np import numpy.typing as npt diff --git a/src/ezmsg/sigproc/detrend.py b/src/ezmsg/sigproc/detrend.py index c274525e..35ab4cdf 100644 --- a/src/ezmsg/sigproc/detrend.py +++ b/src/ezmsg/sigproc/detrend.py @@ -1,3 +1,5 @@ +"""Remove linear or constant trends from data along an axis.""" + import ezmsg.core as ez import scipy.signal as sps from ezmsg.baseproc import BaseTransformerUnit diff --git a/src/ezmsg/sigproc/downsample.py b/src/ezmsg/sigproc/downsample.py index d4bb1fc2..dd52abbb 100644 --- a/src/ezmsg/sigproc/downsample.py +++ b/src/ezmsg/sigproc/downsample.py @@ -1,3 +1,5 @@ +"""Integer downsampling by selecting every Nth sample along an axis.""" + import ezmsg.core as ez import numpy as np from ezmsg.baseproc import ( diff --git a/src/ezmsg/sigproc/ewma.py b/src/ezmsg/sigproc/ewma.py index 11782949..d63b92c7 100644 --- a/src/ezmsg/sigproc/ewma.py +++ b/src/ezmsg/sigproc/ewma.py @@ -1,3 +1,5 @@ +"""Exponentially weighted moving average (EWMA) utilities and parameter conversion.""" + import functools from dataclasses import field diff --git a/src/ezmsg/sigproc/ewmfilter.py b/src/ezmsg/sigproc/ewmfilter.py index 2e22ee81..4ae81899 100644 --- a/src/ezmsg/sigproc/ewmfilter.py +++ b/src/ezmsg/sigproc/ewmfilter.py @@ -1,3 +1,5 @@ +"""Exponentially weighted moving average filter for streaming normalization.""" + import asyncio import typing diff --git a/src/ezmsg/sigproc/extract_axis.py b/src/ezmsg/sigproc/extract_axis.py index 8af74165..bf7a568b 100644 --- a/src/ezmsg/sigproc/extract_axis.py +++ b/src/ezmsg/sigproc/extract_axis.py @@ -1,3 +1,5 @@ +"""Extract a named axis from an AxisArray as a standalone array.""" + import ezmsg.core as ez import numpy as np from ezmsg.baseproc import BaseTransformer, BaseTransformerUnit diff --git a/src/ezmsg/sigproc/fbcca.py b/src/ezmsg/sigproc/fbcca.py index bfcf4b06..6d8695ad 100644 --- a/src/ezmsg/sigproc/fbcca.py +++ b/src/ezmsg/sigproc/fbcca.py @@ -1,3 +1,5 @@ +"""Filter-bank canonical correlation analysis (FBCCA) for frequency detection.""" + import math import typing from dataclasses import field diff --git a/src/ezmsg/sigproc/filter.py b/src/ezmsg/sigproc/filter.py index 3454a1a0..076f0b0f 100644 --- a/src/ezmsg/sigproc/filter.py +++ b/src/ezmsg/sigproc/filter.py @@ -1,3 +1,5 @@ +"""Core IIR/FIR filtering infrastructure with BA and SOS coefficient support.""" + import typing from abc import ABC, abstractmethod from dataclasses import dataclass, field diff --git a/src/ezmsg/sigproc/filterbank.py b/src/ezmsg/sigproc/filterbank.py index 0961a648..3f374c23 100644 --- a/src/ezmsg/sigproc/filterbank.py +++ b/src/ezmsg/sigproc/filterbank.py @@ -1,3 +1,5 @@ +"""Parallel filterbank processing with convolution, FFT, or automatic mode selection.""" + import functools import math import typing diff --git a/src/ezmsg/sigproc/filterbankdesign.py b/src/ezmsg/sigproc/filterbankdesign.py index b3e9ae37..949e4d90 100644 --- a/src/ezmsg/sigproc/filterbankdesign.py +++ b/src/ezmsg/sigproc/filterbankdesign.py @@ -1,3 +1,5 @@ +"""Kaiser-window filterbank design with configurable bands and processing mode.""" + import typing import ezmsg.core as ez diff --git a/src/ezmsg/sigproc/fir_hilbert.py b/src/ezmsg/sigproc/fir_hilbert.py index 8cdd3c5d..66ba1a12 100644 --- a/src/ezmsg/sigproc/fir_hilbert.py +++ b/src/ezmsg/sigproc/fir_hilbert.py @@ -1,3 +1,5 @@ +"""FIR Hilbert transform filter for analytic signal and envelope extraction.""" + import functools import typing @@ -243,17 +245,15 @@ class FIRHilbertEnvelopeTransformer( Optional normalization frequency in Hz for gain normalization. If None, no normalization is applied. - Example: - ----------------------------- - ```python - processor = FIRHilbertEnvelopeTransformer( - settings=FIRHilbertFilterSettings( - order=170, - f_lo=1.0, - f_hi=50.0, + Example:: + + processor = FIRHilbertEnvelopeTransformer( + settings=FIRHilbertFilterSettings( + order=170, + f_lo=1.0, + f_hi=50.0, + ) ) - ) - ``` """ @@ -320,17 +320,16 @@ class FIRHilbertEnvelopeUnit( This unit provides a plug-and-play interface for calculating the envelope using the FIR Hilbert transform on a signal in an ezmsg graph-based system. It takes in `AxisArray` inputs and outputs processed data in the same format. - Example: - -------- - ```python - unit = FIRHilbertEnvelopeUnit( - settings=FIRHilbertFilterSettings( - order=170, - f_lo=1.0, - f_hi=50.0, + Example:: + + unit = FIRHilbertEnvelopeUnit( + settings=FIRHilbertFilterSettings( + order=170, + f_lo=1.0, + f_hi=50.0, + ) ) - ) - ``` + """ SETTINGS = FIRHilbertFilterSettings diff --git a/src/ezmsg/sigproc/fir_pmc.py b/src/ezmsg/sigproc/fir_pmc.py index 87a7e1d7..f9c1f816 100644 --- a/src/ezmsg/sigproc/fir_pmc.py +++ b/src/ezmsg/sigproc/fir_pmc.py @@ -1,3 +1,5 @@ +"""Parks-McClellan (Remez) optimal equiripple FIR filter design and application.""" + import functools import typing diff --git a/src/ezmsg/sigproc/firfilter.py b/src/ezmsg/sigproc/firfilter.py index 55da5d0c..a2eb0f75 100644 --- a/src/ezmsg/sigproc/firfilter.py +++ b/src/ezmsg/sigproc/firfilter.py @@ -1,3 +1,5 @@ +"""FIR filter design and application using scipy.signal.firwin.""" + import functools import typing @@ -54,10 +56,11 @@ class FIRFilterSettings(FilterBaseSettings): """ Set to True to scale the coefficients so that the frequency response is exactly unity at a certain frequency. That frequency is either: + * 0 (DC) if the first passband starts at 0 (i.e. pass_zero is True) * fs/2 (the Nyquist frequency) if the first passband ends at fs/2 - (i.e the filter is a single band highpass filter); - center of first passband otherwise + (i.e the filter is a single band highpass filter); + center of first passband otherwise """ wn_hz: bool = True diff --git a/src/ezmsg/sigproc/gaussiansmoothing.py b/src/ezmsg/sigproc/gaussiansmoothing.py index d3951c2d..6a697962 100644 --- a/src/ezmsg/sigproc/gaussiansmoothing.py +++ b/src/ezmsg/sigproc/gaussiansmoothing.py @@ -1,3 +1,5 @@ +"""Gaussian kernel smoothing filter.""" + import warnings from typing import Callable diff --git a/src/ezmsg/sigproc/kaiser.py b/src/ezmsg/sigproc/kaiser.py index d4f1eb47..aa659853 100644 --- a/src/ezmsg/sigproc/kaiser.py +++ b/src/ezmsg/sigproc/kaiser.py @@ -1,3 +1,5 @@ +"""Kaiser window FIR filter design and application.""" + import functools import typing diff --git a/src/ezmsg/sigproc/messages.py b/src/ezmsg/sigproc/messages.py index f3c80997..08ec4307 100644 --- a/src/ezmsg/sigproc/messages.py +++ b/src/ezmsg/sigproc/messages.py @@ -1,3 +1,5 @@ +"""Deprecated message types; use :class:`ezmsg.util.messages.axisarray.AxisArray` instead.""" + import time import warnings diff --git a/src/ezmsg/sigproc/quantize.py b/src/ezmsg/sigproc/quantize.py index 3d11cd84..47efa3be 100644 --- a/src/ezmsg/sigproc/quantize.py +++ b/src/ezmsg/sigproc/quantize.py @@ -1,3 +1,5 @@ +"""Quantize signal data to a fixed number of bits.""" + import ezmsg.core as ez from array_api_compat import get_namespace from ezmsg.baseproc import BaseTransformer, BaseTransformerUnit diff --git a/src/ezmsg/sigproc/resample.py b/src/ezmsg/sigproc/resample.py index 1d3ed2fa..58445e61 100644 --- a/src/ezmsg/sigproc/resample.py +++ b/src/ezmsg/sigproc/resample.py @@ -1,3 +1,5 @@ +"""Resample signals to a target rate using interpolation.""" + import asyncio import math import time diff --git a/src/ezmsg/sigproc/rollingscaler.py b/src/ezmsg/sigproc/rollingscaler.py index fdeadf80..50e21aec 100644 --- a/src/ezmsg/sigproc/rollingscaler.py +++ b/src/ezmsg/sigproc/rollingscaler.py @@ -1,3 +1,5 @@ +"""Rolling z-score normalization using a sliding window of recent samples.""" + import math from collections import deque @@ -91,15 +93,14 @@ class RollingScalerProcessor(BaseAdaptiveTransformer[RollingScalerSettings, Axis k_samples: int Number of previous samples to use for rolling statistics. - Example: - ----------------------------- - ```python - processor = RollingScalerProcessor( - settings=RollingScalerSettings( - k_samples=20 # Number of previous samples to use for rolling statistics + Example:: + + processor = RollingScalerProcessor( + settings=RollingScalerSettings( + k_samples=20 # Number of previous samples to use for rolling statistics + ) ) - ) - ``` + """ def _hash_message(self, message: AxisArray) -> int: @@ -224,15 +225,14 @@ class RollingScalerUnit( statistics (mean and variance) over the last `k_samples` samples received. When processing an `AxisArray` message, it normalizes the data using the current rolling statistics. - Example: - ----------------------------- - ```python - unit = RollingScalerUnit( - settings=RollingScalerSettings( - k_samples=20 # Number of previous samples to use for rolling statistics + Example:: + + unit = RollingScalerUnit( + settings=RollingScalerSettings( + k_samples=20 # Number of previous samples to use for rolling statistics + ) ) - ) - ``` + """ SETTINGS = RollingScalerSettings diff --git a/src/ezmsg/sigproc/sampler.py b/src/ezmsg/sigproc/sampler.py index 2e1c34a0..9cd304db 100644 --- a/src/ezmsg/sigproc/sampler.py +++ b/src/ezmsg/sigproc/sampler.py @@ -1,3 +1,5 @@ +"""Triggered windowed sampling from a buffered signal stream.""" + import asyncio import copy import traceback @@ -54,9 +56,9 @@ class SamplerSettings(ez.Settings): estimate_alignment: bool = True """ If true, use message timestamp fields and reported sampling rate to estimate - sample-accurate alignment for samples. - If false, sampling will be limited to incoming message rate -- "Block timing" - NOTE: For faster-than-realtime playback -- Incoming timestamps must reflect + sample-accurate alignment for samples. + If false, sampling will be limited to incoming message rate -- "Block timing". + NOTE: For faster-than-realtime playback -- Incoming timestamps must reflect "realtime" operation for estimate_alignment to operate correctly. """ diff --git a/src/ezmsg/sigproc/scaler.py b/src/ezmsg/sigproc/scaler.py index 51e6b63e..79b0648f 100644 --- a/src/ezmsg/sigproc/scaler.py +++ b/src/ezmsg/sigproc/scaler.py @@ -1,3 +1,5 @@ +"""Adaptive standard scaling using exponentially weighted moving statistics.""" + import typing import ezmsg.core as ez diff --git a/src/ezmsg/sigproc/signalinjector.py b/src/ezmsg/sigproc/signalinjector.py index 390439da..ec660f2b 100644 --- a/src/ezmsg/sigproc/signalinjector.py +++ b/src/ezmsg/sigproc/signalinjector.py @@ -1,3 +1,7 @@ +"""Inject synthetic sinusoidal signals into a data stream.""" + +import typing + import ezmsg.core as ez import numpy as np import numpy.typing as npt @@ -58,7 +62,7 @@ async def _aprocess(self, message: AxisArray) -> AxisArray: class SignalInjector(BaseTransformerUnit[SignalInjectorSettings, AxisArray, AxisArray, SignalInjectorTransformer]): SETTINGS = SignalInjectorSettings - INPUT_FREQUENCY = ez.InputStream(float | None) + INPUT_FREQUENCY = ez.InputStream(typing.Optional[float]) INPUT_AMPLITUDE = ez.InputStream(float) @ez.subscriber(INPUT_FREQUENCY) diff --git a/src/ezmsg/sigproc/slicer.py b/src/ezmsg/sigproc/slicer.py index c43a8e47..cdd4ca8b 100644 --- a/src/ezmsg/sigproc/slicer.py +++ b/src/ezmsg/sigproc/slicer.py @@ -1,3 +1,5 @@ +"""Select a subset of data along a named axis using slice notation.""" + import ezmsg.core as ez import numpy as np import numpy.typing as npt diff --git a/src/ezmsg/sigproc/spectral.py b/src/ezmsg/sigproc/spectral.py index 1072a36a..d432d447 100644 --- a/src/ezmsg/sigproc/spectral.py +++ b/src/ezmsg/sigproc/spectral.py @@ -1,3 +1,5 @@ +"""Backwards-compatible re-exports from :mod:`.spectrum`.""" + from .spectrum import OptionsEnum as OptionsEnum from .spectrum import SpectralOutput as SpectralOutput from .spectrum import SpectralTransform as SpectralTransform diff --git a/src/ezmsg/sigproc/spectrogram.py b/src/ezmsg/sigproc/spectrogram.py index ad0020ed..9a971bfc 100644 --- a/src/ezmsg/sigproc/spectrogram.py +++ b/src/ezmsg/sigproc/spectrogram.py @@ -1,3 +1,5 @@ +"""Time-frequency spectrogram via windowed short-time Fourier transform.""" + from typing import Generator import ezmsg.core as ez diff --git a/src/ezmsg/sigproc/spectrum.py b/src/ezmsg/sigproc/spectrum.py index bdc5c597..b7272452 100644 --- a/src/ezmsg/sigproc/spectrum.py +++ b/src/ezmsg/sigproc/spectrum.py @@ -1,3 +1,5 @@ +"""FFT-based power spectrum estimation with configurable window functions.""" + import enum import math import typing diff --git a/src/ezmsg/sigproc/util/buffer.py b/src/ezmsg/sigproc/util/buffer.py index ec54239c..48f1a70a 100644 --- a/src/ezmsg/sigproc/util/buffer.py +++ b/src/ezmsg/sigproc/util/buffer.py @@ -184,9 +184,9 @@ def peek(self, n_samples: int | None = None, out: Array | None = None) -> Array: n_samples: The number of samples to retrieve. If None, returns all unread samples. out: Optionally, a destination array to store the samples. - If provided, must have shape (n_samples, *other_shape) where + If provided, must have shape ``(n_samples, *other_shape)`` where other_shape matches the shape of the samples in the buffer. - If `out` is provided then the data will always be copied into it, + If ``out`` is provided then the data will always be copied into it, even if they are contiguous in the buffer. Returns: @@ -303,8 +303,9 @@ def _flush_if_needed(self, n_samples: int | None = None): def flush(self): """ Transfers all data from the deque to the circular buffer. + Note: This may overwrite data depending on the overflow strategy, - which will invalidate previous state variables. + which will invalidate previous state variables. """ if not self._deque: return diff --git a/src/ezmsg/sigproc/util/sparse.py b/src/ezmsg/sigproc/util/sparse.py index f9eac393..38048c58 100644 --- a/src/ezmsg/sigproc/util/sparse.py +++ b/src/ezmsg/sigproc/util/sparse.py @@ -35,9 +35,8 @@ def sliding_win_oneaxis(s: sparse.SparseArray, nwin: int, axis: int, step: int = - Accepts a single `nwin` and a single `axis`. - Inserts a new 'win' axis immediately BEFORE the original target axis. - Output shape: - s.shape[:axis] + (W,) + (nwin,) + s.shape[axis+1:] - where W = s.shape[axis] - (nwin - 1). + Output shape: ``s.shape[:axis] + (W,) + (nwin,) + s.shape[axis+1:]`` + where ``W = s.shape[axis] - (nwin - 1)``. - If `step > 1`, stepping is applied by slicing along the new windows axis (same observable behavior as doing `slice_along_axis(result, slice(None, None, step), axis)` in the dense version). diff --git a/src/ezmsg/sigproc/wavelets.py b/src/ezmsg/sigproc/wavelets.py index 790d027f..4b63480d 100644 --- a/src/ezmsg/sigproc/wavelets.py +++ b/src/ezmsg/sigproc/wavelets.py @@ -1,3 +1,5 @@ +"""Continuous wavelet transform (CWT) for streaming time-frequency analysis.""" + import typing import ezmsg.core as ez @@ -170,7 +172,7 @@ def cwt( scales: The scales to use. If None, the scales will be calculated from the frequencies. Note: Scales will be sorted from largest to smallest. Note: Use of scales is deprecated in favor of frequencies. Convert scales to frequencies using - `pywt.scale2frequency(wavelet, scales, precision=10) * fs` where fs is the sampling frequency. + ``pywt.scale2frequency(wavelet, scales, precision=10) * fs`` where fs is the sampling frequency. Returns: A primed Generator object that expects an :obj:`AxisArray` via `.send(axis_array)` of continuous data diff --git a/src/ezmsg/sigproc/window.py b/src/ezmsg/sigproc/window.py index d5354ddf..02e9928d 100644 --- a/src/ezmsg/sigproc/window.py +++ b/src/ezmsg/sigproc/window.py @@ -1,3 +1,5 @@ +"""Sliding and tumbling window segmentation of streaming data.""" + import enum import traceback import typing