Documentation and guides on large binary

[jpambrun]

I am struggling to understand how to best move large amounts of data to the front end. I am trying to build a medial image viewer and a typical CT scan would weight in at about 500MB of numpy array. Experimenting quickly I notice this data can be transferred in one of two ways: 1) if it's set within the constructor, it will be sent as html via the websocket in the format data-initial-value=pixel_data": "\u00ee\u00bf\u0012Q\u001dE\... which appear to be 6-7 times heavier than the initial data, or 2) if updated later, it will be base64 encoded and send via websocket which is 20%+ less efficient and compute heavy.

Those two approaches appear very suboptimal for large 500MB+ arrays. Are those the only ways? If websocket is the only way, it would make more sense to send a binary frame of the data on its own when a buffer is above say 100KB. Alternatively, plain http transfer might be best.

[manzt]

Hi there! Could you clarify which environment you're experimenting in (e.g., JupyterLab, VS Code, or another platform)? The behavior you described likely depends on the host platform's implementation supporting anywidget.

In Jupyter-like environments, binary data is sent directly to the frontend without base64 encoding (since websockets support binary data transfer). Many widgets in the anywidget community gallery (e.g., Altair, Mosaic, Longboard) efficiently transfer large binary data. For example, I've worked on vizarr, a zarr-based image viewer.

I recommend checking out the anywidget video, which includes a section on sending binary data (a numpy array) to the frontend. The written materials are in the SciPy 2024 anywidget tutorial (notebooks/ipymario.ipynb), which you can follow along to learn more.

[jpambrun]

I am using marimo. I did watch that first video before posting. I am using the Bytes traits and the tobytes function. This is how I observed the above. I did watch the scipy video as well a few weeks ago. This is what got me excited about anywidget in the first place.

Here is a minimum working example, it should transfoer 80kb of binary data:

import marimo

__generated_with = "0.10.6"
app = marimo.App(width="medium")


@app.cell
def _():
    import anywidget
    import traitlets
    import marimo as mo
    import numpy as np


    class CounterWidget(anywidget.AnyWidget):
        _esm = """
           function render({ model, el }) {}
            export default { render }; 
        """
        _css = ""
        pixel_data = traitlets.Bytes().tag(sync=True)

        def __init__(self, pixel_data=b""):
            self.pixel_data = pixel_data
            super().__init__()

    pixel_data = np.random.default_rng().standard_normal(10000).tobytes()
    widget = mo.ui.anywidget(CounterWidget(pixel_data=pixel_data))
    widget
    return CounterWidget, anywidget, mo, np, pixel_data, traitlets, widget


@app.cell
def _(np, widget):
    widget.pixel_data = np.random.default_rng().standard_normal(10000).tobytes()
    return


if __name__ == "__main__":
    app.run()