Increasing the relevance of MONAI to CAI (computer assisted intervention).

[charliebudd]

MONAI has become widely used in the field of medical imaging, thanks primarily to its ability to help process 3D imaging modalities common/unique to the field.

However, a large part of the medical engineering community doesn't deal with such modalities. For instance, contributors to the field of computer assisted intervention work far more commonly with 2D natural images such as in endoscopic computer vision, or with other modalities more specific to medical application such as ultrasound. While it is possible to use MONAI with these modalities, there is currently little value added over, for example, using torchvision for 2D natural images.

I believe there is significant scope for improved support in these areas which would help broaden MONAI’s adoption across the community and accelerate development in these fields. This is an exciting opportunity to identify pain points within the broader medical engineering community that can best be solved by a centralised movement such as MONAI.

After discussion with @ericspod, he suggested I capture some ideas on here. I will start us off with a few ideas below but I wanted to leave things open to other perspectives so please do chime in if you have any ideas.

[charliebudd]

Task standardization and benchmarking

As collection of interventional data requires integration into live operating theaters and clinics, the data landscape has developed in a challenging way. We have many disparate and small datasets, each with there own distribution biases and annotation schemas. This means many models developed on these datasets tend to be too specific to the training data. This is either by construction, such as a model which predicts very specific tool classes that don't align with classes on other datasets, or just by over fitting to the feature distribution of the small training dataset.

MONAI's position as a centralised body could help solve these problems in two ways.

Firstly, MONAI could provide a standardized label schema for various common tasks. This would involve deciding on a best approach for labeling various task, converting or relabeling datasets to match the schema, and finally disseminating this information so new datasets are annotated to match.

Secondly, MONAI could host blind testing servers for benchmarking models in common tasks in interventional AI. Due to the size and biases of our datasets, many evaluations tell us little about the generalisability of models. A benchmarking server would provide a reliable way to evaluate our models on unseen data. This would involve collection of novel data, annotation, and hosting.

[charliebudd]

Flexible pythonic realtime edge streaming

As methods in computer intervention mature, deployment in clinical trials will become a more significant part of the work in research labs. This requires low latency realtime streaming from different devices and sensors, development of which can take significant time investment. Existing infrastructure for production ready edge inference like gstreamer/deepstream, coupled with TensorRT for example, have significant learning curves and can be fiddly to setup and modify, making them a poor choice for academic environments where projects and the people working on them change rapidly. A trade off of a small amount of additional latency for a more flexible streaming solution would be beneficial to labs focused on realtime edge inference.

I have a prototype solution I developed for a few projects at KCL. It aims to provide a opencv-like python interface while being pytroch and cuda first.
https://github.com/charliebudd/torchbeam

[aylward]

This is outstanding! It is well motivated and has broad potential.

One idea, we are forming subgroups in the Ultrasound WG. There are two potential subgroups that would be interested in this work: (1) Robotics for US and (2) DICOM and Direct (where Direct refers to the direct transfer of data from a probe).

If you are interested and willing, you can join the Ultrasound WG: https://groups.google.com/u/0/g/monai-wg-ultrasound

You can then indicate your interest in these subgroups by filing out this form: https://docs.google.com/forms/d/e/1FAIpQLSdYT_Ox483pCbbxOLnbE_GfeHJL8X5bGYs9-BhxKQqYUIHQoQ/viewform?usp=dialog

Thanks for considering this!