diff --git a/notebooks/cnmfe.ipynb b/notebooks/cnmfe.ipynb new file mode 100644 index 0000000..b098134 --- /dev/null +++ b/notebooks/cnmfe.ipynb @@ -0,0 +1,1262 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "67669023-30e9-4661-931c-4ab2c554ff12", + "metadata": { + "pycharm": { + "name": "#%%\n" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "from mesmerize_core import *\n", + "import numpy as np\n", + "from copy import deepcopy\n", + "import pandas as pd\n", + "import tifffile\n", + "from caiman.motion_correction import high_pass_filter_space\n", + "from caiman.summary_images import correlation_pnr" + ] + }, + { + "cell_type": "markdown", + "id": "4198e3da-d8d1-4a26-84ce-9907f5af8870", + "metadata": {}, + "source": [ + "**You will need `fastplotlib` installed for the visualizations**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b1546b14-2939-4077-adef-e98179f33a1c", + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "from fastplotlib import ImageWidget\n", + "from ipywidgets import VBox, IntSlider, Layout" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d6412b39-20ff-46cb-972e-52e8dcc1eb00", + "metadata": {}, + "outputs": [], + "source": [ + "pd.options.display.max_colwidth = 120" + ] + }, + { + "cell_type": "markdown", + "id": "3bac0da4-e3eb-4717-b5dc-1e34cfd79e74", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "# Paths\n", + "These are the only variables you will need to modify in this demo notebook. You will need to set the paths according to your own `caiman_data` dir path\n", + "\n", + "Explanation:\n", + "\n", + "`set_parent_raw_data_path()` - This function from `mesmerize_core` sets the **top level raw data directory**. It is generally the top level directory for your raw experimental data. This allows you to move your experiment directory structure between computers, as long as you keep everything under the parent path the same.\n", + "\n", + "For example,\n", + "\n", + "On Linux based systems if you have your experimental data in the following dir:\n", + "\n", + "`/data/my_name/exp_top_level/....`\n", + "\n", + "You could set `/data/my_name` as the \"parent raw data path\", and you can then move `exp_top_level/...` between computers.\n", + "\n", + "On windows:\n", + "\n", + "`D:/my_name/exp_top_level/...`\n", + "\n", + "You could set `D:/my_name` as the parent raw data path, and you can then move `exp_top_level/...` between computers.\n", + "\n", + "**Even on windows just use `/`, you do not have to worry about the annoying issue of `\\\\` and `\\` on windows if you use `pathlib` :D**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4ef02320-6407-4d29-833c-33e9bcd00703", + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "# for this demo set this dir as the path to your `caiman_data` dir\n", + "set_parent_raw_data_path(\"/home/kushal/caiman_data/\")" + ] + }, + { + "cell_type": "markdown", + "id": "ddde637b-598c-4d99-9c5e-38409d7634fc", + "metadata": {}, + "source": [ + "### Batch path, this is where caiman outputs will be organized\n", + "\n", + "This can be anywhere, it does not need to be under the parent raw data path.\n", + "\n", + "**We recommend using [pathlib](https://docs.python.org/3/library/pathlib.html) instead of manually managing paths as strings. `pathlib` is just a part of the Python standard library, it makes it much easier to deal with paths and saves a lot of time in the long-run! It also makes your paths compatible across operating systems.**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1e356e0e-36f7-45d3-86b6-1a25a8287f13", + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "batch_path = get_parent_raw_data_path().joinpath(\"mesmerize-cnmfe/batch.pickle\")" + ] + }, + { + "cell_type": "markdown", + "id": "4c777a10-64dd-4061-99e1-e9c9172d8bf6", + "metadata": {}, + "source": [ + "# Create a new batch\n", + "\n", + "This creates a new pandas `DataFrame` with the columns that are necessary for mesmerize. In mesmerize we call this the **batch DataFrame**. You can add additional columns relevant to your experiment, but do not modify columns used by mesmerize.\n", + "\n", + "Note that when you create a DataFrame you will need to use `load_batch()` to load it later. You cannot use `create_batch()` to overwrite an existing batch DataFrame" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c0f0faca-e5a4-4f84-8290-9e48b8f88631", + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "# create a new batch\n", + "df = create_batch(batch_path)\n", + "# to load existing batches use `load_batch()`\n", + "# df = load_batch(batch_path)" + ] + }, + { + "cell_type": "markdown", + "id": "50199499-c3f7-4aab-81ef-84913450d63c", + "metadata": {}, + "source": [ + "# View the dataframe\n", + "\n", + "It is empty with the appropriate columns for mesmerize" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "263e6e3a-51ac-4edc-ad63-29bad143684e", + "metadata": {}, + "outputs": [], + "source": [ + "df" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "38d9d814-7bde-431e-8c98-abcd6aa231cd", + "metadata": {}, + "outputs": [], + "source": [ + "movie_path = get_parent_raw_data_path().joinpath(\"example_movies/data_endoscope.tif\")" + ] + }, + { + "cell_type": "markdown", + "id": "2996e126-8f11-449b-86b3-85bbc9ff03d4", + "metadata": { + "tags": [] + }, + "source": [ + "# gSig_filt\n", + "\n", + "A high-pass spatial filter is useful for motion correction of miniscope 1p data, or other data which has large amounts of low frequency background flutuations.\n", + "\n", + "The `gSig_filt` param sets the `sigma` of the gaussian kernel used for filtering. We can use fastplotlib to visualize the effects of this parameter. We want to remove the low frequency spatial information from the image to create better template images for motion correction.\n", + "\n", + "Note that this is different from the `gSig` parameter used in CNMF!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ab5d9f48-c1c6-4366-bc5a-2137691df44f", + "metadata": {}, + "outputs": [], + "source": [ + "# get out input movie\n", + "# if it is memmapable you can use tifffile.memmap\n", + "# for other formats you can try LazyTiff, or any suitable lazy loader\n", + "input_movie = tifffile.imread(movie_path)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ab7646bd-f3e2-4058-9d67-ac43ba43c020", + "metadata": {}, + "outputs": [], + "source": [ + "# create a slider for gSig_filt\n", + "slider_gsig_filt = IntSlider(min=1, max=33, step=2, description=\"gSig_filt\")\n", + "\n", + "def apply_filter(frame):\n", + " # read slider value\n", + " gSig_filt = (slider_gsig_filt.value, slider_gsig_filt.value)\n", + " \n", + " # apply filter\n", + " return high_pass_filter_space(frame, gSig_filt)\n", + "\n", + "# we can use frame_apply feature of `ImageWidget` to apply \n", + "# the filter before displaying frames\n", + "funcs = {\n", + " # data_index: function\n", + " 1: apply_filter # filter shown on right plot, index 1\n", + "}\n", + "\n", + "# input movie will be shown on left, filtered on right\n", + "iw_gs = ImageWidget(\n", + " data=[input_movie, input_movie.copy()],\n", + " frame_apply=funcs,\n", + " vmin_vmax_sliders=True,\n", + " cmap=\"gnuplot2\"\n", + ")\n", + "\n", + "def force_update(*args):\n", + " # kinda hacky but forces the images to update \n", + " # when the gSig_filt slider is moved\n", + " iw_gs.current_index = iw_gs.current_index\n", + " iw_gs.reset_vmin_vmax()\n", + "\n", + "iw_gs.reset_vmin_vmax()\n", + " \n", + "slider_gsig_filt.observe(force_update, \"value\")\n", + "\n", + "VBox([iw_gs.show(), slider_gsig_filt])" + ] + }, + { + "cell_type": "markdown", + "id": "511192a7-6e25-49cf-8175-0ecfb0001662", + "metadata": {}, + "source": [ + "# reset vmin vmax when necessary!!!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "79cd6ec4-d7a4-4230-9996-4bfc5d27a7d1", + "metadata": {}, + "outputs": [], + "source": [ + "# the filtered images will have much lower min and max\n", + "# this resets the vmin vmax sliders on the ImageWidget\n", + "iw_gs.reset_vmin_vmax()" + ] + }, + { + "cell_type": "markdown", + "id": "cbca4882-7d04-41fa-adbb-74c1a8eb4b13", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "# Motion correction parameters\n", + "\n", + "Parameters for all algos have the following structure:\n", + "\n", + "```python\n", + "{\"main\": {... params directly passed to caiman}}\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cdf4a68d-e97f-4738-bdec-29905590af60", + "metadata": {}, + "outputs": [], + "source": [ + "params =\\\n", + "{\n", + " \"main\":\n", + " {\n", + " \"gSig_filt\": (3, 3), # a gSig_filt value that brings out \"landmarks\" in the movie\n", + " \"pw_rigid\": True,\n", + " \"max_shifts\": (5, 5),\n", + " \"strides\": (48, 48),\n", + " \"overlaps\": (24, 24),\n", + " \"max_deviation_rigid\": 3,\n", + " \"border_nan\": \"copy\",\n", + " }\n", + "}" + ] + }, + { + "cell_type": "markdown", + "id": "32cecd0f-334d-4bc1-ab85-752a25d5518b", + "metadata": { + "pycharm": { + "name": "#%% md\n" + }, + "tags": [] + }, + "source": [ + "# Add a \"batch item\", this is the combination of:\n", + "* algorithm to run, `algo`\n", + "* input movie to run the algorithm on, `input_movie_path`\n", + "* parameters for the specified algorithm, `params`\n", + "* a name for you to keep track of things, usually the same as the movie filename, `item_name`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6b7156e8-0868-4de9-af6a-b3f4959ade3a", + "metadata": {}, + "outputs": [], + "source": [ + "df.caiman.add_item(\n", + " algo=\"mcorr\",\n", + " input_movie_path=movie_path,\n", + " params=params,\n", + " item_name=movie_path.stem\n", + ")\n", + "\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "bcc2afb2-d282-4227-8bdd-fbd498dc5144", + "metadata": { + "pycharm": { + "name": "#%% md\n" + }, + "tags": [] + }, + "source": [ + "## We can now see that there is one item, a.k.a. row or pandas `Series`, in the batch dataframe, we can add another item with the same input movie but with different parameters.\n", + "\n", + "### **When adding batch items with the same `input_movie_path` (i.e. same input movie but different parameters) it is useful to give them the same `item_name`.**\n", + "\n", + "Let's just try one with different `gSig_filt`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fc683ee4-1d4d-4407-953a-82a215740fe5", + "metadata": {}, + "outputs": [], + "source": [ + "params2 =\\\n", + "{\n", + " \"main\":\n", + " {\n", + " \"gSig_filt\": (1, 1), # a gSig_filt value that brings out \"landmarks\" in the movie\n", + " \"pw_rigid\": True,\n", + " \"max_shifts\": (5, 5),\n", + " \"strides\": (48, 48),\n", + " \"overlaps\": (24, 24),\n", + " \"max_deviation_rigid\": 3,\n", + " \"border_nan\": \"copy\",\n", + " }\n", + "}\n", + "\n", + "df.caiman.add_item(\n", + " algo=\"mcorr\",\n", + " input_movie_path=movie_path,\n", + " params=params2,\n", + " item_name=movie_path.stem\n", + ")\n", + "\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "9012a748-79d5-4e4a-887f-cc8f26b86cc6", + "metadata": {}, + "source": [ + "## We can see that there are two batch items for the same input movie.\n", + "\n", + "### We can also use a `for` loop to add multiple different parameter variants more efficiently." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6b758dfd-c7af-4e9f-879e-7b3e9456dde5", + "metadata": {}, + "outputs": [], + "source": [ + "# copy the mcorr_params2 dict to make some changes\n", + "new_params = deepcopy(params)\n", + "\n", + "# some variants of max_shifts\n", + "# but this can be any params\n", + "for shifts in [1, 3, 10]: \n", + " # deep copy is the safest way to copy dicts\n", + " new_params = deepcopy(new_params)\n", + " \n", + " # assign the \"max_shifts\"\n", + " new_params[\"main\"][\"max_shifts\"] = (shifts, shifts)\n", + " \n", + " df.caiman.add_item(\n", + " algo='mcorr',\n", + " item_name=movie_path.stem,\n", + " input_movie_path=movie_path,\n", + " params=new_params\n", + " )" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4b1ffb38-4107-46bc-9a74-e3bdfcae2d14", + "metadata": {}, + "outputs": [], + "source": [ + "df" + ] + }, + { + "cell_type": "markdown", + "id": "e0b37050-9065-4b1f-96fb-753f6f33fc65", + "metadata": { + "tags": [] + }, + "source": [ + "## Now we can see that there are many parameter variants, but it is not easy to see the differences in parameters between the rows that have the same `item_name`.\n", + "\n", + "### We can use the `caiman.get_params_diffs()` extension to see the unique parameters between rows with the same `item_name`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "efd5b4e1-c8fc-40e0-9a77-a0e3403ce120", + "metadata": {}, + "outputs": [], + "source": [ + "diffs = df.caiman.get_params_diffs(algo=\"mcorr\", item_name=df.iloc[0][\"item_name\"])\n", + "diffs" + ] + }, + { + "cell_type": "markdown", + "id": "7390441c-6244-4eac-93e6-cf688b6d76fb", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "# Indexing rows and running batch item(s)\n", + "#### You can run a single batch item by calling `caiman.run()` on a `Series` (row) of the DataFrame. One way to get the row is integer indexing using `df.iloc[index]`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3c11e0c4-8dd5-4c33-a5fa-4677a6f10bde", + "metadata": {}, + "outputs": [], + "source": [ + "# get the first batch item\n", + "row = df.iloc[0]" + ] + }, + { + "cell_type": "markdown", + "id": "7ead6a13-6531-4cf4-b82e-b3aef28e4fab", + "metadata": {}, + "source": [ + "### You can see how the various `pandas.Series` extensions are accessible at the level of dataframe rows.\n", + "\n", + "Move the cursor to the end of the following line and press `Tab` on your keyboard. You can select the `caiman.run()` function and press `Shift + Tab` to see the docstring. You can also instead refer to the API docs. https://mesmerize-core.readthedocs.io/en/latest/api/common.html#mesmerize_core.CaimanSeriesExtensions\n", + "\n", + "Note tab completion doesn't work if you use `df.iloc[i].caiman.`, you need to apply the indexer to see the docstring." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3893e8e9-1130-4cb6-8434-5632afe0da41", + "metadata": {}, + "outputs": [], + "source": [ + "row.caiman." + ] + }, + { + "cell_type": "markdown", + "id": "eec440e0-7c95-463b-95bf-deab0c7d4e6f", + "metadata": {}, + "source": [ + "# Run a single batch item\n", + "\n", + "Run the row that we have selected above, on Linux & Mac it will run in subprocess but on Windows it will run in the local kernel. If using the subprocess backend you can use `run(wait=False)` to not block the current kernel when running." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "67a0d36b-ec96-410a-94f3-fafdea28d08c", + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "# run the first \"batch item\"\n", + "# this will run in a subprocess by default on Linux & Mac\n", + "# on windows it will run locally\n", + "process = row.caiman.run()\n", + "\n", + "# reload dataframe from disk when done\n", + "df = df.caiman.reload_from_disk()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "be469f61-b712-4832-b8be-5118cd59f5c6", + "metadata": { + "scrolled": true, + "tags": [] + }, + "outputs": [], + "source": [ + "for i, row in df.iterrows():\n", + " if not i > 0: # skip the first item since we've run it already\n", + " continue\n", + " process = row.caiman.run()\n", + " \n", + " # on Windows you MUST reload the batch dataframe after every iteration because it uses the `local` backend.\n", + " # this is unnecessary on Linux & Mac\n", + " # \"DummyProcess\" is used for local backend so this is automatic\n", + " if process.__class__.__name__ == \"DummyProcess\":\n", + " df = df.caiman.reload_from_disk()" + ] + }, + { + "cell_type": "markdown", + "id": "3b4c526a-777f-4769-8aa9-5e3b0459e908", + "metadata": { + "pycharm": { + "name": "#%% md\n" + }, + "tags": [] + }, + "source": [ + "# Reload the DataFrame to see the outputs information for the mcorr batch item\n", + "### It is necessary to ALWAYS use `df = df.caiman.reload_from_disk()` after running a single batch item or a loop of batch items. You must not add new batch items until you reload it if you have ran items!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fbbe5f0f-84a0-4e87-9c3e-8d922c7cfd79", + "metadata": {}, + "outputs": [], + "source": [ + "df = df.caiman.reload_from_disk()" + ] + }, + { + "cell_type": "markdown", + "id": "895120e6-49b5-4f78-99ec-7c294d640604", + "metadata": {}, + "source": [ + "## We can see that the `outputs` column has been filled in" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6fdc3d5a-7eec-490b-b53e-26aadc086559", + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "df" + ] + }, + { + "cell_type": "markdown", + "id": "5eadeee7-61e1-474a-ad33-4aac34b7b4a1", + "metadata": {}, + "source": [ + "# Visualization using `fastplotlib`\n", + "You will need `fastplotlib` installed for this, see https://github.com/kushalkolar/fastplotlib" + ] + }, + { + "cell_type": "markdown", + "id": "9da1df35-f215-482b-a70b-5b44c50ee716", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "# Get the input movie and mcorr so we can visualize them\n", + "\n", + "Note that you DO NOT need to manually work with file paths. For tiff input files it returns it as a memmaped array (if possible) with lazy loading. It will try to use a mesmerize `LazyArray` if the file cannot be memmaped.\n", + "\n", + "For other file formats you can pass a function that returns a sliceable object, ideally you want to use lazy loading." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f8a7ff3f-27e7-4b59-9e84-5ea3db83a9a7", + "metadata": {}, + "outputs": [], + "source": [ + "# you can change the index to look at the mcorr results of different batch items\n", + "index = 0\n", + "\n", + "# load input movie, this demo movie isn't memmapable and LazyTiff isn't fully read yet so we'll use tifffile.imread\n", + "input_movie = df.iloc[index].caiman.get_input_movie(tifffile.imread)\n", + "\n", + "# load mcorr output movie, this loads it as a memmap\n", + "mcorr_movie = df.iloc[index].mcorr.get_output()" + ] + }, + { + "cell_type": "markdown", + "id": "5885441e-a049-4412-9306-fb716751d6dc", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "# Visualize raw & MCorr movie side-by-side\n", + "\n", + "### fastplotlib `ImageWidget` to visualize raw & mcorr movie side by side\n", + "\n", + "`ImageWidget` assumes `\"txy\"` dimension order by default for 2D movies. You can set other orders using the `dims_order` kwarg\n", + "\n", + "### High pass filtering seems useful again to see movement more easily" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a687037f-9cad-41d1-aa63-7b34ce0b7950", + "metadata": {}, + "outputs": [], + "source": [ + "# high pass filter the data to see shifts more easily\n", + "filt = lambda x: high_pass_filter_space(x, df.iloc[index][\"params\"][\"main\"][\"gSig_filt\"])\n", + "\n", + "funcs = {\n", + " 0: filt,\n", + " 1: filt\n", + "}\n", + "\n", + "iw = ImageWidget(\n", + " [input_movie, mcorr_movie.astype(np.float32)],\n", + " vmin_vmax_sliders=True, \n", + " frame_apply=funcs, # the filter func\n", + " names=[\"input\", \"mcorr\"],\n", + " cmap=\"gnuplot2\",\n", + ")\n", + "\n", + "iw.reset_vmin_vmax()\n", + "\n", + "iw.show()" + ] + }, + { + "cell_type": "markdown", + "id": "bc864ddd-cb70-4c3d-8a78-e0b50df0abf4", + "metadata": {}, + "source": [ + "# Frame averaging with a rolling window using `ImageWidget` \"window functions\".\n", + "\n", + "## This makes it easier to visually inspect motion" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0afe7eab-a35f-49f0-931d-4d39c7a3c2d7", + "metadata": {}, + "outputs": [], + "source": [ + "iw.window_funcs = {\"t\": (np.mean, 11)}" + ] + }, + { + "cell_type": "markdown", + "id": "95b8d84a-5dd0-47e3-870a-df2f6505ced2", + "metadata": {}, + "source": [ + "## Close the canvas to free up GPU processing time, not necessary if you have a powerful GPU" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f3a0f83c-051a-4498-a578-fbefbaef3f77", + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "iw.plot.canvas.close()" + ] + }, + { + "cell_type": "markdown", + "id": "70575247-f15a-4db8-99ac-b3119763689a", + "metadata": {}, + "source": [ + "## With `ImageWidget` you can view all 5 mcorr results simultaneously!\n", + "\n", + "### This depends on your hard drive's capabilities" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a3dabee5-d4cf-473d-8dc0-934da0f423f8", + "metadata": {}, + "outputs": [], + "source": [ + "# first item is just the raw movie\n", + "movies = [df.iloc[0].caiman.get_input_movie(tifffile.imread)]\n", + "\n", + "# subplot titles\n", + "subplot_names = [\"raw\"]\n", + "\n", + "# we will use the mean images later\n", + "means = [df.iloc[0].caiman.get_projection(\"mean\")]\n", + "\n", + "# add all the mcorr outputs to the list\n", + "for i, row in df.iterrows():\n", + " # add to the list of movies to plot\n", + " movies.append(row.mcorr.get_output())\n", + " \n", + " # subplot title to show dataframe index\n", + " subplot_names.append(f\"ix: {i}\")\n", + "\n", + "# filter using the same sigma to make visualization easier\n", + "# same filter for all movies, this syntax is just dictionary comprehension\n", + "filt = {subplot_ix: lambda frame: high_pass_filter_space(frame, (1, 1)) for subplot_ix in range(len(movies))}\n", + "\n", + "# create the widget\n", + "mcorr_iw_multiple = ImageWidget(\n", + " data=movies, # list of movies\n", + " window_funcs={\"t\": (np.mean, 3)}, # window_funcs is also a kwarg\n", + " vmin_vmax_sliders=True,\n", + " frame_apply=filt, # same func for all\n", + " names=subplot_names, # subplot names used for titles\n", + " cmap=\"gnuplot2\"\n", + ")\n", + "\n", + "mcorr_iw_multiple.reset_vmin_vmax()\n", + "\n", + "mcorr_iw_multiple.show()" + ] + }, + { + "cell_type": "markdown", + "id": "8145967d-5fab-491c-bf6e-f304cb95eb72", + "metadata": {}, + "source": [ + "# reset vmin vmax if necessary " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "583b30f8-50eb-41b2-82bf-1973bfe8c00e", + "metadata": {}, + "outputs": [], + "source": [ + "mcorr_iw_multiple.reset_vmin_vmax()" + ] + }, + { + "cell_type": "markdown", + "id": "9ad69731-e057-4ad7-acf7-dd61269ecd72", + "metadata": {}, + "source": [ + "# All the movies here look pretty good so I'll continue with `index = 0`. You can cleanup the DataFrame and remove all other items.\n", + "\n", + "### You can remove batch items (i.e. rows) using `df.caiman.remove_item()`\n", + "\n", + "**Note that this also cleans up the output data in the batch directory!**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6177e29e-c6af-4ead-8925-bad600e797b2", + "metadata": {}, + "outputs": [], + "source": [ + "# make a list of rows we want to keep using the uuids\n", + "rows_keep = [df.iloc[3].uuid]\n", + "rows_keep" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3e661d4e-989b-4478-9aeb-42520fd78531", + "metadata": {}, + "outputs": [], + "source": [ + "for i, row in df.iterrows():\n", + " if row.uuid not in rows_keep:\n", + " df.caiman.remove_item(row.uuid)\n", + "\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "3c80220c-73ce-40c8-83b5-add404b805ad", + "metadata": {}, + "source": [ + "# CNMF\n", + "## corr-pnr seeding\n", + "\n", + "This visualization is to help determine values for `min_corr` (correlation) and `min_pnr` (peak to noise ratio) for seeding CNMFE. Pixels below these thresholds will be excluded from the results.\n", + "\n", + "If `correlation_pnr` takes a long time you can increase the subsample to make it larger than `2`. Example: `mcorr_movie[::5]`\n", + "\n", + "You should try different values of `gSig`, this is different from `gSig_filt`. You will use this gSig as a CNMFE param as well." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "02472f11-ec6c-4c02-9c6e-38620f65773b", + "metadata": {}, + "outputs": [], + "source": [ + "gSig = 3\n", + "corr, pnr = correlation_pnr(mcorr_movie[::2], gSig=gSig, swap_dim=False)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d4b657a1-25cf-422b-a18c-202d4d9eb72f", + "metadata": {}, + "outputs": [], + "source": [ + "# to show the correlation and pnr images\n", + "iw_corr_pnr = ImageWidget(\n", + " [corr, pnr], \n", + " names=[\"corr\", \"pnr\"],\n", + " vmin_vmax_sliders=True, \n", + " cmap=\"turbo\"\n", + ")\n", + "\n", + "# some slider stuff\n", + "iw_corr_pnr.vmin_vmax_sliders[0].description = \"corr\"\n", + "iw_corr_pnr.vmin_vmax_sliders[1].description = \"pnr\"\n", + "\n", + "iw_corr_pnr.vmin_vmax_sliders[0].max = corr.max()\n", + "iw_corr_pnr.vmin_vmax_sliders[1].max = pnr.max()\n", + "\n", + "# mcorr vids, we will display thresholded mcorr vids\n", + "mcorr_vids = [mcorr_movie.astype(np.float32) for i in range(4)]\n", + "\n", + "# sync the threshold image widget with the corr-pnr plot\n", + "grid_plot_kwargs = {\n", + " \"controllers\": [[iw_corr_pnr.plot[\"corr\"].controller]*2]*2\n", + "}\n", + "\n", + "iw_thres_movie = ImageWidget(\n", + " mcorr_vids, \n", + " names=[\"> corr\", \"> pnr\", \"< corr\", \"< pnr\"],\n", + " vmin_vmax_sliders=True,\n", + " # sync this with the corr-pnr plot\n", + " grid_plot_kwargs=grid_plot_kwargs,\n", + " cmap=\"gnuplot2\"\n", + ")\n", + "\n", + "# display threshold of the spatially filtered movie\n", + "def spatial_filter(frame):\n", + " f = high_pass_filter_space(frame, (3, 3))\n", + " return f\n", + "\n", + "\n", + "# threshold\n", + "def threshold(frame, mask):\n", + " # optionally use spatial filter\n", + " t = spatial_filter(frame)\n", + " \n", + " t = t.copy()\n", + " \n", + " t[mask] = t.min()\n", + " \n", + " return t\n", + "\n", + "# dict of threshold lambda wrappers to set on ImageWidget\n", + "# this sets the frame_apply for each subplot\n", + "threshold_funcs = {\n", + " 0: lambda frame: threshold(frame, corr < iw_corr_pnr.vmin_vmax_sliders[0].value[0]),\n", + " 1: lambda frame: threshold(frame, pnr < iw_corr_pnr.vmin_vmax_sliders[1].value[0]),\n", + " 2: lambda frame: threshold(frame, corr > iw_corr_pnr.vmin_vmax_sliders[0].value[0]),\n", + " 3: lambda frame: threshold(frame, pnr > iw_corr_pnr.vmin_vmax_sliders[1].value[0])\n", + "}\n", + "\n", + "# set the dict of lambda wrappers\n", + "iw_thres_movie.frame_apply = threshold_funcs\n", + "\n", + "# hacky way to force the threshold plots to update\n", + "# when the corr pnr sliders move\n", + "def force_update_thresholds_plots(*args):\n", + " iw_thres_movie.current_index = iw_thres_movie.current_index\n", + "\n", + "iw_corr_pnr.vmin_vmax_sliders[0].observe(force_update_thresholds_plots)\n", + "iw_corr_pnr.vmin_vmax_sliders[1].observe(force_update_thresholds_plots)\n", + "\n", + "force_update_thresholds_plots()\n", + "\n", + "# iw_thres_movie.reset_vmin_vmax()\n", + " \n", + "iw_corr_pnr.plot.canvas.set_logical_size(650, 300)\n", + "iw_thres_movie.plot.canvas.set_logical_size(650, 600)\n", + "\n", + "VBox([iw_corr_pnr.show(), iw_thres_movie.show()])" + ] + }, + { + "cell_type": "markdown", + "id": "851a1982-5503-4327-acd2-50ced4f80f2a", + "metadata": {}, + "source": [ + "# reset vmin vmax when necessary!!!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ac9d5314-462c-4b86-8ce3-fd674cb0811f", + "metadata": {}, + "outputs": [], + "source": [ + "iw_thres_movie.reset_vmin_vmax()" + ] + }, + { + "cell_type": "markdown", + "id": "d66b40c7-72f6-40d2-97e4-29811d31e532", + "metadata": {}, + "source": [ + "### corr and pnr values from the plot" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a3421b7c-b838-4435-b6f4-b8d9559eb5a1", + "metadata": {}, + "outputs": [], + "source": [ + "corr_pnr = {\n", + " 'min_corr': iw_corr_pnr.vmin_vmax_sliders[0].value[0], # corr value from previous plot\n", + " 'min_pnr': iw_corr_pnr.vmin_vmax_sliders[1].value[0], # PNR value from previous plot\n", + "}\n", + "corr_pnr" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "257a7c96-c5ce-4f66-9a91-33c26fa8b1a7", + "metadata": {}, + "outputs": [], + "source": [ + "params_cnmfe =\\\n", + "{\n", + " \"main\":\n", + " {\n", + " 'method_init': 'corr_pnr', # use this for 1 photon\n", + " 'K': None,\n", + " 'gSig': (gSig, gSig),\n", + " 'gSiz': (4 * gSig + 1, 4 * gSig + 1),\n", + " 'merge_thr': 0.7,\n", + " 'p': 1,\n", + " 'tsub': 2,\n", + " 'ssub': 1,\n", + " 'rf': 40,\n", + " 'stride': 20,\n", + " 'only_init': True, # set it to True to run CNMF-E\n", + " 'nb': 0,\n", + " 'nb_patch': 0,\n", + " 'method_deconvolution': 'oasis', # could use 'cvxpy' alternatively\n", + " 'low_rank_background': None,\n", + " 'update_background_components': True, # sometimes setting to False improve the results\n", + " 'normalize_init': False, # just leave as is\n", + " 'center_psf': True, # leave as is for 1 photon\n", + " 'ssub_B': 2,\n", + " 'ring_size_factor': 1.4,\n", + " 'del_duplicates': True, # whether to remove duplicates from initialization\n", + " **corr_pnr # unpack corr_pnr vals into here\n", + " }\n", + "}" + ] + }, + { + "cell_type": "markdown", + "id": "fe711552-fb1c-4f3d-8098-6967a1ca46cf", + "metadata": {}, + "source": [ + "### Add a single cnmf item to the batch\n", + "\n", + "When you use `algo=\"cnmfe\"`, it basically forces the following parameters:\n", + "```python\n", + "\"method_init\": \"corr_pnr\",\n", + "\"n_processes\": n_processes,\n", + "\"only_init\": True, # for 1p\n", + "\"center_psf\": True, # for 1p\n", + "\"normalize_init\": False, # for 1p\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "736afb50-7090-4433-bbb8-28b924da57d9", + "metadata": {}, + "outputs": [], + "source": [ + "df.caiman.add_item(\n", + " algo=\"cnmfe\",\n", + " input_movie_path=df.iloc[0],\n", + " params=params_cnmfe,\n", + " item_name=df.iloc[0][\"item_name\"]\n", + ")\n", + "\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "102e295a-5543-4ad3-aaa5-fb76e819950a", + "metadata": {}, + "source": [ + "### Just like with motion correction, we can use loops to add multiple parameter variants. This is useful to perform a parameter search to find the params that work best for your dataset. Here I will use `itertools.product` which is better than deeply nested loops." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c9910214-ca37-47c2-b309-76623113616b", + "metadata": {}, + "outputs": [], + "source": [ + "from itertools import product\n", + "\n", + "# variants of several parameters\n", + "# you can make lists for as many params as you want\n", + "K_variants = [None, 10]\n", + "merge_thr_variants = [0.6, 0.8, 0.9, 0.98]\n", + "\n", + "# always use deepcopy like before\n", + "new_params_cnmf = deepcopy(params_cnmfe)\n", + "\n", + "# create a parameter grid\n", + "# product is a nice way to create all combinations of multiple iterables like lists\n", + "parameter_grid = product(K_variants, merge_thr_variants)\n", + "\n", + "# a single for loop to go through all the various parameter combinations\n", + "for K, merge_thr in parameter_grid:\n", + " # deep copy params dict just like before\n", + " new_params_cnmf = deepcopy(new_params_cnmf)\n", + " \n", + " # one set of parameter combinations\n", + " new_params_cnmf[\"main\"][\"K\"] = K\n", + " new_params_cnmf[\"main\"][\"merge_thr\"] = merge_thr\n", + " \n", + " # add param combination variant to batch\n", + " df.caiman.add_item(\n", + " algo=\"cnmfe\",\n", + " item_name=df.iloc[0][\"item_name\"],\n", + " input_movie_path=df.iloc[0],\n", + " params=new_params_cnmf\n", + " )" + ] + }, + { + "cell_type": "markdown", + "id": "162090e3-5298-4843-a4cb-108d3ad19631", + "metadata": {}, + "source": [ + "### See that there are a lot of new cnmf batch items" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "93531399-5b80-4c76-ae28-c8156a84a1d6", + "metadata": {}, + "outputs": [], + "source": [ + "df" + ] + }, + { + "cell_type": "markdown", + "id": "aca2e87c-76ce-4fa3-93b4-146aad957aa9", + "metadata": {}, + "source": [ + "## Since it is difficult to see the different parameter variants above, we can just view the diffs\n", + "\n", + "### The index numbers on the diffs correspond to the indices in the parent DataFrame above" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "737f0066-ea1e-4974-8fb2-9186618c9e48", + "metadata": {}, + "outputs": [], + "source": [ + "df.caiman.get_params_diffs(algo=\"cnmfe\", item_name=df.iloc[1][\"item_name\"])" + ] + }, + { + "cell_type": "markdown", + "id": "4e26265c-5cb7-461f-8e89-5744e2f5e82d", + "metadata": {}, + "source": [ + "# Run the added CNMFE items\n", + "\n", + "### First, this is how you can filter a pandas DataFrame using multiple columns. This gives you the rows (batch items) using the \"cnmf\" `\"algo\"` and those that match a particular `\"item_name\"`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c11826e5-1409-4411-9617-84f6408b4bdd", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "df[\n", + " (df[\"algo\"] == \"cnmfe\") & # algo\n", + " (df[\"item_name\"] == df.iloc[0][\"item_name\"]) # item name\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f8f67319-9343-41a6-a4f8-30adbdae032a", + "metadata": { + "scrolled": true, + "tags": [] + }, + "outputs": [], + "source": [ + "for i, row in df[\n", + " (df[\"algo\"] == \"cnmfe\") &\n", + " (df[\"item_name\"] == df.iloc[0][\"item_name\"])\n", + "].iterrows():\n", + " \n", + " process = row.caiman.run()\n", + " \n", + " # on Windows you MUST reload the batch dataframe after every iteration because it uses the `local` backend.\n", + " # this is unnecessary on Linux & Mac\n", + " # \"DummyProcess\" is used for local backend so this is automatic\n", + " if process.__class__.__name__ == \"DummyProcess\":\n", + " df = load_batch(df.paths.get_batch_path())" + ] + }, + { + "cell_type": "markdown", + "id": "c339a43e-a8bc-4f04-abb7-a31e226f7803", + "metadata": { + "pycharm": { + "name": "#%% md\n" + }, + "tags": [] + }, + "source": [ + "### We now have CNMF outputs" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0bb66e97-513a-4012-861a-61ac77d9f4f4", + "metadata": {}, + "outputs": [], + "source": [ + "df = df.caiman.reload_from_disk()\n", + "df[df[\"algo\"] == \"cnmfe\"]" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}