WIP: ICA clean raw data #296
Conversation
…ipeline into rest_annotate
|
I would suggest to remove all auto reject related code here and to focus on the rest. We can add auto reject in a next PR |
|
If this is because the checks fail, I think it would be an easy fix, but it's probably best to keep autoreject separate for now. |
| selection = epochs.selection | ||
|
|
||
| # Not sure why it is necessary to recreate epochs when we already have them | ||
| if config.no_epoching: |
There was a problem hiding this comment.
The base code recreates an epoch object, but I don't understand why. Here I just reuse the epochs created last step, not sure if that's okay
There was a problem hiding this comment.
but I don't understand why.
Because we filter the raw data again with a high-pass filter suitable for ICA. We then create new epochs from the newly-filtered data. You'd have to do the same for the "no epoching" case
There was a problem hiding this comment.
I completely missed that, thanks
| msg = f'Rejecting ICs: {ica.exclude}' | ||
| logger.info(gen_log_message(message=msg, step=5, subject=subject, |
There was a problem hiding this comment.
This should be logged for both cases, i.e. should be outside the if block
There was a problem hiding this comment.
yep that's what I intended to do, will fix
| @@ -184,7 +184,9 @@ def process(config: PathLike, | |||
| script_paths = [*SCRIPT_PATHS['init'], *script_paths] | |||
|
|
|||
| for script_path in script_paths: | |||
| step_name = script_path.name.replace('.py', '')[3:] | |||
There was a problem hiding this comment.
Changed because it did not work with file names such as recon_all.py or 04a-run_ica.py
|
I looked at how mne handles the computation of the source PSD for raw data ( We could maybe take inspiration from this? I could adapt the PR |
|
as soon as you have an epoch you're blocked to ignore/remove transient artifacts |
|
yes that's true but that will be always be a problem anyway. If I can't epoch my data because I need long recordings, then there's not much I can about it either way, unless I missed something. It would be nice to have mne functions on raw data that ignore transient artifacts based on annotations, but I don't think that's available yet? For instance if I want the compute the PSD on raw data, it does not take into account BAD annotations |
|
yes that's true but that will be always be a problem anyway. If I can't
epoch my data because I need long recordings, then there's not much I can
about it either way, unless I missed something.
honestly I am wondering how realistic this is. If you take the full camcan
data > 600 subjects how many subjects have a rest session
with no artifacts beyond flux jumps?
as discussed last friday let's do one thing at a time.
first the annotate_flux_jump function in mne
then a function to fix them by temporal interpolation
then we see how to use this in the pipeline
… |
|
As of #840 ICA (and SSP) are now also applied to raw data and saved to disk as |
This code works for one dataset, on one subject.
I will do the tests soon when I get the time.
I open this PR for feedback.
Some ironing out will need to be done, this is very much still a work in progress.