[ENH] OASIS3-to-BIDS : Add OASIS3 FTP PET (18F-AV1451)

[davidaquilue]

I have reviewed the contribution guide (after sending an email through Google Groups, sorry about that...) and I believe that this Issue is a good place to document some of the necessary choices that need to be made in order to implement the FTP dataset to the OASIS3 to BIDS converter.

Currently, the main problem that I would like to address is the difference between the "Raw" PET data in the ADNI3 dataset vs the OASIS3 dataset. In the ADNI dataset, the pre-BIDS raw data is actually the ADNI-consortia-processed data (step2, co-registered and averaged). It seems that it is not the same for the OASIS3, where the PET data (currently PiB, AV45, FDG) is directly the raw scan data (4D volumes, with different 3D temporal acquisitions).

For me this results in several problems:

• Inconsistency with the ADNI converter (probably not too important but I do not know how this may impact down-the-line PET processing pipelines from clinica).
• PET raw data requires additional and custom co-registration and averaging techniques which may over-complicate runnign the pipeline to the user who focuses on directly obtaining SUVR values without much interest in the middle steps.
• (most important for current development) While it seems that the currently implemented PET data (AV45, PiB, FDG) is already somewhat pre-selected in the raw data of the OASIS3, the FTP (AV1451) data is mostly in its rawest state, which means that we can have three different recording protocols (each with a different combination of NIFTI files/runs). The three different protocols are documented here (page 22).

I have already worked through the last issue and currently have a pipeline that (through some manual checking) can parse through the FTP raw PET scans, select the adequate runs, and co-register and average the relevant time-slices into a single NIFTI file.

I believe that it would be optimal to first apply co-registration and averaging to the adequate FTP scans, and then convert these processed images to BIDS, in a similar manner to what is done in the ADNI-to-BIDS converter. However, I am unsure of the best way of adding this pipeline to the converter.

Otherwise, we could first convert the raw 4D volumes to BIDS (as is currently done with the other tracers) and then apply the coreg-avg pipeline. I think this would bloat a bit the converter but we can find the workaround anyways from my current pipeline.

Anyways, happy to hear your thoughts!

All the best,
David

[davidaquilue]

TODO LIST
Will probably edit it during development...

• Re-download and document OASIS3-AV1451 and OASIS3-AV1451L
• Document necessary steps to prepare the AV1451 data for conversion
• Add NIFTI PET selection pipeline to repo
• Decide when / how to implement selection pipeline and coreg+avg of selected timeslices.
• Document manually checked files and decisions and how can this be implemented in the pipeline while maintaining data usage agreement.
• Generate clinical data general table with key columns from current available cohort and study data files. Consider keeping same column names to ensure validity with previous versions of the pipeline.
• Update documentation / tutorial on OASIS-3 converter.
• Evaluate BIDS converter with tau PET data and new clinical data, add small logger.

[davidaquilue]

Dear @AliceJoubert,

I have added the pipeline we previously discussed in our Google Conversation here.

I still need to test and debug the CLI entry point to run the pipeline, but I believe that the scripts can be more or less understood by themselves together with the README file.

An example of the issue this pipeline aims to solve can be found for the subject OAS30113 in session d4429. In this session's raw data, we have two different pet files:

• ses_folder/pet1/NIFTI/file_root_run-01_pet.nii.gz, with 38 temporal volumes (last one 3300s/55mins after start of recording)
• ses_folder/pet2/NIFTI/file_root_run-03_pet.nii.gz with 6 temporal volumes
  This is an example of the hybrid recording procedure (third protocol in the AV-1451 section in page 22 here) where the participant is first scanned for ~60 minutes, has a break and goes back into the scanner for the final 30 minutes recording, which is the one to be used for SUVR computation.

After some reflection, I believe that we should use part of my pipeline (steps _01*, _02*, _03*) to determine what specific runs are actually relevant to transform to BIDS (coregistration and averaging can be applied later on).
However, I understand that maybe the BIDS convention could be to convert to BIDS format all the runs, and then apply the pipeline starting from the BIDS dataset (although in ADNI we already do the co-registration and averaging step before conversion).

Will wait for a decision on this to continue the development of the FTP conversion to BIDS as this will determine what files to copy and what modifications will be necessary for the pipeline.

Best,
David

[AliceJoubert]

Hello @davidaquilue ,

Thanks for providing your pipeline ! If I understood well in your pipeline step 2 consists of a manual review to decide whether or not a session/scan will need to undergo the processing. Could you explain why this is needed ? And if you performed yourself the manual review, what were your criteria to take a decision ?

For us the best compromise with BIDS specifications would be to have the converter copy 4D images as is, then have another pipeline (yours) to do the co-registration and averaging. That way, the default output of the converter stays close to BIDS specs. In terms of user experience we could add an option to the converter to use this other pipeline right away and avoid having to type out a second command line to get the co-registered, averaged BIDS. We usually place such pipelines in the io-tools section of clinica.

Tell me how that sounds,

Best,
Alice

[davidaquilue]

Hi @AliceJoubert ,

So, as a summary, there are three different FTP PET scan protocols:
1- After injection, subject stays in the scan for 105minutes
2- After injection, subject rests quietly for 75minutes, then undergoes a 30minute PET scan
3- After injection, subject undergoes 60minute scan, then has a 15minute rest, then undergoes a 30minute PET scan 75minutes post-injection.

In the end, the important temporal slices are those of the last 30 minutes (6 slices of 5 minutes), but they have been recorded in different ways. Protocols 1 and 2 require a single PET run while Protocol 3 (problematic) requires two PET runs and therefore generates 2 NIFTI files.

Unfortunately, the .json files for some of the sessions recorded through protocol 3 are missing accurate injection times, and therefore it is not straightforward to determine what run is the one containing the 6 slices for the last 30 minutes. Also, some sessions contain only a single run with a single time-slice for some unknown reason.

Step 2 automatically detects those sessions that do not have a clearly identifiable valid PET run, and flags them for manual review. During manual review, by inspecting the images and the amount of slices / time duration, it is possible to determine whether a given run belongs to the last 30 minutes. Only ~50 PET runs / files need to be manually checked, which is a very small minority, most of them being directly identifiable through the sidecar .json files.

Anyways, I have seen that the current version of the converter directly copies AV45 and PiB PET 4D files, so I will go ahead and apply the same logic.

Then, I will adapt the coregistration pipeline to start from the BIDS format and produce the coregistered and averaged images (I assume best to output them in a derivatives folder, right?).
I will also try to code my way out of the manual review, as I believe adding some brute-force else/if statements could override the need for the manual review.

Thanks for your comments, hope I made myself more clear and will continue with development these days :)

[davidaquilue]

Sorry to bother again @AliceJoubert , I am now checking and it seems that the previous ADRC clinical files that were shared as a workaround for the newer releases of the data do not account for some of the FTP data (session date is much later than the last session in the clinical files).

I think I will try to regenerate an ADRC-like dataframe through the current data files provided, so that new users do not need to ask for the workaround. The clinical data_files provide a huge amount of variables (demographical, physical evaluation, neurological evaluation, familiar history, etc.), should I keep the selection from the ADRC here or add all the information so that the users can sift through it however they want.

Is it okay if I undergo this change also in the FTP branch, or should I create a new issue + branch?

Let me know if me asking these questions is okay or whether you prefer for me to go ahead with my considerations and you will raise issues during review.

[AliceJoubert]

Hello @davidaquilue ,

Thanks for your answer, I understand better the need for manual review now. To me if you already did the work of checking which runs are the right ones you can directly integrate this to the pipeline by hardcoding which is which as you said. Then we only need to document our choice in the OASIS3 converter documentation.

For where to put the processed data either you produce a new BIDS called something like BIDS-coregistered or you use derivatives. Depends on whether or not what is more practical for users in this case (what would you use best ?).

Yes indeed we have an issue with the ADRC clinical data and intended to fix it as the converter still asks for one and OASIS3 does not give it anymore. If you want to you are very much free to correct it. You can do this on the same branch, no problem (as long as its it different pull requests). For how to tackle it :

• either you re-create a ADRC like file if you can
• either that information is present elsewhere in the given tabular files and can be used from there (so no ADRC like file is needed)
  You can pick whichever you prefer to do (re-creation if it is).

For what clinical data to consider, I would say maybe having everything is not very practical. Is there any additional info that is not present and would be very useful ? If yes we can add it but I would be against adding everything as a default. Maybe that is a question for later (giving an option to specify which the user wants or so). Will depend on how much time you have to work on this. What do you think ?

It's ok to ask these questions, especially if it would save you some work to do and redo !

[davidaquilue]

Good morning @AliceJoubert ,

Cool, then:

• I will create a BIDS-coregistered directory for the outputs of the coregistration pipeline, as this should allow to use same pipelines as used for the ADNI BIDS dataset with minimal modifications.
• I will load the relevant tabular data and create a df that contains relevant metadata that acts as a substitute to the ADRC file. Will start from there as my first PR, then work from there to add PET TAU data.
• Will keep the same clinical data as current version. The new clinical files contain a lot of information that I agree that may be redundant to add. If I have more time to dedicate to this (which I can't promise right now), I can try to come up with a function mapper that populates the *_sessions.tsv files with clinical data from any given .csv file in the clinical data so that the user can choose to add whatever clinical data they deem relevant.

Will keep you posted on updates!

[AliceJoubert]

Perfect, thanks you for your time and work !