You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Lab phylosophy: do not lose track of code and data aka use version control (git/github, datalad, etc.) and universal data format (BIDS).
4
4
5
-
In the lab we (try) to stick to version control for code with git/github and data with datalad and gin. Somewhere else is already mentioned but repetita iuvant so
5
+
CPP member to ask for help: Marco Barilari
6
6
7
-
1. Install git on your computer
8
-
2. Create a GITHUB account and ask to be added to the lab organization [https://github.com/cpp-lln-lab]
9
-
3. Make sure you have your SSH key for GITHUB following this tutorial
10
-
4. Have a look at this doc to have a first idea on how to contribute on github
11
-
5. Have a look around cause we already have a lot of ready to use code ;)
12
-
6. Make sure you are set up with the following tools having a look at this doc’s first section
13
-
1. Datalad (in a nutshell git for data but checkout more here https://handbook.datalad.org/en/latest/)
14
-
2. Gin (host server for data, good for “unlimited” and free cloud back and private and public data sharing)
15
-
7. Ask to be added to the Gin organization of the lab to start storing your data there
7
+
## Version contro tools on a "daily" basis
16
8
17
-
## Data management
18
-
for MRI data the lab is moving towards using the BIDS data structure
9
+
### Code aka git/github
19
10
20
-
* bids starter kit
21
-
* bids specification
11
+
1.[Install git](https://git-scm.com/downloads).
12
+
2. Create a Github account and ask to be added to the [CPP-lln-lab organization](https://github.com/cpp-lln-lab).
22
13
23
-
### Official GIN/DATALAD CPP dataset organization
14
+
!!! TIP
15
+
**Github account name**
24
16
17
+
Better to chose a full name account name 'davidbowie' than a nickname 'ziggysturdust', collegues and and other researchers or your future supervisor that might be interested in your code will have a hard time to find you while using your cool nickname.
25
18
26
-
[WIP] Checklist to make an official “raw” repo (bids, bids-validator checked, standard CPP data organization (eg group naming, task naming, readme with link to published/preprint, etc))
19
+
3. Make sure you have your SSH key for GITHUB following this [tutorial](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent).
20
+
4. Have a look at this doc to have a first idea on how to contribute on [github](https://github.com/cpp-lln-lab/.github).
21
+
5. Have a look around cause we already have a lot of ready-to-use code [here in the CPP github organization](https://github.com/cpp-lln-lab) and [here where we store "not-so-organized" cod in the CPP brewery](https://github.com/cpp-lln-lab/CPP_brewery) ;)
27
22
28
-
[WIP - Just dropping useful links here for now]
23
+
### Data aka Datalad and G-node GIN
24
+
25
+
1.[Install Datalad](https://handbook.datalad.org/en/latest/intro/installation.html): in a nutshell it is "git for data", checkout more at the offical [datalad handbook](https://handbook.datalad.org/en/latest/).
26
+
2. Create an account on [G-node GIN](https://gin.g-node.org/): GIN is a host server for Neuro Imaging data, good for “unlimited” and free cloud back-up and private and public data sharing.
27
+
3. Ask to be added to the [GIN CPP organization](https://gin.g-node.org/cpp-lln-lab) and the [GIN CPP Brewery](https://gin.g-node.org/cpp_brewery) of the lab to start storing your data there.
28
+
4. Check this [datalat crush course by the CPP lab](https://github.com/cpp-lln-lab/datalad_crash_course)
29
+
30
+
## Data storage ("daily" use) and long standing back ups
31
+
32
+
Since it is important to not lose our important data, it is advised to have several back-ups stored in different places.
33
+
34
+
### Brain Imaging Data Structure - BIDS
35
+
36
+
For MRI, EEG, etc. data, the lab is (moving towards) using the BIDS data structure. For more information check the [bids specification](https://bids-specification.readthedocs.io/en/stable/).
37
+
38
+
Here below some in-house tools in Github repos that helped us to convert the data into BIDS format and maintain them:
[WIP] Checklist to make an official “CPP BIDS raw” repo using BIDS:
48
+
49
+
- bids-validator checked
50
+
- standard CPP data organization (eg group naming, task naming, readme with link to published/preprint, etc)
51
+
- add source data eg dicoms and code to convert them
52
+
- add stimulation code used while colleting data (eg task eeg and fmri)
53
+
- add pdf of the pdf for MRI sequence
54
+
55
+
### Datalad/Gin/Github CPP dataset organization
29
56
30
57
Data:
31
58
32
-
[GIN organization](https://gin.g-node.org/cpp-lln-lab) for source, raw, and preprocessed data for conversion control and sharing
59
+
[CPP GIN organization](https://gin.g-node.org/cpp-lln-lab) for source, raw, and preprocessed data for version control and sharing. Repos could be public or private.
60
+
[CPP brewerey](https://gin.g-node.org/cpp_brewery) for data not ready to be officially distributed (like sandbox repos or data still under processing)
33
61
34
62
Dataset summary that puts together all the bidsified raw and derivatives repos of the lab and summarizes metadata in a table from the GIN repo above:
35
63
36
64
-[website with summary table](https://cpp-lln-lab.github.io/CPP_Datasets/index.html)
Datalad superdataset to grab all the repo together (these are submodules in the [Githuba repo CPP_Datasets](https://github.com/cpp-lln-lab/CPP_Datasets):
67
+
Github/Datalad superdataset to grab all the repo together. These repos are submodules in the [Githuba repo CPP_Datasets](https://github.com/cpp-lln-lab/CPP_Datasets):
1. Share the link to this guidline [URL to be added one this page is rendered].
79
+
2. Once received the GIN account, add this person to the specific repo to be shared. Open the repo, Settings > Collaboration > Add the user name to the empty space and click on "Add New Collaborato". (Note to self: check if it works/is possible to add only for read mode.)
80
+
3. Inform about this so that the external collaborato can now download the data.
53
81
82
+
For the external collaborator (copied from above):
54
83
84
+
1.[Install Datalad](https://handbook.datalad.org/en/latest/intro/installation.html): in a nutshell it is "git for data", checkout more at the offical [datalad handbook](https://handbook.datalad.org/en/latest/).
85
+
2. Create an account on [G-node GIN](https://gin.g-node.org/): GIN is a host server for Neuro Imaging data, good for “unlimited” and free cloud back-up and private and public data sharing.
86
+
3. Share your gin account to the person in charge to share the data with you and ask to be added as a "collaborator" to the specific repos you want to be share with you
87
+
4. Use datalad to get the data, use this script below by changing paths anbd URL:
6. Add the opened folder as a favorite (top of the right panel)
103
+
# Download all the data
104
+
datalad get . -J 2
105
+
106
+
# OR download only the anatomical data (assuming there is no session folder)
107
+
datalad get */anat -J 2
108
+
109
+
# OR downlaod only the fmri data for a specific task (e.g. 'rest') and the antomical (assuming there is no session folder)
110
+
datalad get */func/*rest* get */anat -J 2
111
+
```
112
+
113
+
5. In case you want to go out of the datalad "environment" (read: avoid headaches with locked/not writeable/linked files/etc), after download copy the files out of the datalad folder and use the data as sandard files
73
114
115
+
This will copy the actual files, in case you downloade only part of the data you will see that the whole dataset has been cleaned but the not downloade files are "empty". Just manually clean them.
116
+
117
+
```bash
118
+
fodler_datalad=
119
+
120
+
folder_no_datalad=
121
+
122
+
mkdir -p $folder_no_datalad
123
+
124
+
cd$fodler_datalad
125
+
126
+
# copy the datalad fodler content one by one, we do not copy the whole fodler at once cause there are hidden git folder that we do not want
127
+
cp -L -vr sub*$folder_no_datalad
128
+
cp -L -v *.tsv $folder_no_datalad
129
+
cp -L -v *.md $folder_no_datalad
130
+
cp -L -v *.json $folder_no_datalad
131
+
132
+
chmod -R 755 $folder_no_datalad
133
+
```
134
+
135
+
### IONS cloud storage
136
+
137
+
Make sure to be connected to the UCLouvain network, if uot side the office first connect with the VPN.
74
138
75
139
Short guidelines:
76
140
@@ -80,17 +144,13 @@ Short guidelines:
80
144
- Add README files to help the future user understand what’s inside that folder (e.g. dicoms from the study XXX published in XXX code for conversion/analyses here github.com/xxx, copy of this dataset is also on gin.g-node.org/xxx)
81
145
- What NOT to put there? Should not become a data dump so not just a new space where a user can free up her/his hard disk
82
146
147
+
Chek the [SSS server wiki](http://sss-intranet.icp.ucl.ac.be/wiki/index.php/Storage_access#Groups)
83
148
149
+
Here below, the cooked step by step process for MacOS users to connet to the CPP server folder:
0 commit comments