22
33This document describes how ReproNim/containers can leverage the refactored datalad-container architecture.
44
5+ STATUS: Very rough draft!
6+
57---
68
79## New Capabilities
@@ -26,6 +28,10 @@ With native format storage and versioned image directories, ReproNim can provide
2628 └── image/
2729```
2830
31+ ** Image naming:** Use ` name:version ` format (like Docker tags):
32+ - ` mriqc:23.1.0 ` → ` .datalad/containers/images/mriqc/23.1.0/image/ `
33+ - ` fmriprep:24.1.0 ` → ` .datalad/containers/images/fmriprep/24.1.0/image/ `
34+
2935### Benefits of OCI format for ReproNim:
3036
3137- ** Layer deduplication** - Many neuroimaging containers share base layers
@@ -37,26 +43,51 @@ With native format storage and versioned image directories, ReproNim can provide
3743
3844## 2. Execution Profiles
3945
40- ReproNim ships curated base profiles alongside images. Users extend these for their specific needs.
46+ ReproNim can ship curated execution profiles alongside images. Users extend these for their specific needs.
47+
48+ ### Available Placeholders
49+
50+ - ` {img} ` - Docker image name (` datalad-container/mriqc:23.1.0 ` )
51+ - ` {img_path} ` - OCI directory path (` .datalad/containers/images/mriqc/23.1.0/image ` )
52+ - ` {cmd} ` - command arguments
4153
4254### ReproNim Base Profiles
4355
56+ ** Option A: Profiles with image (ready to use)**
57+
4458``` yaml
4559# .datalad/containers/profiles/mriqc.yaml
4660# Base MRIQC profile - sane defaults for most users
4761
48- image : mriqc/ 23.1.0
49- exec : apptainer exec --cleanenv {img } {cmd}
62+ image : mriqc: 23.1.0
63+ exec : apptainer exec --cleanenv oci:{img_path } {cmd}
5064` ` `
5165
5266` ` ` yaml
5367# .datalad/containers/profiles/fmriprep.yaml
5468# Base fMRIPrep profile
5569
56- image : fmriprep/ 23.2.0
57- exec : apptainer exec --cleanenv {img } {cmd}
70+ image : fmriprep: 23.2.0
71+ exec : apptainer exec --cleanenv oci:{img_path } {cmd}
5872` ` `
5973
74+ **Option B: Runtime-only profiles (user specifies image)**
75+
76+ ` ` ` yaml
77+ # .datalad/containers/profiles/apptainer-default.yaml
78+ # No image - user must provide --image
79+
80+ exec : apptainer exec oci:{img_path} {cmd}
81+ ` ` `
82+
83+ ` ` ` yaml
84+ # .datalad/containers/profiles/docker-default.yaml
85+
86+ exec : docker run --rm --user $(id -u):$(id -g) -v $(pwd):/work -w /work {img} {cmd}
87+ ` ` `
88+
89+ Usage: ` datalad containers-run --profile apptainer-default --image mriqc:23.1.0 ...`
90+
6091# ## User Extensions
6192
6293Users create their own profiles that extend ReproNim's base :
@@ -65,17 +96,17 @@ Users create their own profiles that extend ReproNim's base:
6596# my-analysis/.datalad/containers/profiles/mriqc-mylab.yaml
6697
6798extends: inputs/containers/.datalad/containers/profiles/mriqc.yaml
68- exec : apptainer exec --cleanenv --nv --bind /scratch:/scratch --bind /data/mylab:/input {img } {cmd}
99+ exec: apptainer exec --cleanenv --nv --bind /scratch:/scratch --bind /data/mylab:/input oci:{img_path } {cmd}
69100` ` `
70101
71102` ` ` yaml
72103# my-analysis/.datalad/containers/profiles/mriqc-gpu.yaml
73104
74105extends: inputs/containers/.datalad/containers/profiles/mriqc.yaml
75- exec : apptainer exec --cleanenv --nv {img } {cmd}
106+ exec: apptainer exec --cleanenv --nv oci:{img_path } {cmd}
76107` ` `
77108
78- **Key point:** ReproNim provides the base. Users clobber ` exec` with their environment-specific settings. No runtime-specific profiles (apptainer-gpu, podman-default, etc.) - users know what they need .
109+ **Key point:** ReproNim provides the base. Users clobber `exec` with their environment-specific settings (clobber semantics - child completely replaces parent's exec, no merging) .
79110
80111---
81112
@@ -129,12 +160,14 @@ Provenance (source URL, digest, fetch time) is stored in git commits, not separa
129160datalad clone https://github.com/ReproNim/containers inputs/containers
130161
131162# List available images and profiles
132- datalad containers-images -d inputs/containers
163+ datalad containers-list -d inputs/containers
133164datalad containers-profiles -d inputs/containers
134165
135- # Run with base profile
136- datalad containers-run -d inputs/containers --profile mriqc \
137- mriqc /bids /outputs participant
166+ # Run with base profile (profile specifies image)
167+ datalad containers-run --profile mriqc -- mriqc /bids /outputs participant
168+
169+ # Or use runtime-only profile with explicit image
170+ datalad containers-run --profile apptainer-default --image mriqc:23.1.0 -- mriqc /bids /outputs participant
138171```
139172
140173### Creating a Lab-Specific Profile
@@ -144,34 +177,40 @@ datalad containers-run -d inputs/containers --profile mriqc \
144177mkdir -p .datalad/containers/profiles
145178cat > .datalad/containers/profiles/mriqc-mylab.yaml << 'EOF '
146179extends: inputs/containers/.datalad/containers/profiles/mriqc.yaml
147- exec: apptainer exec --cleanenv --nv --bind /scratch:/scratch --bind /gpfs/mylab:/data {img } {cmd}
180+ exec: apptainer exec --cleanenv --nv --bind /scratch:/scratch --bind /gpfs/mylab:/data oci:{img_path } {cmd}
148181EOF
149182
183+ # Save to dataset
184+ datalad save -m " Add mriqc-mylab profile" .datalad/containers/profiles/
185+
150186# Use it
151- datalad containers-run --profile mriqc-mylab \
152- mriqc /data/bids /data/outputs participant
187+ datalad containers-run --profile mriqc-mylab -- mriqc /data/bids /data/outputs participant
153188```
154189
155190### One-Off Override
156191
157192``` bash
158- # Use base profile but override exec for this run
159- datalad containers-run -d inputs/containers --profile mriqc \
160- --exec " apptainer exec --cleanenv --nv {img} {cmd}" \
161- mriqc /bids /outputs participant
193+ # Use base profile but override exec for this run (adds GPU support)
194+ datalad containers-run --profile mriqc \
195+ --exec " apptainer exec --cleanenv --nv oci:{img_path} {cmd}" \
196+ -- mriqc /bids /outputs participant
197+
198+ # Use base profile but override image (use newer version)
199+ datalad containers-run --profile mriqc --image mriqc:24.0.0 \
200+ -- mriqc /bids /outputs participant
162201```
163202
164203### Using a Specific Version
165204
166205``` bash
167206# Create profile for newer version
168207cat > .datalad/containers/profiles/mriqc-24.yaml << 'EOF '
169- image: mriqc/ 24.0.0
170- exec: apptainer exec --cleanenv {img } {cmd}
208+ image: mriqc: 24.0.0
209+ exec: apptainer exec --cleanenv oci:{img_path } {cmd}
171210EOF
172211
173- datalad containers-run --profile mriqc-24 \
174- mriqc /bids /outputs participant
212+ datalad save -m " Add mriqc-24 profile " .datalad/containers/profiles/
213+ datalad containers-run --profile mriqc-24 -- mriqc /bids /outputs participant
175214```
176215
177216---
@@ -242,12 +281,15 @@ exec: >-
242281 --contain
243282 -H code/containers/binds/HOME
244283 -B code/containers/binds/zoneinfo/UTC:/etc/localtime
245- -B {pwd}
246- --pwd {pwd}
247- {img}
284+ oci:{img_path}
248285 {cmd}
249286` ` `
250287
288+ **Available placeholders:**
289+ - ` {img}` - Docker image name (for docker/podman)
290+ - ` {img_path}` - OCI directory path (for apptainer with `oci:` prefix)
291+ - ` {cmd}` - command arguments
292+
251293# ## Upstream RFE: Advanced Profile Features
252294
253295To fully replace the shim, datalad-container would need these profile extensions :
@@ -298,7 +340,7 @@ With these extensions, a profile could fully replace the shim:
298340` ` ` yaml
299341# .datalad/containers/profiles/mriqc-repronim.yaml
300342
301- image: mriqc/ 23.1.0
343+ image: mriqc: 23.1.0
302344
303345pre-run: code/containers/scripts/setup-env.sh
304346post-run: code/containers/scripts/cleanup.sh
@@ -315,10 +357,7 @@ exec: >-
315357 -B {env.TMPDIR}:/tmp
316358 -B {env.TMPDIR}/var:/var/tmp
317359 -B code/containers/binds/zoneinfo/UTC:/etc/localtime
318- -B {pwd}
319- --pwd {pwd}
320- -W {env.TMPDIR}
321- {img}
360+ oci:{img_path}
322361 {cmd}
323362` ` `
324363
@@ -330,35 +369,27 @@ Instead of runtime detection, use a separate profile:
330369# .datalad/containers/profiles/mriqc-repronim-docker.yaml
331370# For non-Linux systems (macOS, Windows with Docker)
332371
333- image: mriqc/ 23.1.0
372+ image: mriqc: 23.1.0
334373
335374pre-run: code/containers/scripts/setup-env-docker.sh
336375
337376env:
338- SINGULARITYENV_MPLCONFIGDIR : /tmp/mpl-config
377+ MPLCONFIGDIR : /tmp/mpl-config
339378
340379exec: >-
341380 docker run
342- --privileged
343381 --rm
344- -e UID={env.UID}
345- -e GID={env.GID}
346- -v {env.TMPDIR}:{env.TMPDIR}
347- -v {pwd}:{pwd}
348- -v code/containers/binds/HOME:{env.BHOME}
349- -w {pwd}
350- repronim/containers:latest
351- exec
352- --cleanenv
353- -H {env.BHOME}
354- -B {env.TMPDIR}:/tmp
355- --pwd {pwd}
382+ --user $(id -u):$(id -g)
383+ -v $(pwd):/work
384+ -w /work
356385 {img}
357386 {cmd}
358387` ` `
359388
360389Users on macOS would use `--profile mriqc-repronim-docker` explicitly.
361390
391+ **Note:** The Docker profile uses `{img}` (the Docker image name like `datalad-container/mriqc:23.1.0`) while apptainer profiles use `{img_path}` (the OCI directory path).
392+
362393# ## Summary: Upstream RFE for datalad-container
363394
364395To enable ReproNim to replace `singularity_cmd` with profiles :
0 commit comments