Variables with different chunking patterns along the same dim?

[TomNicholas]

Is it a problem to have multiple variables with different chunking patterns along the same dimension?

e.g. a 3D variable with chunks {time: 1, z_t: 60, nlat: 384, nlon: 320} and a 2D variable with chunks {time: 60, nlat: 384, nlon: 320} alongside each other in the same zarr store / xarray Dataset?

I remember some issue where calling ds.chunksizes on such an object would raise an error asking you to call unify_chunks first...

Is this what unify_chunks is intended for correcting? (That function has basically no documentation...)

[dcherian]

Is it a problem to have multiple variables with different chunking patterns along the same dimension?

No. ds.chunksizes and xr.map_blocks won't work, but nothing else should be affected.

Is this what unify_chunks is intended for correcting?

Yes, but it makes ad-hoc choices. You're better off just calling .chunk manually if you need it.

[TomNicholas]

Okay thanks. I just realized I can't do this in my use case anyway, because the chunks I'm talking about will actually be netCDF files pointed to by a virtualizarr chunk manifest, so I can't use different chunking for different variables when they live in the same files. In this case maybe I just have to have quite small (500KB) chunks for the 2D variables...

[dcherian]

Now I'm confused. netCDF chunking can be specified per variable no?

[TomNicholas]

The files as they come are one netCDF file per time slice. So I can't have multiple time slices per chunk(/file) for the 2D variables without changing that pattern.

I could potentially re-save as internally-chunked netCDF... But then I should just be using Zarr in the first place!

[dcherian]

then I should just be using Zarr in the first place!

You have reached enlightenment.