Possibly excessive memory usage when saving an F03 ancillary file

[hdyson]

🐜 Bug Report

This was observed in an earlier version of ANTS - step 1 should be verifying the problem still exists before any attempted fix.

How to reproduce

Step by step guide to reproduce the behaviour:

1. Prepare a large netCDF file with one netCDF chunk per field (i.e. per time/level)
2. Save a large cube as an F03 ancillary file (e.g. CMIP6 ozone)
3. Examine memory usage in scheduler job report

Expected behaviour is that one field at a time should be realised. Given delays with garbage collection etc, this probably means a small number of fields, rather than one.

Instead, all fields are retained in memory, and total memory usage is slightly over the output file size.

Version

This bug exists in ANTS 1.0

Additional Context

Previous attempted fix was to wrap a copy.deepcopy() around self.ppfield.data here. This was insufficient to fix the issue in real world testing.