Missing file from cellpainting-gallery causes errors within tests

[d33bs]

I noticed that there are tests which rely on a file from the cellpainting-gallery which now appears to be missing: s3://cellpainting-gallery/cpg0016-jump/source_4/workspace/load_data_csv/2021_08_23_Batch12/BR00126114/load_data_with_illum.parquet. When looking in the related object path I don't see that there's a Parquet file anymore at that location. This change must have been recent (within the last week or so) as tests were passing before now (to my knowledge).

[d33bs]

Relates to job failure: https://github.com/cytomining/pycytominer/actions/runs/13026293667/job/36335910653

[d33bs]

Hi @shntnu - we noticed there might have been a change to the JUMP data within the Cell Painting Gallery on S3. The change pertains to a missing Parquet file which is a dependency for a Pycytominer test. Would you have any guidance on how we should proceed?

[shntnu]

Sorry for the annoyance. We had recently restructured load_data_csv to include brightfield images, then moved the old ones (without brightfield) to load_data_csv_orig, so you'd need to update the URL to s3://cellpainting-gallery/cpg0016-jump/source_4/workspace/load_data_csv_orig/2021_08_23_Batch12/BR00126114/load_data_with_illum.parquet

I wish we could guarantee a permalink, but that's tricky. Instead, .e try to avoid breaking changes but this one was unavoidable :D

Note that CSV files are available at that location (s3://cellpainting-gallery/cpg0016-jump/source_4/workspace/load_data_csv/2021_08_23_Batch12/BR00126114/load_data_with_illum.csv) just not Parquet files.

cc @ErinWeisbart
Internal issue: https://github.com/jump-cellpainting/datasets-private/issues/31#issuecomment-2620232263

[d33bs]

Thanks @shntnu and no worries at all! I'll plan to circle to the Cell Locations functionality in Pycytominer and enable CSV compatibility with the CSV file you referenced (please correct me if I'm wrong on the assumptions here).

[shntnu]

Thanks @shntnu and no worries at all! I'll plan to circle to the Cell Locations functionality in Pycytominer and enable CSV compatibility with the CSV file you referenced (please correct me if I'm wrong on the assumptions here).

Note that the CSV file does not have cell locations information. The simplest fixe is to replace load_data_csv with load_data_csv_orig in your paths and everything should work

[d33bs]

Thanks so much @shntnu I'll give that a try!