What's the issue?
Departitioning a previously partitioned asset leads to run failure as the existing asset folders are not deleted before they are replaced with pickled files.
PermissionError: [Errno 13] Permission denied: 'dagster_home\\io\\asset_name'
What did you expect to happen?
After introducing partitioning, pickled files from previous materialisations are wiped and replaced with a folder structure inside DAGSTER_HOME\io\asset_name\partition_name
However, this does not occur on the reverse.
How to reproduce?
- Materialize a partitioned asset using
dagster.FilesystemIOManager
- Remove partitioning
- Materialize the now departitioned asset
Dagster version
1.7.7
Deployment type
None
Deployment details
Python 3.9, Windows Server 2019
Additional information
I need to check if this occurs on the latest version of dagster.
Manual workaround: Delete the affected partition folder(s), i.e: DAGSTER_HOME\io\asset_name and then rematerialize
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
What's the issue?
Departitioning a previously partitioned asset leads to run failure as the existing asset folders are not deleted before they are replaced with pickled files.
What did you expect to happen?
After introducing partitioning, pickled files from previous materialisations are wiped and replaced with a folder structure inside
DAGSTER_HOME\io\asset_name\partition_nameHowever, this does not occur on the reverse.
How to reproduce?
dagster.FilesystemIOManagerDagster version
1.7.7
Deployment type
None
Deployment details
Python 3.9, Windows Server 2019
Additional information
I need to check if this occurs on the latest version of dagster.
Manual workaround: Delete the affected partition folder(s), i.e:
DAGSTER_HOME\io\asset_nameand then rematerializeMessage from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.