Skip to content

Add passthrough for storage cls args#106

Open
cisaacstern wants to merge 4 commits intomainfrom
storage-cls-args
Open

Add passthrough for storage cls args#106
cisaacstern wants to merge 4 commits intomainfrom
storage-cls-args

Conversation

@cisaacstern
Copy link
Member

@cisaacstern cisaacstern commented Sep 21, 2023

pangeo-forge/pangeo-forge-recipes#630 introduces the possibility that storage classes could take args.

This PR adds the ability to passthrough such args.

@codecov
Copy link

codecov bot commented Sep 21, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (3360d5d) 96.46% compared to head (4c0fcc1) 96.48%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #106      +/-   ##
==========================================
+ Coverage   96.46%   96.48%   +0.02%     
==========================================
  Files          15       15              
  Lines         509      512       +3     
==========================================
+ Hits          491      494       +3     
  Misses         18       18              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@moradology moradology left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have one minor gripe but it is ultimately aesthetic. Feel free to just hit 'ready for review' again if you think I'm wrong!

target_cls_args_str = ", ".join(
f"{k}={v}" for k, v in self.pangeo_forge_target_class_args.items()
)
return base + (f", {target_cls_args_str}" if target_cls_args_str else "") + ")"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be equivalent and a bit terser/easier to read: f"{base}, {target_cls_args_str})"

(I think join here should produce "" if no iterations happen from items(), so the check is not necessary)

Comment on lines 82 to +85
self.fsspec_class(**self.fsspec_args),
root_path=self.root_path.format(job_name=job_name),
fsspec_kwargs=self.fsspec_args,
**self.pangeo_forge_target_class_args,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to disambiguate the intents of pangeo_forge_target_class_args and fsspec_args

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants