Open
Description
Describe the bug
When running the object sagemaker.estimator.Estimator in local mode:
sagemaker_session.config = {'local': {'local_code': True}}
dependencies are not copied to the container.
To reproduce
- We created HPO step with an estimator in it.
- We configure the session to be local (see above)
- We run only the estimator with fit
- We received an error that it can't find our dependency code
Expected behavior
- A similar interface such as processing input which allow you to control were in the container the code in being placed
- Running the estimator in local mode, copied the dependencies to the right folder
System information
A description of your system. Please provide:
- SageMaker Python SDK version: 2.126.0
- Framework name (eg. PyTorch) or algorithm (eg. KMeans): sagemaker.tuner.HyperparameterTuner
- Framework version:
- Python version: 3.9.12
- CPU or GPU: CPU
- Custom Docker image (Y/N): Y
Additional context
We had to create the following workaround in order to it to work:
# Add repository to container so the code can use its dependencies'
tuning_step.inputs.update({'pipelines': 'file://{}'.format(str(Path(BASE_DIR).parents[2]))})
# Add The above pipelines stream to containers' PYTHONPATH so packages could be found
tuning_step.tuner.estimator.environment = {"PYTHONPATH": str(PurePosixPath("/opt/ml/input/data/pipelines"))}