entry_point modified when setting source_dir in Estimator if local mode execution

[Milogav]

Describe the bug
Setting source_dir and entry_point arguments for Estimator in local mode results in wrong entry_point path if the script is not located at the source dir root but inside a sub folder

sagemaker-python-sdk/src/sagemaker/estimator.py

Lines 884 to 890 in 697c465

	if self.sagemaker_session.local_mode and local_code:
	# if there is no source dir, use the directory containing the entry point.
	if self.source_dir is None:
	self.source_dir = os.path.dirname(self.entry_point)
	self.entry_point = os.path.basename(self.entry_point)
	code_dir = "file://" + self.source_dir

To reproduce

Using for example

source_dir='/home/ubuntu/src
entry_point='pipeline/program/run.py

results in

self.source_dir='/home/ubuntu/src
self.entry_point='run.py

Which raises a ValueError due to the wrong path:

sagemaker-python-sdk/src/sagemaker/fw_utils.py

Line 199 in 697c465

if not os.path.isfile(os.path.join(directory, script)):

Expected behavior
Keep entry point path unchanged if source_dir is provided

Screenshots or logs
If applicable, add screenshots or logs to help explain your problem.

System information
A description of your system. Please provide:

• SageMaker Python SDK version: 2.179.0
• Framework name (eg. PyTorch) or algorithm (eg. KMeans):
• Framework version:
• Python version: 3.10
• CPU or GPU:
• Custom Docker image (Y/N):

Additional context
Add any other context about the problem here.

[trungleduc]

Using for example

source_dir='/home/ubuntu/src
entry_point='pipeline/program/run.py

According to the documentation:

The absolute or relative path to the local Python source file that should be executed as the entry point to training. (Default: None). If source_dir is specified, then entry_point must point to a file located at the root of source_dir.

Your entry point is not at the root of your source dir

[vsimkus]

Came here to report the same. Is there a reason for having entry_point point to a file at the root of source_dir?

I am asking this because it works when running code on AWS with entry_point set to a file in a subdir of source_dir, but only fails in local mode due to the aforementioned line that strips the directory.

[vsimkus]

For anyone who might want a quick workaround, you can subclass the Estimator/Framework class to fix this. For example, see _script_mode_hyperparam_update method in the following code:

class CustomFramework(Framework):
    def __init__(
        self,
        entry_point,
        source_dir=None,
        hyperparameters=None,
        py_version=None,
        framework_version=None,
        image_uri=None,
        distributions=None,
        **kwargs,
    ):
        super(CustomFramework, self).__init__(
            entry_point, source_dir, hyperparameters, image_uri=image_uri, **kwargs
        )
        self.true_entry_point = entry_point
        self.framework_version = framework_version
        self.py_version = py_version

    def _script_mode_hyperparam_update(self, code_dir: str, script: str) -> None:
        local_code = get_config_value("local.local_code", self.sagemaker_session.config)
        if self.sagemaker_session.local_mode and local_code:
            # Workaround if in local mode
            return super()._script_mode_hyperparam_update(code_dir, self.true_entry_point)
        else:
            # Otherwise execute as normal
            return super()._script_mode_hyperparam_update(code_dir, script)