Clarify Use of `azureml.core.runconfig.DockerConfiguration`

[hamelsmu]

@keijik @cody-dkdc

In this example notebook, you are using deprecated code

from azureml.core import Environment

fastai_env = Environment("fastai")
fastai_env.docker.enabled = True

I get the warning

'enabled' is deprecated. Please use the azureml.core.runconfig.DockerConfiguration object with the 'use_docker' param instead.

However, from the docs its a bit ambiguous on how to use it. It appears that the below code is incorrect. What is the correct way to use this instead? Can the documentation be improved? Also can you provide a better warning stating the version number at which this will be deprecated, or say that it will be deprecated in the future since it isn't deprecated as of right now?

from azureml.core.runconfig import DockerConfiguration
dc = DockerConfiguration(use_docker=True)
fastai_env.docker = dc

The documentation for this doesn't really provide many hints on where to use this configuration correctly. Can an example be made for this?

[hamelsmu]

cc: @gregce

[hamelsmu]

From much trial and error it appears that the pattern might be this? Not 100% sure

from azureml.core import Environment
fastai_env = Environment("fastai")

from azureml.core.runconfig import DockerConfiguration
dc = DockerConfiguration(use_docker=True, arguments=['-e', 'MY_NAME="foobarrr"'])

fastai_env.docker.base_image = "<image_name>"
fastai_env.python.user_managed_dependencies = True

fastai_config = ScriptRunConfig(source_directory='fastai-example',
                                script='train.py',
                                compute_target=compute_target,
                                environment=fastai_env,
                                docker_runtime_config=dc)

However, this API doesn't make any sense to me? Why does the user have to specify use_docker=True? Obstensively, if the user is passing this configuration, they always want to use Docker? Why would you ever set use_docker=False? And why is False the default? The meaning and usage of this API is very confusing

[vizhur]

@hamelsmu that pattern is correct, sorry for inconvenience. use_docker matters only in some cases like for local run. for run on AzureML Compute it will be a docker run despite the setting you set

[andrescodas]

Hi @vizhur,

I'm looking into how to inject a azureml.core.runconfig.DockerConfiguration into a azureml.pipeline.steps.ParallelRunConfig. I need to add docker run arguments such as "--cap-add", "SYS_ADMIN"

Is there an API for this end?

[vizhur]

https://docs.microsoft.com/en-us/python/api/azureml-pipeline-steps/azureml.pipeline.steps.parallel_run_config.parallelrunconfig?view=azure-ml-py#remarks
this should have a code sample, let me know if that is what are you looking for

[andrescodas]

https://docs.microsoft.com/en-us/python/api/azureml-pipeline-steps/azureml.pipeline.steps.parallel_run_config.parallelrunconfig?view=azure-ml-py#remarks this should have a code sample, let me know if that is what are you looking for

The code example there does not contain what I'm looking for :(

[vizhur]

sorry, my bad. ParallelRunStep as any other step should have a runconfig as a prop. It is not public for some reason but you should update from there a DockerConfiguration for the instance of your step

ParallelRunStep_instance = ParallelRunStep (…)
docker_config = DockerConfiguration(…)

ParallelRunStep_instance._runconfig.docker = docker_config