Add example for a job writing to a Unity Catalog volume #51
Conversation
| @@ -0,0 +1,60 @@ | |||
| { | |||
There was a problem hiding this comment.
This should render fine in the Github repo files. For example: https://github.com/databricks/cli/blob/main/internal/testdata/notebooks/py1.ipynb
shreyas-goenka
requested review from
pietern,
denik,
andrewnester and
lennartkats-db
pietern
changed the title
pietern
changed the title
knowledge_base/save_job_result_to_volume/resources/my_volume.volume.yml
Outdated
Show resolved
Hide resolved
knowledge_base/save_job_result_to_volume/resources/top_ten_trips.job.yml
Outdated
Show resolved
Hide resolved
knowledge_base/save_job_result_to_volume/resources/top_ten_trips.job.yml
Outdated
Show resolved
Hide resolved
| name: my_volume | ||
| # We use the ${resources.schemas...} interpolation syntax to force the creation | ||
| # of the schema before the volume. Usage of the ${resources.schemas...} syntax | ||
| # allows Databricks Asset Bundles to form a dependency graph between resources. |
There was a problem hiding this comment.
Do we need to go into this here?
There was a problem hiding this comment.
We have had multiple SAs reach out to us and ask how to sequence the resource creation. Given that folks will often use schemas with their volumes it feels relevant to keep this bit here.
There was a problem hiding this comment.
Thanks. Linking the warning PR for posterity: databricks/cli#1989
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "file_path = dbutils.widgets.get(\"file_path\")\n", |
There was a problem hiding this comment.
This doesn't actually work does it? Without a dbutils.widgets.text() and/or widgets section in the ipynb JSON below
There was a problem hiding this comment.
Works fine:
(.venv) ➜ cli git:(detect/schema-dep) databricks fs cat dbfs:/Volumes/main/shreyas_goenka_hello_world/my_volume/hello_world.txt -p dogfood
Hello World!%
knowledge_base/write_from_job_to_volume/resources/hello_world.job.yml
Outdated
Show resolved
Hide resolved
This example demonstrates how a job can write a file to a Unity Catalog volume.