diff --git a/docs/_quarto.yml b/docs/_quarto.yml index 0d4a2f9..bf66891 100644 --- a/docs/_quarto.yml +++ b/docs/_quarto.yml @@ -209,7 +209,8 @@ book: - part: "Google Colab In-Depth" chapters: - notes/dev-tools/google-colab/table-of-contents.qmd - - notes/dev-tools/google-colab/filesystem.ipynb + #- notes/dev-tools/google-colab/filesystem.ipynb + - notes/dev-tools/google-colab/files.qmd - notes/dev-tools/google-colab/form-inputs.ipynb - notes/dev-tools/google-colab/notebook-secrets.qmd - notes/dev-tools/google-colab/advanced-integrations.ipynb @@ -223,10 +224,12 @@ book: - notes/python-modules/datetime.qmd #- notes/python-modules/getpass.ipynb - #- part: "The Programming Environment" - # #href: notes/python-modules/index.qmd - # chapters: - # - notes/python-lang/file-operations.qmd + - part: "The Programming Environment" + #href: notes/python-modules/index.qmd + chapters: + - href: notes/python-modules/os.ipynb + text: "Accessing the Filesystem with the `os` Module" + - notes/python-lang/file-operations.qmd - part: "Custom Functions In-Depth" chapters: @@ -254,7 +257,7 @@ book: #body-footer: "© Copyright 2024, Your Name Here" page-footer: - center: "© Copyright 2024 - 2025, Michael J Rossetti" + center: "© Copyright 2025, Michael J Rossetti" #bibliography: references.bib diff --git a/docs/images/colab-file-download.png b/docs/images/colab-file-download.png new file mode 100644 index 0000000..026fe27 Binary files /dev/null and b/docs/images/colab-file-download.png differ diff --git a/docs/images/colab-file-upload.png b/docs/images/colab-file-upload.png new file mode 100644 index 0000000..5f465e2 Binary files /dev/null and b/docs/images/colab-file-upload.png differ diff --git a/docs/images/colab-form-dropdown.png b/docs/images/colab-form-dropdown.png new file mode 100644 index 0000000..70ac6b3 Binary files /dev/null and b/docs/images/colab-form-dropdown.png differ diff --git a/docs/images/colab-form-inputs.png b/docs/images/colab-form-inputs.png new file mode 100644 index 0000000..188fc8f Binary files /dev/null and b/docs/images/colab-form-inputs.png differ diff --git a/docs/images/frustration-with-computers.png b/docs/images/frustration-with-computers.png new file mode 100644 index 0000000..a1a13ce Binary files /dev/null and b/docs/images/frustration-with-computers.png differ diff --git a/docs/images/jupyter-widgets-dropdown.png b/docs/images/jupyter-widgets-dropdown.png new file mode 100644 index 0000000..e7d8fb6 Binary files /dev/null and b/docs/images/jupyter-widgets-dropdown.png differ diff --git a/docs/images/pypi-logo.png b/docs/images/pypi-logo.png new file mode 100644 index 0000000..1e149de Binary files /dev/null and b/docs/images/pypi-logo.png differ diff --git a/docs/notes/dev-tools/google-colab/files.qmd b/docs/notes/dev-tools/google-colab/files.qmd new file mode 100644 index 0000000..20e3dd1 --- /dev/null +++ b/docs/notes/dev-tools/google-colab/files.qmd @@ -0,0 +1,32 @@ +# Files in Google Colab + +Let's take a few moments to explore the \"Files\" menu in the Google Colab left sidebar. + +We see there are some example files in the \"sample_data\" directory. + +![Example files in the Colab filesystem](../../../images/colab-filesystem.png){height=350 fig-align="center"} + +## Downloading Files + +Observe, it is possible to download files like these from the Colab filesystem to your local machine, by right-clicking on them. + + +![Downloading files from the Colab filesystem.](../../../images/colab-file-download.png){height=350 fig-align="center"} + +## Uploading Files + +And it is possible to upload files from your local machine to the Colab filesystem as well, using the "Files > Upload to session storage" menu option (i.e. the button with the file upload icon). + +![Uploading files to the Colab filesystem.](../../../images/colab-file-upload.png){height=350 fig-align="center"} + +## Accessing and Manipulating Files + +Once we have the files in the Colab filesystem, we can write Python code to access and manipulate them. + + + One way of interacting with the filesystem in Python is by using the capabilities of [the `os` module](../../python-modules/os.ipynb). + + + For reading and writing text (\".txt\") files, we can leverage the `open` function (see [Text File Operations](../../python-lang/file-operations.qmd)). + + + For reading and writing tabular data (\".csv\") files, we can leverage the `pandas` package (see [Getting Started with Pandas](https://prof-rossetti.github.io/applied-data-science-python-book/notes/pandas/obtaining-dataframes.html)). + +Some of these examples might seem a bit complicated at the moment for beginners. For now, the main take-away is understanding there are ways for us to write Python code to interact with the surrounding environment, specifically accessing and manipulating the filesystem. diff --git a/docs/notes/dev-tools/google-colab/filesystem.ipynb b/docs/notes/dev-tools/google-colab/filesystem.ipynb index 19c91e6..21ee555 100644 --- a/docs/notes/dev-tools/google-colab/filesystem.ipynb +++ b/docs/notes/dev-tools/google-colab/filesystem.ipynb @@ -1,1054 +1,1069 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "EfWz5wVcHyYH" - }, - "source": [ - "# Accessing the Filesystem in Google Colab" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Jwr6NMZUIuYU" - }, - "source": [ - "Let's take a few moments to explore the \\\"Files\\\" menu in the Google Colab left sidebar.\n", - "\n", - "We see there are some example files in the \\\"sample_data\\\" directory.\n", - "\n", - "![Example files in the Colab filesystem](../../../images/colab-filesystem.png){height=350 fig-align=\"center\"}\n", - "\n", - "\n", - "Observe, it is possible to download files like these from the Colab filesystem to your local machine, and upload files from your local machine to the Colab filesystem as well.\n", - "\n", - "Once we have the files in the Colab filesystem, we can write Python code to access and manipulate them. One way of interacting with the filesystem in Python is by using the capabilities of the `os` module. We can read and write text (\\\".txt\\\") files using the `open` function, but we rarely need to do so. More commonly, we will be reading and writing \\\".csv\\\" files in Python using the `pandas` package.\n", - "\n", - "The examples below are meant to demonstrate capabilities related to the Colab filesystem. For now, the main take-away is understanding there are ways for us to write Python code to interact with the surrounding environment, specifically accessing and manipulating the filesystem. The Python syntax may be unfamiliar at this time. Feel free to return to these examples later, once you have practical need to read and write files." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "8Acd8FvtU3RS" - }, - "source": [ - "## Accessing the Filesystem" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "WEMM19k9Ifoy" - }, - "source": [ - "Using the [`os` module](https://docs.python.org/3/library/os.html) to access the filesystem:" - ] - }, - { - "cell_type": "code", - "execution_count": 51, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "iad7l1qMH6TG", - "outputId": "c1f72a01-c10c-4dcf-e4db-d8315dc47f27" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "['anscombe.json',\n", - " 'README.md',\n", - " 'california_housing_train.csv',\n", - " 'mnist_test.csv',\n", - " 'california_housing_test.csv',\n", - " 'mnist_train_small.csv']" - ] - }, - "execution_count": 51, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "import os\n", - "\n", - "os.listdir(\"sample_data\")" - ] - }, - { - "cell_type": "code", - "execution_count": 52, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "tSewxeFLIQnu", - "outputId": "d8dc3d95-b0f0-444f-97ce-5b4d287abe6e" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "sample_data/california_housing_test.csv\n" - ] - }, - { - "data": { - "text/plain": [ - "True" - ] - }, - "execution_count": 52, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "csv_filepath = \"sample_data/california_housing_test.csv\"\n", - "print(csv_filepath)\n", - "\n", - "os.path.isfile(csv_filepath)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "UBnLQyVAUHPI" - }, - "source": [ - "## Reading and Writing Text Files" - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "EfWz5wVcHyYH" + }, + "source": [ + "# Accessing the Filesystem in Google Colab" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Jwr6NMZUIuYU" + }, + "source": [ + "Let's take a few moments to explore the \\\"Files\\\" menu in the Google Colab left sidebar.\n", + "\n", + "We see there are some example files in the \\\"sample_data\\\" directory.\n", + "\n", + "![Example files in the Colab filesystem](../../../images/colab-filesystem.png){height=350 fig-align=\"center\"}\n", + "\n", + "\n", + "Observe, it is possible to download files like these from the Colab filesystem to your local machine, by right-clicking on them. \n", + "\n", + "\n", + "![Downloading files from the Colab filesystem.](../../../images/colab-file-download.png){height=350 fig-align=\"center\"}\n", + "\n", + "\n", + "And it is possible to upload files from your local machine to the Colab filesystem as well, using the \"Files > Upload to session storage\" menu option.\n", + "\n", + "![Uploading files to the Colab filesystem.](../../../images/colab-file-upload.png){height=350 fig-align=\"center\"}\n", + "\n", + "\n", + "Once we have the files in the Colab filesystem, we can write Python code to access and manipulate them.\n", + "\n", + " + One way of interacting with the filesystem in Python is by using the capabilities of [the `os` module](../../python-modules/os.ipynb). \n", + "\n", + " + For reading and writing text (\\\".txt\\\") files, we can leverage the `open` function (see [File Operations](../../python-lang/file-operations.qmd)). \n", + "\n", + " + For reading and writing tabular data (\\\".csv\\\") files, we can leverage the `pandas` package. This topic is outside the scope of this book, but is covered in [Getting Started with Pandas](https://prof-rossetti.github.io/applied-data-science-python-book/notes/pandas/obtaining-dataframes.html) chapters in the professor's Applied Data Science book).\n", + "\n", + "The examples below are meant to demonstrate capabilities related to the Colab filesystem. For now, the main take-away is understanding there are ways for us to write Python code to interact with the surrounding environment, specifically accessing and manipulating the filesystem. The Python syntax may be unfamiliar at this time. Feel free to return to these examples later, once you have practical need to read and write files." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8Acd8FvtU3RS" + }, + "source": [ + "## Accessing the Filesystem" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "WEMM19k9Ifoy" + }, + "source": [ + "Using the [`os` module](https://docs.python.org/3/library/os.html) to access the filesystem:" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "iad7l1qMH6TG", + "outputId": "c1f72a01-c10c-4dcf-e4db-d8315dc47f27" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "tu0zquU5KDXY" - }, - "source": [ - "Using the [`open` function](https://docs.python.org/3/library/functions.html#open) in \"r\" (reader) mode to read text files:" + "data": { + "text/plain": [ + "['anscombe.json',\n", + " 'README.md',\n", + " 'california_housing_train.csv',\n", + " 'mnist_test.csv',\n", + " 'california_housing_test.csv',\n", + " 'mnist_train_small.csv']" ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import os\n", + "\n", + "os.listdir(\"sample_data\")" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "tSewxeFLIQnu", + "outputId": "d8dc3d95-b0f0-444f-97ce-5b4d287abe6e" + }, + "outputs": [ { - "cell_type": "code", - "execution_count": 53, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "cVkjPerPKHZg", - "outputId": "479cbd4a-58a3-4ad4-ac01-60cd5d42578b" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "sample_data/README.md\n", - "This directory includes a few sample datasets to get you started.\n", - "\n", - "* `california_housing_data*.csv` is California housing data from the 1990 US\n", - " Census; more information is available at:\n", - " https://developers.google.com/machine-learning/crash-course/california-housing-data-description\n", - "\n", - "* `mnist_*.csv` is a small sample of the\n", - " [MNIST database](https://en.wikipedia.org/wiki/MNIST_database), which is\n", - " described at: http://yann.lecun.com/exdb/mnist/\n", - "\n", - "* `anscombe.json` contains a copy of\n", - " [Anscombe's quartet](https://en.wikipedia.org/wiki/Anscombe%27s_quartet); it\n", - " was originally described in\n", - "\n", - " Anscombe, F. J. (1973). 'Graphs in Statistical Analysis'. American\n", - " Statistician. 27 (1): 17-21. JSTOR 2682899.\n", - "\n", - " and our copy was prepared by the\n", - " [vega_datasets library](https://github.com/altair-viz/vega_datasets/blob/4f67bdaad10f45e3549984e17e1b3088c731503d/vega_datasets/_data/anscombe.json).\n", - "\n" - ] - } - ], - "source": [ - "md_filepath = \"sample_data/README.md\"\n", - "print(md_filepath)\n", - "\n", - "with open(md_filepath, mode=\"r\") as f:\n", - " print(f.read())" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "sample_data/california_housing_test.csv\n" + ] }, { - "cell_type": "markdown", - "metadata": { - "id": "K2X-zV6XUuKA" - }, - "source": [ - "Writing to text file, using the `open` function in \"w\" (writer) mode:" + "data": { + "text/plain": [ + "True" ] + }, + "execution_count": 52, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "csv_filepath = \"sample_data/california_housing_test.csv\"\n", + "print(csv_filepath)\n", + "\n", + "os.path.isfile(csv_filepath)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UBnLQyVAUHPI" + }, + "source": [ + "## Reading and Writing Text Files" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tu0zquU5KDXY" + }, + "source": [ + "Using the [`open` function](https://docs.python.org/3/library/functions.html#open) in \"r\" (reader) mode to read text files:" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "cVkjPerPKHZg", + "outputId": "479cbd4a-58a3-4ad4-ac01-60cd5d42578b" + }, + "outputs": [ { - "cell_type": "code", - "execution_count": 54, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "K1uZrckcUOfa", - "outputId": "356bbda0-aaea-4995-ec0d-619458673c97" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "WRITING TO: my_message.txt\n", - "DONE!\n" - ] - } - ], - "source": [ - "new_md_filepath = \"my_message.txt\"\n", - "print(\"WRITING TO:\", new_md_filepath)\n", - "\n", - "with open(new_md_filepath, \"w\") as f:\n", - " f.write(\"Hello World!\")\n", - "\n", - "print(\"DONE!\")" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "sample_data/README.md\n", + "This directory includes a few sample datasets to get you started.\n", + "\n", + "* `california_housing_data*.csv` is California housing data from the 1990 US\n", + " Census; more information is available at:\n", + " https://developers.google.com/machine-learning/crash-course/california-housing-data-description\n", + "\n", + "* `mnist_*.csv` is a small sample of the\n", + " [MNIST database](https://en.wikipedia.org/wiki/MNIST_database), which is\n", + " described at: http://yann.lecun.com/exdb/mnist/\n", + "\n", + "* `anscombe.json` contains a copy of\n", + " [Anscombe's quartet](https://en.wikipedia.org/wiki/Anscombe%27s_quartet); it\n", + " was originally described in\n", + "\n", + " Anscombe, F. J. (1973). 'Graphs in Statistical Analysis'. American\n", + " Statistician. 27 (1): 17-21. JSTOR 2682899.\n", + "\n", + " and our copy was prepared by the\n", + " [vega_datasets library](https://github.com/altair-viz/vega_datasets/blob/4f67bdaad10f45e3549984e17e1b3088c731503d/vega_datasets/_data/anscombe.json).\n", + "\n" + ] + } + ], + "source": [ + "md_filepath = \"sample_data/README.md\"\n", + "print(md_filepath)\n", + "\n", + "with open(md_filepath, mode=\"r\") as f:\n", + " print(f.read())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "K2X-zV6XUuKA" + }, + "source": [ + "Writing to text file, using the `open` function in \"w\" (writer) mode:" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "K1uZrckcUOfa", + "outputId": "356bbda0-aaea-4995-ec0d-619458673c97" + }, + "outputs": [ { - "cell_type": "code", - "execution_count": 55, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "D2MhemRnUZ_6", - "outputId": "85107947-fc6f-4719-f1d1-ad2e9bf1d4f7" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "READING FROM: my_message.txt\n", - "Hello World!\n", - "DONE!\n" - ] - } - ], - "source": [ - "# verifying:\n", - "print(\"READING FROM:\", new_md_filepath)\n", - "\n", - "with open(new_md_filepath, \"r\") as f:\n", - " contents = f.read()\n", - " print(contents)\n", - "\n", - "print(\"DONE!\")" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "WRITING TO: my_message.txt\n", + "DONE!\n" + ] + } + ], + "source": [ + "new_md_filepath = \"my_message.txt\"\n", + "print(\"WRITING TO:\", new_md_filepath)\n", + "\n", + "with open(new_md_filepath, \"w\") as f:\n", + " f.write(\"Hello World!\")\n", + "\n", + "print(\"DONE!\")" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "D2MhemRnUZ_6", + "outputId": "85107947-fc6f-4719-f1d1-ad2e9bf1d4f7" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "p5aN1o3Z3KaC" - }, - "source": [ - "## Reading and Writing CSV Files" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "READING FROM: my_message.txt\n", + "Hello World!\n", + "DONE!\n" + ] + } + ], + "source": [ + "# verifying:\n", + "print(\"READING FROM:\", new_md_filepath)\n", + "\n", + "with open(new_md_filepath, \"r\") as f:\n", + " contents = f.read()\n", + " print(contents)\n", + "\n", + "print(\"DONE!\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "p5aN1o3Z3KaC" + }, + "source": [ + "## Reading and Writing CSV Files" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "bVcFYZjUIkPD" + }, + "source": [ + "Using the [`read_csv` function](https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html) from the `pandas` package to read \".csv\" file contents:" + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 244 }, + "id": "DrTmr36NHopZ", + "outputId": "f0809bd0-a22d-4042-e03b-d991e5513d63" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "bVcFYZjUIkPD" - }, - "source": [ - "Using the [`read_csv` function](https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html) from the `pandas` package to read \".csv\" file contents:" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "READING FROM: sample_data/california_housing_test.csv\n" + ] }, { - "cell_type": "code", - "execution_count": 56, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 244 - }, - "id": "DrTmr36NHopZ", - "outputId": "f0809bd0-a22d-4042-e03b-d991e5513d63" + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"df\",\n \"rows\": 3000,\n \"fields\": [\n {\n \"column\": \"longitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.9949362939550175,\n \"min\": -124.18,\n \"max\": -114.49,\n \"num_unique_values\": 607,\n \"samples\": [\n -121.15,\n -121.46,\n -121.02\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"latitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2.129669523343834,\n \"min\": 32.56,\n \"max\": 41.92,\n \"num_unique_values\": 587,\n \"samples\": [\n 40.17,\n 33.69,\n 39.61\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"housing_median_age\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12.555395554955753,\n \"min\": 1.0,\n \"max\": 52.0,\n \"num_unique_values\": 52,\n \"samples\": [\n 14.0,\n 49.0,\n 7.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_rooms\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2155.593331625581,\n \"min\": 6.0,\n \"max\": 30450.0,\n \"num_unique_values\": 2215,\n \"samples\": [\n 1961.0,\n 1807.0,\n 680.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_bedrooms\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 415.6543681363229,\n \"min\": 2.0,\n \"max\": 5419.0,\n \"num_unique_values\": 1055,\n \"samples\": [\n 532.0,\n 764.0,\n 2162.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"population\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1030.5430124122424,\n \"min\": 5.0,\n \"max\": 11935.0,\n \"num_unique_values\": 1802,\n \"samples\": [\n 947.0,\n 1140.0,\n 2019.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"households\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 365.42270980552627,\n \"min\": 2.0,\n \"max\": 4930.0,\n \"num_unique_values\": 1026,\n \"samples\": [\n 646.0,\n 629.0,\n 504.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"median_income\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.854511729691479,\n \"min\": 0.4999,\n \"max\": 15.0001,\n \"num_unique_values\": 2578,\n \"samples\": [\n 1.725,\n 0.7403,\n 2.6964\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"median_house_value\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 113119.68746964628,\n \"min\": 22500.0,\n \"max\": 500001.0,\n \"num_unique_values\": 1784,\n \"samples\": [\n 71900.0,\n 63000.0,\n 115800.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "df" }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "READING FROM: sample_data/california_housing_test.csv\n" - ] - }, - { - "data": { - "application/vnd.google.colaboratory.intrinsic+json": { - "summary": "{\n \"name\": \"df\",\n \"rows\": 3000,\n \"fields\": [\n {\n \"column\": \"longitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.9949362939550175,\n \"min\": -124.18,\n \"max\": -114.49,\n \"num_unique_values\": 607,\n \"samples\": [\n -121.15,\n -121.46,\n -121.02\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"latitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2.129669523343834,\n \"min\": 32.56,\n \"max\": 41.92,\n \"num_unique_values\": 587,\n \"samples\": [\n 40.17,\n 33.69,\n 39.61\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"housing_median_age\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12.555395554955753,\n \"min\": 1.0,\n \"max\": 52.0,\n \"num_unique_values\": 52,\n \"samples\": [\n 14.0,\n 49.0,\n 7.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_rooms\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2155.593331625581,\n \"min\": 6.0,\n \"max\": 30450.0,\n \"num_unique_values\": 2215,\n \"samples\": [\n 1961.0,\n 1807.0,\n 680.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_bedrooms\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 415.6543681363229,\n \"min\": 2.0,\n \"max\": 5419.0,\n \"num_unique_values\": 1055,\n \"samples\": [\n 532.0,\n 764.0,\n 2162.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"population\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1030.5430124122424,\n \"min\": 5.0,\n \"max\": 11935.0,\n \"num_unique_values\": 1802,\n \"samples\": [\n 947.0,\n 1140.0,\n 2019.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"households\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 365.42270980552627,\n \"min\": 2.0,\n \"max\": 4930.0,\n \"num_unique_values\": 1026,\n \"samples\": [\n 646.0,\n 629.0,\n 504.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"median_income\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.854511729691479,\n \"min\": 0.4999,\n \"max\": 15.0001,\n \"num_unique_values\": 2578,\n \"samples\": [\n 1.725,\n 0.7403,\n 2.6964\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"median_house_value\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 113119.68746964628,\n \"min\": 22500.0,\n \"max\": 500001.0,\n \"num_unique_values\": 1784,\n \"samples\": [\n 71900.0,\n 63000.0,\n 115800.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", - "type": "dataframe", - "variable_name": "df" - }, - "text/html": [ - "\n", - "
\n", - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_value
0-122.0537.3727.03885.0661.01537.0606.06.6085344700.0
1-118.3034.2643.01510.0310.0809.0277.03.5990176500.0
2-117.8133.7827.03589.0507.01484.0495.05.7934270500.0
3-118.3633.8228.067.015.049.011.06.1359330000.0
4-119.6736.3319.01241.0244.0850.0237.02.937581700.0
\n", - "
\n", - "
\n", - "\n", - "
\n", - " \n", - "\n", - " \n", - "\n", - " \n", - "
\n", - "\n", - "\n", - "
\n", - " \n", - "\n", - "\n", - "\n", - " \n", - "
\n", - "\n", - "
\n", - "
\n" - ], - "text/plain": [ - " longitude latitude housing_median_age total_rooms total_bedrooms \\\n", - "0 -122.05 37.37 27.0 3885.0 661.0 \n", - "1 -118.30 34.26 43.0 1510.0 310.0 \n", - "2 -117.81 33.78 27.0 3589.0 507.0 \n", - "3 -118.36 33.82 28.0 67.0 15.0 \n", - "4 -119.67 36.33 19.0 1241.0 244.0 \n", - "\n", - " population households median_income median_house_value \n", - "0 1537.0 606.0 6.6085 344700.0 \n", - "1 809.0 277.0 3.5990 176500.0 \n", - "2 1484.0 495.0 5.7934 270500.0 \n", - "3 49.0 11.0 6.1359 330000.0 \n", - "4 850.0 237.0 2.9375 81700.0 " - ] - }, - "execution_count": 56, - "metadata": {}, - "output_type": "execute_result" - } + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_value
0-122.0537.3727.03885.0661.01537.0606.06.6085344700.0
1-118.3034.2643.01510.0310.0809.0277.03.5990176500.0
2-117.8133.7827.03589.0507.01484.0495.05.7934270500.0
3-118.3633.8228.067.015.049.011.06.1359330000.0
4-119.6736.3319.01241.0244.0850.0237.02.937581700.0
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "\n", + "
\n", + "
\n" ], - "source": [ - "from pandas import read_csv\n", - "\n", - "print(\"READING FROM:\", csv_filepath)\n", - "\n", - "df = read_csv(csv_filepath)\n", - "df.head()" + "text/plain": [ + " longitude latitude housing_median_age total_rooms total_bedrooms \\\n", + "0 -122.05 37.37 27.0 3885.0 661.0 \n", + "1 -118.30 34.26 43.0 1510.0 310.0 \n", + "2 -117.81 33.78 27.0 3589.0 507.0 \n", + "3 -118.36 33.82 28.0 67.0 15.0 \n", + "4 -119.67 36.33 19.0 1241.0 244.0 \n", + "\n", + " population households median_income median_house_value \n", + "0 1537.0 606.0 6.6085 344700.0 \n", + "1 809.0 277.0 3.5990 176500.0 \n", + "2 1484.0 495.0 5.7934 270500.0 \n", + "3 49.0 11.0 6.1359 330000.0 \n", + "4 850.0 237.0 2.9375 81700.0 " ] + }, + "execution_count": 56, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from pandas import read_csv\n", + "\n", + "print(\"READING FROM:\", csv_filepath)\n", + "\n", + "df = read_csv(csv_filepath)\n", + "df.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2ZkDDOmlVKMJ" + }, + "source": [ + "Using the [`to_csv` method](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_csv.html) of the resulting `pandas.DataFrame` object, to write data back to a CSV file:" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "JADJ0sgjUjt4", + "outputId": "6a46a690-96aa-45a9-b795-ee6874f5bc74" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "2ZkDDOmlVKMJ" - }, - "source": [ - "Using the [`to_csv` method](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_csv.html) of the resulting `pandas.DataFrame` object, to write data back to a CSV file:" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "WRITING TO: my_copy.csv\n" + ] + } + ], + "source": [ + "new_csv_filepath = \"my_copy.csv\"\n", + "print(\"WRITING TO:\", new_csv_filepath)\n", + "\n", + "df.to_csv(new_csv_filepath, index=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 244 }, + "id": "XjBG1WoDVYpY", + "outputId": "e8a5199e-63db-4136-e044-3518dce4e432" + }, + "outputs": [ { - "cell_type": "code", - "execution_count": 57, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "JADJ0sgjUjt4", - "outputId": "6a46a690-96aa-45a9-b795-ee6874f5bc74" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "WRITING TO: my_copy.csv\n" - ] - } - ], - "source": [ - "new_csv_filepath = \"my_copy.csv\"\n", - "print(\"WRITING TO:\", new_csv_filepath)\n", - "\n", - "df.to_csv(new_csv_filepath, index=False)" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "READING BACK FROM: my_copy.csv\n" + ] }, { - "cell_type": "code", - "execution_count": 58, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 244 - }, - "id": "XjBG1WoDVYpY", - "outputId": "e8a5199e-63db-4136-e044-3518dce4e432" + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"new_df\",\n \"rows\": 3000,\n \"fields\": [\n {\n \"column\": \"longitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.9949362939550175,\n \"min\": -124.18,\n \"max\": -114.49,\n \"num_unique_values\": 607,\n \"samples\": [\n -121.15,\n -121.46,\n -121.02\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"latitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2.129669523343834,\n \"min\": 32.56,\n \"max\": 41.92,\n \"num_unique_values\": 587,\n \"samples\": [\n 40.17,\n 33.69,\n 39.61\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"housing_median_age\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12.555395554955753,\n \"min\": 1.0,\n \"max\": 52.0,\n \"num_unique_values\": 52,\n \"samples\": [\n 14.0,\n 49.0,\n 7.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_rooms\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2155.593331625581,\n \"min\": 6.0,\n \"max\": 30450.0,\n \"num_unique_values\": 2215,\n \"samples\": [\n 1961.0,\n 1807.0,\n 680.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_bedrooms\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 415.6543681363229,\n \"min\": 2.0,\n \"max\": 5419.0,\n \"num_unique_values\": 1055,\n \"samples\": [\n 532.0,\n 764.0,\n 2162.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"population\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1030.5430124122424,\n \"min\": 5.0,\n \"max\": 11935.0,\n \"num_unique_values\": 1802,\n \"samples\": [\n 947.0,\n 1140.0,\n 2019.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"households\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 365.42270980552627,\n \"min\": 2.0,\n \"max\": 4930.0,\n \"num_unique_values\": 1026,\n \"samples\": [\n 646.0,\n 629.0,\n 504.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"median_income\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.854511729691479,\n \"min\": 0.4999,\n \"max\": 15.0001,\n \"num_unique_values\": 2578,\n \"samples\": [\n 1.725,\n 0.7403,\n 2.6964\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"median_house_value\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 113119.68746964628,\n \"min\": 22500.0,\n \"max\": 500001.0,\n \"num_unique_values\": 1784,\n \"samples\": [\n 71900.0,\n 63000.0,\n 115800.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "new_df" }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "READING BACK FROM: my_copy.csv\n" - ] - }, - { - "data": { - "application/vnd.google.colaboratory.intrinsic+json": { - "summary": "{\n \"name\": \"new_df\",\n \"rows\": 3000,\n \"fields\": [\n {\n \"column\": \"longitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.9949362939550175,\n \"min\": -124.18,\n \"max\": -114.49,\n \"num_unique_values\": 607,\n \"samples\": [\n -121.15,\n -121.46,\n -121.02\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"latitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2.129669523343834,\n \"min\": 32.56,\n \"max\": 41.92,\n \"num_unique_values\": 587,\n \"samples\": [\n 40.17,\n 33.69,\n 39.61\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"housing_median_age\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12.555395554955753,\n \"min\": 1.0,\n \"max\": 52.0,\n \"num_unique_values\": 52,\n \"samples\": [\n 14.0,\n 49.0,\n 7.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_rooms\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2155.593331625581,\n \"min\": 6.0,\n \"max\": 30450.0,\n \"num_unique_values\": 2215,\n \"samples\": [\n 1961.0,\n 1807.0,\n 680.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_bedrooms\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 415.6543681363229,\n \"min\": 2.0,\n \"max\": 5419.0,\n \"num_unique_values\": 1055,\n \"samples\": [\n 532.0,\n 764.0,\n 2162.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"population\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1030.5430124122424,\n \"min\": 5.0,\n \"max\": 11935.0,\n \"num_unique_values\": 1802,\n \"samples\": [\n 947.0,\n 1140.0,\n 2019.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"households\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 365.42270980552627,\n \"min\": 2.0,\n \"max\": 4930.0,\n \"num_unique_values\": 1026,\n \"samples\": [\n 646.0,\n 629.0,\n 504.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"median_income\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.854511729691479,\n \"min\": 0.4999,\n \"max\": 15.0001,\n \"num_unique_values\": 2578,\n \"samples\": [\n 1.725,\n 0.7403,\n 2.6964\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"median_house_value\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 113119.68746964628,\n \"min\": 22500.0,\n \"max\": 500001.0,\n \"num_unique_values\": 1784,\n \"samples\": [\n 71900.0,\n 63000.0,\n 115800.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", - "type": "dataframe", - "variable_name": "new_df" - }, - "text/html": [ - "\n", - "
\n", - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_value
0-122.0537.3727.03885.0661.01537.0606.06.6085344700.0
1-118.3034.2643.01510.0310.0809.0277.03.5990176500.0
2-117.8133.7827.03589.0507.01484.0495.05.7934270500.0
3-118.3633.8228.067.015.049.011.06.1359330000.0
4-119.6736.3319.01241.0244.0850.0237.02.937581700.0
\n", - "
\n", - "
\n", - "\n", - "
\n", - " \n", - "\n", - " \n", - "\n", - " \n", - "
\n", - "\n", - "\n", - "
\n", - " \n", - "\n", - "\n", - "\n", - " \n", - "
\n", - "\n", - "
\n", - "
\n" - ], - "text/plain": [ - " longitude latitude housing_median_age total_rooms total_bedrooms \\\n", - "0 -122.05 37.37 27.0 3885.0 661.0 \n", - "1 -118.30 34.26 43.0 1510.0 310.0 \n", - "2 -117.81 33.78 27.0 3589.0 507.0 \n", - "3 -118.36 33.82 28.0 67.0 15.0 \n", - "4 -119.67 36.33 19.0 1241.0 244.0 \n", - "\n", - " population households median_income median_house_value \n", - "0 1537.0 606.0 6.6085 344700.0 \n", - "1 809.0 277.0 3.5990 176500.0 \n", - "2 1484.0 495.0 5.7934 270500.0 \n", - "3 49.0 11.0 6.1359 330000.0 \n", - "4 850.0 237.0 2.9375 81700.0 " - ] - }, - "execution_count": 58, - "metadata": {}, - "output_type": "execute_result" - } + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_value
0-122.0537.3727.03885.0661.01537.0606.06.6085344700.0
1-118.3034.2643.01510.0310.0809.0277.03.5990176500.0
2-117.8133.7827.03589.0507.01484.0495.05.7934270500.0
3-118.3633.8228.067.015.049.011.06.1359330000.0
4-119.6736.3319.01241.0244.0850.0237.02.937581700.0
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "\n", + "
\n", + "
\n" ], - "source": [ - "# verifying:\n", - "print(\"READING BACK FROM:\", new_csv_filepath)\n", - "new_df = read_csv(new_csv_filepath)\n", - "new_df.head()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "zFZL1Wch4QKf" - }, - "source": [ - "For now, the main take-away is understanding there are ways for us to write Python code to interact with the surrounding environment, specifically accessing and manipulating the filesystem. We will return to cover data processing with `pandas` in much more detail in a later unit." + "text/plain": [ + " longitude latitude housing_median_age total_rooms total_bedrooms \\\n", + "0 -122.05 37.37 27.0 3885.0 661.0 \n", + "1 -118.30 34.26 43.0 1510.0 310.0 \n", + "2 -117.81 33.78 27.0 3589.0 507.0 \n", + "3 -118.36 33.82 28.0 67.0 15.0 \n", + "4 -119.67 36.33 19.0 1241.0 244.0 \n", + "\n", + " population households median_income median_house_value \n", + "0 1537.0 606.0 6.6085 344700.0 \n", + "1 809.0 277.0 3.5990 176500.0 \n", + "2 1484.0 495.0 5.7934 270500.0 \n", + "3 49.0 11.0 6.1359 330000.0 \n", + "4 850.0 237.0 2.9375 81700.0 " ] + }, + "execution_count": 58, + "metadata": {}, + "output_type": "execute_result" } - ], - "metadata": { - "colab": { - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - }, - "language_info": { - "name": "python" - } + ], + "source": [ + "# verifying:\n", + "print(\"READING BACK FROM:\", new_csv_filepath)\n", + "new_df = read_csv(new_csv_filepath)\n", + "new_df.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "zFZL1Wch4QKf" + }, + "source": [ + "For now, the main take-away is understanding there are ways for us to write Python code to interact with the surrounding environment, specifically accessing and manipulating the filesystem. We will return to cover data processing with `pandas` in much more detail in a later unit." + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" }, - "nbformat": 4, - "nbformat_minor": 0 + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 } diff --git a/docs/notes/dev-tools/google-colab/form-inputs.ipynb b/docs/notes/dev-tools/google-colab/form-inputs.ipynb index 345dedc..3e6d05d 100644 --- a/docs/notes/dev-tools/google-colab/form-inputs.ipynb +++ b/docs/notes/dev-tools/google-colab/form-inputs.ipynb @@ -1,498 +1,518 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "QwBmtMzEQ5ib" - }, - "source": [ - "# Forms and Inputs in Google Colab" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "--0cSuSxR1bU" - }, - "source": [ - "When creating data dashboards and user facing applications, it may be helpful to ask the user for inputs.\n", - "\n", - "Besides capturing a textual inputs using the `input` function, we can use a form to capture a variety of other kinds of inputs.\n", - "\n", - "To implement a form in Google Colab, we can use Colab Form Inputs, or leverage the capabilities of the Jupyter Widgets package." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "tQdbmlndUtYe" - }, - "source": [ - "## The `input` Function" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "jNVWlz3sTRUo", - "outputId": "0b94ad85-ff20-4632-a50a-c28b92c58fac" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Please enter a stock symbol (e.g. 'MSFT'): NVDA\n", - "NVDA\n" - ] - } - ], - "source": [ - "symbol = input(\"Please enter a stock symbol (e.g. 'MSFT'): \")\n", - "print(symbol)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "0Vp7ncedUu_w" - }, - "source": [ - "## The `getpass` Module" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "RgDVmooKUxfo" - }, - "source": [ - "When using the `input` function in Python to capture user inputs, we end up seeing the resulting value as the user types it.\n", - "\n", - "But what if we want to ask for more sensitive inputs, like secret passwords or credentials? We need a way to prevent them from showing up in the program's output.\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "YevMLazFVEOh" - }, - "source": [ - "To hide or \"mask\" user-inputted values, we can use the `getpass` function from the [`getpass` module](https://docs.python.org/3/library/getpass.html) instead:\n" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "PsvqMVgEU5EC", - "outputId": "0f2420ba-a21e-470f-9a91-e1e96e404a9e" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Please input your secret password: ··········\n", - "super-secret\n" - ] - } - ], - "source": [ - "from getpass import getpass\n", - "\n", - "my_password = getpass(\"Please input your secret password: \")\n", - "print(my_password)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "zTQYRQV5QxJ0" - }, - "source": [ - "\n", - "## Colab Form Inputs" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "7aL9Xk0DRQAm" - }, - "source": [ - "One helpful way to create form inputs is to add the \"Adding form fields\" code snippet, which provides example form fields of various kinds. Adding the snippet yeilds the following example form:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "ADEXhZVqRNVA" - }, - "outputs": [], - "source": [ - "# @title Example form fields\n", - "# @markdown Forms support many types of fields.\n", - "\n", - "no_type_checking = '' # @param\n", - "string_type = 'example' # @param {type: \"string\"}\n", - "slider_value = 142 # @param {type: \"slider\", min: 100, max: 200}\n", - "number = 102 # @param {type: \"number\"}\n", - "date = '2024-06-23' # @param {type: \"date\"}\n", - "pick_me = \"monday\" # @param ['monday', 'tuesday', 'wednesday', 'thursday']\n", - "select_or_input = \"apples\" # @param [\"apples\", \"bananas\", \"oranges\"] {allow-input: true}\n", - "# @markdown ---" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "m3Ebd61t2KfI", - "outputId": "adc6fda7-a324-4992-d51b-3f3600ef3a98" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "142\n", - "2024-06-23\n" - ] - } - ], - "source": [ - "print(slider_value)\n", - "print(date)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "2WQQfsxiQs2F" - }, - "source": [ - "We see the form inputs are implemented by structured comments to the right of the variable assignment." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "mbi4JQfxRUIL" - }, - "source": [ - "\n", - "\n", - "In practice, we'll probably only need one or two of these, and we can copy the corresponding code snippet and adapt it to fit the use case.\n", - "\n", - "For example, creating a drop-down menu prompting a user to select a stock symbol:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "fEkxfBb1QsO5", - "outputId": "62b63fb1-bf9c-46ab-def4-6c5651aff972" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "NVDA\n" - ] - } - ], - "source": [ - "symbol = \"NVDA\" # @param ['MSFT', 'GOOGL', 'NVDA', 'SPOT', 'NFLX']\n", - "print(symbol)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "jMIl-crD12m6" - }, - "source": [ - ":::{.callout-note title=\"Note\"}\n", - "To make the Colab form inputs work, we first choose an option from the form on the right. This updates the code cell contents on the left. Then we run the cell to store the selected value in memory.\n", - ":::" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "knUJVbRympxg" - }, - "source": [ - "## Jupyter Widgets" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "MAJ3W3-Qmru5" - }, - "source": [ - "You'll notice the Colab form approach requires us to hard-code the selectable options in code comments. This may work in many cases, but there may be some situations where you would want to dynamically compile the list of options instead.\n", - "\n", - "There's another option for creating form inputs in a Python notebook, using the [Jupyter Widgets (`ipywidgets`) package](https://ipywidgets.readthedocs.io/en/stable/).\n", - "\n", - "\n" - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "QwBmtMzEQ5ib" + }, + "source": [ + "# Forms and Inputs in Google Colab" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "--0cSuSxR1bU" + }, + "source": [ + "When creating data dashboards and user facing applications, it may be helpful to ask the user for inputs.\n", + "\n", + "Besides capturing a textual inputs using the `input` function, we can use a form to capture a variety of other kinds of inputs.\n", + "\n", + "To implement a form in Google Colab, we can use Colab Form Inputs, or leverage the capabilities of the Jupyter Widgets package." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tQdbmlndUtYe" + }, + "source": [ + "## The `input` Function" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "jNVWlz3sTRUo", + "outputId": "0b94ad85-ff20-4632-a50a-c28b92c58fac" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "cyaonQYOXhD3" - }, - "source": [ - "Compiling dropdown selections from a list:\n" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "Please enter a stock symbol (e.g. 'MSFT'): NVDA\n", + "NVDA\n" + ] + } + ], + "source": [ + "symbol = input(\"Please enter a stock symbol (e.g. 'MSFT'): \")\n", + "print(symbol)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0Vp7ncedUu_w" + }, + "source": [ + "## The `getpass` Module" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "RgDVmooKUxfo" + }, + "source": [ + "When using the `input` function in Python to capture user inputs, we end up seeing the resulting value as the user types it.\n", + "\n", + "But what if we want to ask for more sensitive inputs, like secret passwords or credentials? We need a way to prevent them from showing up in the program's output.\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YevMLazFVEOh" + }, + "source": [ + "To hide or \"mask\" user-inputted values, we can use the `getpass` function from the [`getpass` module](https://docs.python.org/3/library/getpass.html) instead:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "PsvqMVgEU5EC", + "outputId": "0f2420ba-a21e-470f-9a91-e1e96e404a9e" + }, + "outputs": [ { - "cell_type": "code", - "execution_count": 1, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "1Md1hpCmt5Ws", - "outputId": "bc74691b-414a-44bf-c45e-fc8e6a91c77f" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "['MSFT', 'GOOGL', 'NVDA', 'SPOT', 'NFLX']\n" - ] - } - ], - "source": [ - "symbols = ['MSFT', 'GOOGL', 'NVDA', 'SPOT', 'NFLX']\n", - "print(symbols)" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "Please input your secret password: ··········\n", + "super-secret\n" + ] + } + ], + "source": [ + "from getpass import getpass\n", + "\n", + "my_password = getpass(\"Please input your secret password: \")\n", + "print(my_password)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For a more modern and streamlined approach for supplying secret values, see [Notebook Secrets](./notebook-secrets.qmd)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "zTQYRQV5QxJ0" + }, + "source": [ + "\n", + "## Colab Form Inputs" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7aL9Xk0DRQAm" + }, + "source": [ + "One helpful way to create form inputs is to add the \"Adding form fields\" code snippet, which provides example form fields of various kinds. Adding the snippet yeilds the following example form:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ADEXhZVqRNVA" + }, + "outputs": [], + "source": [ + "# @title Example form fields\n", + "# @markdown Forms support many types of fields.\n", + "\n", + "no_type_checking = '' # @param\n", + "string_type = 'example' # @param {type: \"string\"}\n", + "slider_value = 142 # @param {type: \"slider\", min: 100, max: 200}\n", + "number = 102 # @param {type: \"number\"}\n", + "date = '2024-06-23' # @param {type: \"date\"}\n", + "pick_me = \"monday\" # @param ['monday', 'tuesday', 'wednesday', 'thursday']\n", + "select_or_input = \"apples\" # @param [\"apples\", \"bananas\", \"oranges\"] {allow-input: true}\n", + "# @markdown ---" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "m3Ebd61t2KfI", + "outputId": "adc6fda7-a324-4992-d51b-3f3600ef3a98" + }, + "outputs": [ { - "cell_type": "code", - "execution_count": 18, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 49, - "referenced_widgets": [ - "af6a2992bee5428aaf1b92e34ab8aee7", - "d7d8264ada2048d59aca65e41182fca4", - "a03a7e16c502430b93c5a9dbc06a2927" - ] - }, - "id": "ohctqwc8m4K_", - "outputId": "2887a32a-fc02-48b7-8b47-9b046738402e" - }, - "outputs": [ - { - "data": { - "application/vnd.jupyter.widget-view+json": { - "model_id": "af6a2992bee5428aaf1b92e34ab8aee7", - "version_major": 2, - "version_minor": 0 - }, - "text/plain": [ - "Dropdown(description='Symbol: ', options=('MSFT', 'GOOGL', 'NVDA', 'SPOT', 'NFLX'), value='MSFT')" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "from ipywidgets.widgets import Dropdown\n", - "\n", - "# configure the dropdown menu:\n", - "dropdown = Dropdown(\n", - " options=symbols,\n", - " value=symbols[0], # default value\n", - " description='Symbol: ',\n", - " disabled=False,\n", - ")\n", - "\n", - "# display the dropdown menu:\n", - "display(dropdown)" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "142\n", + "2024-06-23\n" + ] + } + ], + "source": [ + "print(slider_value)\n", + "print(date)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2WQQfsxiQs2F" + }, + "source": [ + "We see the form inputs are implemented by structured comments to the right of the variable assignment.\n", + "\n", + "And these inputs are displayed to the right of the code cell in Colab:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![Form inputs in Google Colab.](../../../images/colab-form-inputs.png)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "mbi4JQfxRUIL" + }, + "source": [ + "\n", + "\n", + "In practice, we'll probably only need one or two of these, and we can copy the corresponding code snippet and adapt it to fit the use case.\n", + "\n", + "For example, creating a drop-down menu prompting a user to select a stock symbol:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "fEkxfBb1QsO5", + "outputId": "62b63fb1-bf9c-46ab-def4-6c5651aff972" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "L8cVJ7aRCF7_" - }, - "source": [ - "Using the selected value:" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "NVDA\n" + ] + } + ], + "source": [ + "symbol = \"NVDA\" # @param ['MSFT', 'GOOGL', 'NVDA', 'SPOT', 'NFLX']\n", + "print(symbol)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "jMIl-crD12m6" + }, + "source": [ + ":::{.callout-note title=\"Note\"}\n", + "To make the Colab form inputs work, we first choose an option from the form on the right. This updates the code cell contents on the left. Then we run the cell to store the selected value in memory.\n", + "\n", + "![Dropdown input in Google Colab.](../../../images/colab-form-dropdown.png)\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "knUJVbRympxg" + }, + "source": [ + "## Jupyter Widgets" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "MAJ3W3-Qmru5" + }, + "source": [ + "You'll notice the Colab form approach requires us to hard-code the selectable options in code comments. This may work in many cases, but there may be some situations where you would want to dynamically compile the list of options instead.\n", + "\n", + "There's another option for creating form inputs in a Python notebook, using the [Jupyter Widgets (`ipywidgets`) package](https://ipywidgets.readthedocs.io/en/stable/).\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cyaonQYOXhD3" + }, + "source": [ + "Compiling dropdown selections from a list:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "1Md1hpCmt5Ws", + "outputId": "bc74691b-414a-44bf-c45e-fc8e6a91c77f" + }, + "outputs": [ { - "cell_type": "code", - "execution_count": 19, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "y-5jjLg7CD0w", - "outputId": "faffe3ed-40c8-4db9-e8d3-d26a35fae2ea" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "NVDA\n" - ] - } - ], - "source": [ - "print(dropdown.value)" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "['MSFT', 'GOOGL', 'NVDA', 'SPOT', 'NFLX']\n" + ] + } + ], + "source": [ + "symbols = ['MSFT', 'GOOGL', 'NVDA', 'SPOT', 'NFLX']\n", + "print(symbols)" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 49, + "referenced_widgets": [ + "af6a2992bee5428aaf1b92e34ab8aee7", + "d7d8264ada2048d59aca65e41182fca4", + "a03a7e16c502430b93c5a9dbc06a2927" + ] }, + "id": "ohctqwc8m4K_", + "outputId": "2887a32a-fc02-48b7-8b47-9b046738402e" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "U8Tso3x0Xnu8" + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "af6a2992bee5428aaf1b92e34ab8aee7", + "version_major": 2, + "version_minor": 0 }, - "source": [ - ":::{.callout-note title=\"Note\"}\n", - "To make the Jupyter Widgets work in this simple use case, we first choose an option from the form. This updates the selected value displayed within the dropdown menu. Then we can reference the currently selected value.\n", - ":::" + "text/plain": [ + "Dropdown(description='Symbol: ', options=('MSFT', 'GOOGL', 'NVDA', 'SPOT', 'NFLX'), value='MSFT')" ] + }, + "metadata": {}, + "output_type": "display_data" } - ], - "metadata": { + ], + "source": [ + "from ipywidgets.widgets import Dropdown\n", + "\n", + "# configure the dropdown menu:\n", + "dropdown = Dropdown(\n", + " options=symbols,\n", + " value=symbols[0], # default value\n", + " description='Symbol: ',\n", + " disabled=False,\n", + ")\n", + "\n", + "# display the dropdown menu:\n", + "display(dropdown)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "L8cVJ7aRCF7_" + }, + "source": [ + "Using the selected value:" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { "colab": { - "provenance": [], - "toc_visible": true + "base_uri": "https://localhost:8080/" }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" + "id": "y-5jjLg7CD0w", + "outputId": "faffe3ed-40c8-4db9-e8d3-d26a35fae2ea" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "NVDA\n" + ] + } + ], + "source": [ + "print(dropdown.value)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "U8Tso3x0Xnu8" + }, + "source": [ + ":::{.callout-note title=\"Note\"}\n", + "To make the Jupyter Widgets work in this simple use case, we first choose an option from the form. This updates the selected value displayed within the dropdown menu. Then we can reference the currently selected value.\n", + "\n", + "![Dropdown input in Google Colab.](../../../images/jupyter-widgets-dropdown.png){height=350}\n", + ":::" + ] + } + ], + "metadata": { + "colab": { + "provenance": [], + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "a03a7e16c502430b93c5a9dbc06a2927": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } }, - "language_info": { - "name": "python" + "af6a2992bee5428aaf1b92e34ab8aee7": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DropdownModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DropdownModel", + "_options_labels": [ + "MSFT", + "GOOGL", + "NVDA", + "SPOT", + "NFLX" + ], + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "DropdownView", + "description": "Symbol: ", + "description_tooltip": null, + "disabled": false, + "index": 2, + "layout": "IPY_MODEL_d7d8264ada2048d59aca65e41182fca4", + "style": "IPY_MODEL_a03a7e16c502430b93c5a9dbc06a2927" + } }, - "widgets": { - "application/vnd.jupyter.widget-state+json": { - "a03a7e16c502430b93c5a9dbc06a2927": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "af6a2992bee5428aaf1b92e34ab8aee7": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DropdownModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DropdownModel", - "_options_labels": [ - "MSFT", - "GOOGL", - "NVDA", - "SPOT", - "NFLX" - ], - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "DropdownView", - "description": "Symbol: ", - "description_tooltip": null, - "disabled": false, - "index": 2, - "layout": "IPY_MODEL_d7d8264ada2048d59aca65e41182fca4", - "style": "IPY_MODEL_a03a7e16c502430b93c5a9dbc06a2927" - } - }, - "d7d8264ada2048d59aca65e41182fca4": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - } - } + "d7d8264ada2048d59aca65e41182fca4": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } } - }, - "nbformat": 4, - "nbformat_minor": 0 + } + } + }, + "nbformat": 4, + "nbformat_minor": 0 } diff --git a/docs/notes/dev-tools/google-colab/overview.qmd b/docs/notes/dev-tools/google-colab/overview.qmd index 6de8ea2..69b930a 100644 --- a/docs/notes/dev-tools/google-colab/overview.qmd +++ b/docs/notes/dev-tools/google-colab/overview.qmd @@ -98,7 +98,7 @@ After writing Python code like this in a code cell, we execute or "run" the code The sidebar menus provide additional functionality. The Table of Contents is fairly straightforward, but you can feel free to skip the rest and return to review other menus in the future, as instructed, when we have practical need for them. + [Table of Contents](./table-of-contents.qmd) - + [Colab Filesystem](./filesystem.ipynb) + + [Filesystem](./files.qmd) + [Forms and Inputs](./form-inputs.ipynb) + [Notebook Secrets](./notebook-secrets.qmd) + [Advanced Integrations](./advanced-integrations.ipynb) diff --git a/docs/notes/dev-tools/google-colab/pip.ipynb b/docs/notes/dev-tools/google-colab/pip.ipynb index 46c9306..17f6723 100644 --- a/docs/notes/dev-tools/google-colab/pip.ipynb +++ b/docs/notes/dev-tools/google-colab/pip.ipynb @@ -16,6 +16,8 @@ "\n", "But it is very common to want to use other packages in the Python ecosystem as well, including any packages we might find shared on [GitHub](https://github.com/), or hosted more officially on the Python Package Index ([PyPI](https://pypi.org/)), which is a centralized repository for all official Python packages.\n", "\n", + "![Python Package Index (PyPI) Logo.](../../../images/pypi-logo.png){height=250}\n", + "\n", "To manage the Python package installation process, we use Python's sidekick, a command-line tool called [Pip](https://packaging.python.org/en/latest/tutorials/installing-packages/#use-pip-for-installing).\n" ] }, diff --git a/docs/notes/dev-tools/index.qmd b/docs/notes/dev-tools/index.qmd index 1926645..e25825b 100644 --- a/docs/notes/dev-tools/index.qmd +++ b/docs/notes/dev-tools/index.qmd @@ -4,11 +4,14 @@ So you want to develop programs and applications in Python? You'll need to familiarize yourself with a Python development environment. One of the hardest parts of computer programming can be installing the language, and getting up and running with local development tools. But we will help guide you through the process, and recommend using tools that offer a low learning curve for beginners. +![Frustration with computers.](../../images/frustration-with-computers.png){height=320} + + In Python, we have a general choice between two development environments: Python notebooks vs local development, each offering its own strengths and trade-offs. Ultimately we recommend Google Colab notebooks for their accessibility and ease of use for beginners. ## Python Notebooks -Python notebooks (\".ipynb\" documents) provide an easy and intuitive learning experience for beginners. They have code \"cells\" we can "play" to see the results and learn along the way. The integration of visual outputs along with code provides strengths in terms of presentation. +**Python notebooks** (\".ipynb\" documents) provide an easy and intuitive learning experience for beginners. They have code \"cells\" we can "play" to see the results and learn along the way. The integration of visual outputs along with code provides strengths in terms of presentation. ![Python notebook with code input and dataviz output.](../../images/python-notebook-inputs-outputs.png){height=320} diff --git a/docs/notes/python-lang/file-operations.qmd b/docs/notes/python-lang/file-operations.qmd index f4a0079..6f31f1c 100644 --- a/docs/notes/python-lang/file-operations.qmd +++ b/docs/notes/python-lang/file-operations.qmd @@ -55,7 +55,7 @@ with open(filepath, "r") as file: ## Removing Files -Finally, we can use the `os` module to detect, and delete files: +As we've seen, we can use the [`os` module](../python-modules/os.ipynb) to detect and delete files: ```{python} import os diff --git a/docs/notes/python-modules/os.ipynb b/docs/notes/python-modules/os.ipynb new file mode 100644 index 0000000..56e8910 --- /dev/null +++ b/docs/notes/python-modules/os.ipynb @@ -0,0 +1,976 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "qNAdiHklyLhf" + }, + "source": [ + "# The `os` Module" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vgTIx3lx4QMT" + }, + "source": [ + "## Filesystem Operations\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "FzZ8bCfCodn6" + }, + "source": [ + "\n", + "The [`os` module](https://docs.python.org/3/library/os.html) helps us access and manipulate the file system.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tHcQfhazBX2m" + }, + "source": [ + "### Current Working Directory" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "o0EYbKxJpScy" + }, + "source": [ + "Detecting the name of the current working directory, using the [`getcwd` function](https://docs.python.org/3/library/os.html#os.getcwd):" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "id": "ZiSowuruobqs", + "outputId": "517edc8c-bcd3-4689-f038-779814517482" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'/content'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 1 + } + ], + "source": [ + "import os\n", + "\n", + "os.getcwd()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "pVJCJphfBX2o" + }, + "source": [ + "We see in Google Colab the default working directory is called \"content\"." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "CtE1TQbIBX2o" + }, + "source": [ + "### Listing Files" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "DpkyEiouqLXg" + }, + "source": [ + "Listing all files and folders that exist in a given directory (for example in the \"content\" directory where we are right now), using the [`listdir` function](https://docs.python.org/3/library/os.html#os.listdir):" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "stXz9D5PqOfC", + "outputId": "0aa7e816-3dee-48a4-b945-f95db7179f6d" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "['.config', 'sample_data']" + ] + }, + "metadata": {}, + "execution_count": 2 + } + ], + "source": [ + "os.listdir(\"/content\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0tzsYbvrrRsR" + }, + "source": [ + "We see there is a \"sample_data\" directory.\n", + "\n", + "After further inspection, we see it contains some example text and data files:" + ] + }, + { + "cell_type": "code", + "source": [ + "os.listdir(\"/content/sample_data\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "3Zjx7ZzJCdNG", + "outputId": "dcc98d5d-12e0-4b47-b812-5230ad3c5448" + }, + "execution_count": 3, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "['anscombe.json',\n", + " 'README.md',\n", + " 'california_housing_train.csv',\n", + " 'mnist_train_small.csv',\n", + " 'mnist_test.csv',\n", + " 'california_housing_test.csv']" + ] + }, + "metadata": {}, + "execution_count": 3 + } + ] + }, + { + "cell_type": "markdown", + "source": [ + ":::{.callout-note}\n", + "So far we have used an absolute file reference, but since we are already in the \"content\" directory, it is possible to use a relative file references instead. These references are relative to the \"content\" directory, where we are right now." + ], + "metadata": { + "id": "vKI0O7xwIMny" + } + }, + { + "cell_type": "code", + "source": [ + "os.listdir()" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "wWzDFlsXLAi0", + "outputId": "44e39f99-5815-410d-a7a3-ffac2fefcb94" + }, + "execution_count": 4, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "['.config', 'sample_data']" + ] + }, + "metadata": {}, + "execution_count": 4 + } + ] + }, + { + "cell_type": "code", + "source": [ + "os.listdir(\"sample_data\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "yvofS2THINPI", + "outputId": "fb385438-9aca-4e99-b22b-d0a3b9c3fb09" + }, + "execution_count": 5, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "['anscombe.json',\n", + " 'README.md',\n", + " 'california_housing_train.csv',\n", + " 'mnist_train_small.csv',\n", + " 'mnist_test.csv',\n", + " 'california_housing_test.csv']" + ] + }, + "metadata": {}, + "execution_count": 5 + } + ] + }, + { + "cell_type": "markdown", + "source": [ + ":::" + ], + "metadata": { + "id": "RHrWnjdFJAHX" + } + }, + { + "cell_type": "markdown", + "metadata": { + "id": "veHbkh2sBX2p" + }, + "source": [ + "### Detecting Directories and Files" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "z4JD64T3rcBq" + }, + "source": [ + "Checking to see whether a given directory or file exists, using the `isdir` and `isfile` functions from the `os.path` sub-module:\n" + ] + }, + { + "cell_type": "code", + "source": [ + "os.path.isdir(\"sample_data\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "QtSSz9FiLnHj", + "outputId": "44859195-f80a-46da-b2bb-05402519d374" + }, + "execution_count": 6, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "True" + ] + }, + "metadata": {}, + "execution_count": 6 + } + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "RLZdJSILrgiy", + "outputId": "ace33995-bd71-45e4-b39d-20ad16cf3c18" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "True" + ] + }, + "metadata": {}, + "execution_count": 7 + } + ], + "source": [ + "os.path.isfile(\"sample_data/README.md\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "rC65gC5IBX2r" + }, + "source": [ + "### Reading and Writing Files" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "c84DC2iZr4Y_" + }, + "source": [ + "See the [Text File Operations](../python-lang/file-operations.qmd) chapter for examples of how to read and write text files using the `open` function:" + ] + }, + { + "cell_type": "code", + "source": [ + "with open(\"sample_data/README.md\", \"r\") as file:\n", + " print(file.read())" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "pw1FwYUEEAiy", + "outputId": "de78fe0f-3ffc-421b-bfe8-17061e551de6" + }, + "execution_count": 8, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "This directory includes a few sample datasets to get you started.\n", + "\n", + "* `california_housing_data*.csv` is California housing data from the 1990 US\n", + " Census; more information is available at:\n", + " https://docs.google.com/document/d/e/2PACX-1vRhYtsvc5eOR2FWNCwaBiKL6suIOrxJig8LcSBbmCbyYsayia_DvPOOBlXZ4CAlQ5nlDD8kTaIDRwrN/pub\n", + "\n", + "* `mnist_*.csv` is a small sample of the\n", + " [MNIST database](https://en.wikipedia.org/wiki/MNIST_database), which is\n", + " described at: http://yann.lecun.com/exdb/mnist/\n", + "\n", + "* `anscombe.json` contains a copy of\n", + " [Anscombe's quartet](https://en.wikipedia.org/wiki/Anscombe%27s_quartet); it\n", + " was originally described in\n", + "\n", + " Anscombe, F. J. (1973). 'Graphs in Statistical Analysis'. American\n", + " Statistician. 27 (1): 17-21. JSTOR 2682899.\n", + "\n", + " and our copy was prepared by the\n", + " [vega_datasets library](https://github.com/altair-viz/vega_datasets/blob/4f67bdaad10f45e3549984e17e1b3088c731503d/vega_datasets/_data/anscombe.json).\n", + "\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "See [Getting Started with Pandas](https://prof-rossetti.github.io/applied-data-science-python-book/notes/pandas/obtaining-dataframes.html) for examples of how to read and write tabular data files using the `pandas` package:" + ], + "metadata": { + "id": "kvSy9q9PHpXq" + } + }, + { + "cell_type": "code", + "source": [ + "from pandas import read_csv\n", + "\n", + "df = read_csv(\"sample_data/california_housing_test.csv\")\n", + "df.head()" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 226 + }, + "id": "kIbHer2eHoPv", + "outputId": "d80821d8-852b-4535-ea5c-80e67207e22a" + }, + "execution_count": 9, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " longitude latitude housing_median_age total_rooms total_bedrooms \\\n", + "0 -122.05 37.37 27.0 3885.0 661.0 \n", + "1 -118.30 34.26 43.0 1510.0 310.0 \n", + "2 -117.81 33.78 27.0 3589.0 507.0 \n", + "3 -118.36 33.82 28.0 67.0 15.0 \n", + "4 -119.67 36.33 19.0 1241.0 244.0 \n", + "\n", + " population households median_income median_house_value \n", + "0 1537.0 606.0 6.6085 344700.0 \n", + "1 809.0 277.0 3.5990 176500.0 \n", + "2 1484.0 495.0 5.7934 270500.0 \n", + "3 49.0 11.0 6.1359 330000.0 \n", + "4 850.0 237.0 2.9375 81700.0 " + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_value
0-122.0537.3727.03885.0661.01537.0606.06.6085344700.0
1-118.3034.2643.01510.0310.0809.0277.03.5990176500.0
2-117.8133.7827.03589.0507.01484.0495.05.7934270500.0
3-118.3633.8228.067.015.049.011.06.1359330000.0
4-119.6736.3319.01241.0244.0850.0237.02.937581700.0
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "variable_name": "df", + "summary": "{\n \"name\": \"df\",\n \"rows\": 3000,\n \"fields\": [\n {\n \"column\": \"longitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.9949362939550161,\n \"min\": -124.18,\n \"max\": -114.49,\n \"num_unique_values\": 607,\n \"samples\": [\n -121.15,\n -121.46,\n -121.02\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"latitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2.1296695233438325,\n \"min\": 32.56,\n \"max\": 41.92,\n \"num_unique_values\": 587,\n \"samples\": [\n 40.17,\n 33.69,\n 39.61\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"housing_median_age\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12.555395554955755,\n \"min\": 1.0,\n \"max\": 52.0,\n \"num_unique_values\": 52,\n \"samples\": [\n 14.0,\n 49.0,\n 7.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_rooms\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2155.59333162558,\n \"min\": 6.0,\n \"max\": 30450.0,\n \"num_unique_values\": 2215,\n \"samples\": [\n 1961.0,\n 1807.0,\n 680.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_bedrooms\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 415.6543681363232,\n \"min\": 2.0,\n \"max\": 5419.0,\n \"num_unique_values\": 1055,\n \"samples\": [\n 532.0,\n 764.0,\n 2162.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"population\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1030.5430124122422,\n \"min\": 5.0,\n \"max\": 11935.0,\n \"num_unique_values\": 1802,\n \"samples\": [\n 947.0,\n 1140.0,\n 2019.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"households\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 365.42270980552604,\n \"min\": 2.0,\n \"max\": 4930.0,\n \"num_unique_values\": 1026,\n \"samples\": [\n 646.0,\n 629.0,\n 504.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"median_income\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1.854511729691481,\n \"min\": 0.4999,\n \"max\": 15.0001,\n \"num_unique_values\": 2578,\n \"samples\": [\n 1.725,\n 0.7403,\n 2.6964\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"median_house_value\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 113119.68746964433,\n \"min\": 22500.0,\n \"max\": 500001.0,\n \"num_unique_values\": 1784,\n \"samples\": [\n 71900.0,\n 63000.0,\n 115800.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 9 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "_17RzoYDBX2q" + }, + "source": [ + "### Deleting Files" + ] + }, + { + "cell_type": "code", + "source": [ + "filepath = \"sample_data/anscombe.json\"\n", + "\n", + "# verifying the file exists:\n", + "os.path.isfile(filepath)" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "it4IQL4sNC7o", + "outputId": "55dc1ebc-ee67-4ec1-d8fd-8aaf2913fdb6" + }, + "execution_count": 10, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "True" + ] + }, + "metadata": {}, + "execution_count": 10 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wLw8JfYL2tdc" + }, + "source": [ + "Deleting a file, using the [`remove` function](https://docs.python.org/3/library/os.html#os.remove):" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "id": "oGSWOswR2u9A" + }, + "outputs": [], + "source": [ + "os.remove(filepath)" + ] + }, + { + "cell_type": "code", + "source": [ + "# verifying the file was deleted:\n", + "os.path.isfile(filepath)" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "zLBd1XF-BdSo", + "outputId": "eb30b9ef-4393-4997-dfab-d9fafa726358" + }, + "execution_count": 12, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "False" + ] + }, + "metadata": {}, + "execution_count": 12 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wYhlyEdzBX2q" + }, + "source": [ + "### Creating and Deleting Directories" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xgRz036LpyiW" + }, + "source": [ + "Creating a new directory using the [`makedirs` function](https://docs.python.org/3/library/os.html#os.makedirs):" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "id": "vugRrjZvpyA5" + }, + "outputs": [], + "source": [ + "os.makedirs(\"my_data\", exist_ok=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "hUSjUFttq24k", + "outputId": "5ad3fd93-8221-40e9-89c9-13cdc10219aa" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "['.config', 'my_data', 'sample_data']" + ] + }, + "metadata": {}, + "execution_count": 14 + } + ], + "source": [ + "# verifying the \"my_data\" directory got created:\n", + "os.listdir()" + ] + }, + { + "cell_type": "code", + "source": [ + "#| echo: false\n", + "\n", + "#df.to_csv(\"my_data/california_housing_test.csv\", index=False)" + ], + "metadata": { + "id": "t_vqxL5IQuUZ" + }, + "execution_count": 19, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "To delete an empty directory, we can use the `rmdir` function from the `os` module, however it only works for empty directories and throws an error if the directory does not exist. So for a more robust solution, we can use the [`rmtree` function](https://docs.python.org/3/library/shutil.html#shutil.rmtree) from the `shutil` module:" + ], + "metadata": { + "id": "2rtE_R03PSFQ" + } + }, + { + "cell_type": "code", + "source": [ + "#os.rmdir(\"my_data\")" + ], + "metadata": { + "id": "x8cRgG0IPqo3" + }, + "execution_count": 21, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "from shutil import rmtree\n", + "\n", + "rmtree(\"my_data\", ignore_errors=True)" + ], + "metadata": { + "id": "LjpFLVTFOoIx" + }, + "execution_count": 16, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "# verifying the \"my_data\" directory got deleted:\n", + "os.listdir()" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "fOk_jjKTOrgJ", + "outputId": "c86a055e-3ba6-42dd-fdce-2bec8269b4e6" + }, + "execution_count": 17, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "['.config', 'sample_data']" + ] + }, + "metadata": {}, + "execution_count": 17 + } + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file diff --git a/docs/notes/python-modules/os.qmd b/docs/notes/python-modules/os.qmd deleted file mode 100644 index e69de29..0000000 diff --git a/docs/why-python.qmd b/docs/why-python.qmd index df71ef5..5b5cd56 100644 --- a/docs/why-python.qmd +++ b/docs/why-python.qmd @@ -44,9 +44,9 @@ The language is well-documented, providing extensive resources and help when nee ## Developer Ecosystem -Python boasts a rich ecosystem of open-source third-party libraries or plugins known as *packages*. These packages provide users with a wide array of tools and functionalities that extend the core capabilities of Python. This extensibility makes it easier to perform complex tasks without needing to write code from scratch, thereby saving time and effort for developers and analysts. +Python boasts a rich ecosystem of open-source third-party libraries or plugins known as **packages**. These packages provide users with a wide array of tools and functionalities that extend the core capabilities of Python. This extensibility makes it easier to perform complex tasks without needing to write code from scratch, thereby saving time and effort for developers and analysts. -Python packages offer powerful capabilities in data processing, analysis, visualization, and modeling. This means professionals can efficiently handle large datasets, perform sophisticated analyses, create insightful visualizations, and train predictive models. This robust ecosystem makes Python an indispensable tool in the field of data science, empowering users to develop advanced applications and decision support tools. +Python packages offer powerful capabilities in software development, as well as in data processing, analysis, visualization, and modeling. This means professionals can efficiently build web applications, handle large datasets, perform sophisticated analyses, create insightful visualizations, and train predictive models. This robust ecosystem makes Python an indispensable tool in the field of data science, empowering users to develop advanced applications and decision support tools. ## Versatility