Add runbooks with examples for localdev/dev/statistics (#23286)

* Add runbooks with examples for localdev/dev/statistics

* Update .dockerignore

* Update .gitignore

* Update settings.py

Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>

---------

Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>

Author: KevinMind

docs/_static/images/statistics-overview-chart.png

@@ -19,6 +19,7 @@ topics/remote_addr
 topics/third-party
 topics/blocklist
 topics/environments
+topics/runbooks/index
 ```
 
 ## Archived Contents

docs/index.md

@@ -0,0 +1,10 @@
+# Dev Environment
+
+Runbooks designed to run on the `dev` environment.
+These should *not* be used in local/stage/production environments.
+
+```{toctree}
+:maxdepth: 2
+
+shell_into_dev_pod
+```

docs/topics/runbooks/dev/index.md

@@ -0,0 +1,3 @@
+# Shell into a dev pod
+
+See <https://mozilla-hub.atlassian.net/wiki/spaces/FDPDT/pages/98795521/DevOps#Bastion-Hosts> for details.

docs/topics/runbooks/dev/shell_into_dev_pod.md

@@ -0,0 +1,33 @@
+# Runbooks
+
+A set of runbooks for the AMO team. These documents should enable anyone to perform
+common tasks involved in triaging issues, verifying patches, and maintaining our codebase.
+
+```{toctree}
+:maxdepth: 2
+
+localdev/index
+dev/index
+```
+
+## Example
+
+Follow this template to create a new runbook.
+
+```{tip}
+Use ### headings to separate steps so steps will be visible in the table of contents.
+```
+
+```markdown
+# Runbook Title
+
+Brief description of the runbook.
+
+## Steps
+
+### Step 1
+
+### Step 2
+
+### Step 3
+```

docs/topics/runbooks/index.md

@@ -0,0 +1,44 @@
+# BigQuery Credentials Setup
+
+Setup bigquery credentials for dev on local environment.
+This will allow you to connect to the dev BigQuery database
+and run queries.
+
+## Steps
+
+### Get dev bigquery credentials json file
+
+[Shell into a dev pod](../dev/shell_into_dev_pod.md)
+
+    a. create a django shell `make djshell`
+    b. get the value of `settings.GOOGLE_APPLICATION_CREDENTIALS_BIGQUERY`
+    c. exit the django shell and cat the the file
+    d. copy the contents to a file on your local machine
+
+```{warning}
+This file contains sensitive secrets. Save it in `private/google-application-credentials.json`
+which is explicitly gitignored to prevent it from being checked into git.
+Do not share this file with anyone.
+```
+
+### Run the project locally
+
+    a. update `local_settings.py` to point `GOOGLE_APPLICATION_CREDENTIALS_BIGQUERY` to the file on your local machine
+    b. run `make up` to start the django server with this value enabled
+
+### Verify it is working
+
+Verify you can connect to bigquery by running `make djshell` and then:
+
+```python
+from olympia.stats.utils import *
+client = create_client()
+client.project
+```
+
+Expect to get a response like:
+
+```bash
+In [3]: client.project
+Out[3]: 'moz-fx-amo-environment-abc123'
+```

docs/topics/runbooks/localdev/bigquery_credentials_setup.md

@@ -0,0 +1,11 @@
+# Local Development
+
+Runbooks designed to run on a local development environment.
+These should *not* be used in dev/stage/production environments.
+
+```{toctree}
+:maxdepth: 2
+
+bigquery_credentials_setup
+test_statistics_dashboard
+```

docs/topics/runbooks/localdev/index.md

@@ -0,0 +1,35 @@
+# Test Statistics Dashboard
+
+Test the devhub statistics dashboard locally with bigquery credentials.
+
+## Steps
+
+### Setup bigquery credentials
+
+[Bigquery Credentials Setup](./bigquery_credentials_setup.md)
+
+### Enable a local addon to read from bigquery dev database
+
+Only a [subset of addons][enabled_dev_guids] in dev push data to bigquery. In order to read from bigquery
+your local addon will need to use a matching `guid`.
+
+Open a django shell `make djshell` and run the following:
+
+- pk: the database id of the addon you want to use
+- guid: the guid from the [list][enabled_dev_guids] you want to use (e.g. `@facebook-container`)
+
+```python
+addon = Addon.objects.get(pk=<pg>)
+addon.update(guid=<guid>)
+addon.save()
+```
+
+### Run the statistics dashboard
+
+Open your addon `<http://olympia.test/en-US/firefox/addon/<pk>/statistics/>`
+
+You should see the default overview chart. It should look like this:
+
+![alt text](../../../_static/images/statistics-overview-chart.png)
+
+[enabled_dev_guids]: https://github.com/mozilla/bigquery-etl/blob/main/sql/moz-fx-data-shared-prod/amo_dev/amo_stats_dau_v2/query.sql

docs/topics/runbooks/localdev/test_statistics_dashboard.md

@@ -171,3 +171,9 @@ def insert_debug_toolbar_middleware(middlewares):
     }
 
 MEMCACHE_MIN_SERVER_COUNT = 1
+
+GOOGLE_APPLICATION_CREDENTIALS_BIGQUERY = env(
+    'GOOGLE_APPLICATION_CREDENTIALS_BIGQUERY',
+    # Set default path to the path explicitly gitignored.
+    default=path('private/google-application-credentials.json'),
+)