updated docs

Author: BrainIsDead

src/docs/build/doctrees/base.tests.doctree

@@ -0,0 +1,91 @@
+Import data from admin interface
+================================
+
+For data import used
+`django-import-export <https://django-import-export.readthedocs.io/en/latest/index.html>`__
+library
+
+The code consists of Django resource classes that are used for importing
+and exporting CSV files using the Django Import-Export library:
+
+Resources
+---------
+
+1. ``SignalResource`` - this resource class is used for importing ``Signal`` models:
+
+   - Defines various fields such as ``name``, ``display_name``, ``pathogen``, ``signal_type``, and ``source``.
+   - Defines the ``before_import_row`` method, which is called before importing each row and allows for pre-processing of the data.
+   - Includes methods like ``is_url_in_domain`` to check if a URL belongs to a specific domain, ``fix_boolean_fields`` to handle boolean fields, and ``process_links`` to process the links field.
+
+2. ``SignalBaseResource`` - This resource class is used for updating
+   already created ``Signal`` models ``base`` fields with base Signals:
+
+   -  Defines various fields such as ``name``, ``display_name``,
+      ``base``, and ``source``.
+   -  Defines the ``before_import_row`` method, which is called before
+      importing each row and allows for pre-processing of the data.
+   -  The ``process_base`` method is responsible for processing the
+      ``base`` field by retrieving the corresponding ``Signal`` object
+      based on the provided ``name`` and ``source``.
+
+3. ``SourceSubdivisionResource`` - this resource class is used for
+   importing ``SourceSubdivision`` models:
+
+   -  It defines fields such as ``name``, ``display_name``,
+      ``description``, ``data_source``, and ``links``.
+   -  It includes the ``before_import_row`` method for pre-processing
+      each row before importing.
+   -  The ``process_links`` method is responsible for processing the
+      ``links`` field by creating ``Link`` objects based on the provided
+      URLs.
+   -  The ``process_datasource`` method processes the ``data_source``
+      field by creating or retrieving a ``DataSource`` object based on
+      the provided name.
+
+These resource classes provide a structured way to import CSV files.
+They define the fields, handle pre-processing of data, and interact with
+the corresponding models and related objects.
+
+Import data flow
+----------------
+
+CSV preparation
+~~~~~~~~~~~~~~~
+
+To import data from a CSV file must meet the requirements:
+   - CSV file should be properly formatted and contains all the required fields for
+     importing, as specified by the resource classes (``SignalResource``, ``SignalBaseResource``, ``SourceSubdivisionResource``).
+   - The header rowof the CSV file should match the field names defined in the resource classes.
+   - It should not contain empty rows (empty rows may cause validation errors during the import)
+   - Colums should be saparateb by ``","``
+
+Othervice you will receive Errors during import process:
+
+|Import errors|
+
+Data import
+~~~~~~~~~~~
+
+1. Import ``SourceSubdivision`` instances with
+   ``SourceSubdivisionResource`` -
+   http://localhost:8000/admin/datasources/sourcesubdivision/import/
+   |Import ``SourceSubdivision`` instances| |Confirm importing
+   ``SourceSubdivision`` instances|
+
+
+2. Import ``Signal`` instances with ``SignalResource`` -
+   http://localhost:8000/admin/signals/signal/import/ |Import ``Signal``
+   instances| |Confirm importing ``Signal`` instances|
+
+
+3. Import ``Signal.base`` fields with ``SignalBaseResource`` -
+   http://localhost:8000/admin/signals/signal/import/ |Import
+   ``Signal.base`` field| |Confirm importing ``Signal.base`` fields|
+
+.. |Import errors| image:: ./_static/import_errors.png
+.. |Import ``SourceSubdivision`` instances| image:: ./_static/import_source_subdivision.png
+.. |Confirm importing ``SourceSubdivision`` instances| image:: ./_static/import_confirm_source_subdivision.png
+.. |Import ``Signal`` instances| image:: ./_static/import_signals.png
+.. |Confirm importing ``Signal`` instances| image:: ./_static/import_confirm_signals.png
+.. |Import ``Signal.base`` field| image:: ./_static/import_signals_base.png
+.. |Confirm importing ``Signal.base`` fields| image:: ./_static/import_confirm_signals_base.png

src/docs/build/doctrees/data_import.doctree

@@ -0,0 +1,32 @@
+Deployment
+==========
+
+This application gets deployed (at a minimum) to two environmetns:
+
+Production -
+`https://delphi.cmu.edu/{app_name} <https://delphi.cmu.edu/{app_name}>`__
+
+Staging -
+`https://staging.delphi.cmu.edu/{app_name} <https://staging.delphi.cmu.edu/{app_name}>`__
+
+Each environment is essentially a bunch of different services all
+governed by ``docker-compose``, running across multiple hosts, with
+various layering of proxies and load balancers.
+
+Basic workflow
+--------------
+
+-  A PR merged to either ``development`` or ``master`` will trigger CI
+   to build container images that are then tagged (based on the branch
+   name and ":latest" respectively) and stored in our GitHub Packages
+   container image repository.
+-  CI triggers a webhook that tells the host systems to pull and run new
+   container images and restart any services that have been updated.
+
+Control of the deployed environment
+-----------------------------------
+
+The environment and secrets used for deployment live in
+https://github.com/cmu-delphi/delphi-ansible-web. Any changes to the
+environment should be made there and then tested and validated by devops
+folks.

src/docs/build/doctrees/datasources.tests.doctree

@@ -10,6 +10,9 @@ Welcome to Signal Documentation's documentation!
    :maxdepth: 2
    :caption: Contents:
 
+   overview
+   data_import
+   deployment
    modules
 
 Indices and tables

src/docs/build/doctrees/deployment.doctree

@@ -1,5 +1,5 @@
-src
-====
+Signal Documentation Modules
+============================
 
 .. toctree::
    :maxdepth: 4

src/docs/build/doctrees/environment.pickle

@@ -0,0 +1,163 @@
+Overview and installation
+=========================
+
+Single Source of Documentation System
+
+.. image:: ./_static/signal_documentation_schema.jpg
+   :alt: Signal Documentation Schema
+
+
+Entity Relationship Diagram
+---------------------------
+
+.. image:: ./_static/models.png
+   :alt: Signal Documentation ERD
+
+Core libs and DB
+----------------
+
+1. `Django <https://www.djangoproject.com/>`__
+2. `Django-filter <https://django-filter.readthedocs.io/en/stable/index.html>`__
+3. `PostgreSQL <https://www.postgresql.org/>`__
+
+All requirements you can find in ``Pipfile``
+
+Getting started
+---------------
+
+Setup Env Vars
+~~~~~~~~~~~~~~
+
+create ``.env`` file and add variables like in ``.env.example``
+
+To run locally
+~~~~~~~~~~~~~~
+
+Install ``python:3.10``, ``pip3``, ``pipenv``
+
+Using `pipenv <https://github.com/pypa/pipenv>`__ run ``pipenv shell``
+and ``pipenv install`` to create virtual environment and install
+dependencies
+
+.. code:: sh
+
+   $ pipenv shell
+   $ pipenv install
+
+Go to ``src`` directory and run
+
+.. code:: sh
+
+   $ python manage.py migrate
+   $ python manage.py test
+   $ python manage.py runserver
+
+load fixtures
+
+.. code:: sh
+
+   $ python manage.py loaddata .\fixtures\available_geography.json
+   $ python manage.py loaddata .\fixtures\pathogens.json
+   $ python manage.py loaddata .\fixtures\signal_categories.json
+   $ python manage.py loaddata .\fixtures\signal_types.json
+
+if you need test coverage
+
+.. code:: sh
+
+   $ coverage erase
+   $ coverage python manage.py test
+   $ coverage report
+
+you can also get test coverage with one command
+
+.. code:: sh
+
+   $ coverage erase && coverage run manage.py test && coverage report
+
+sort imports
+
+.. code:: sh
+
+   $  isort .
+
+check flake
+
+.. code:: sh
+
+   $  flake8 --show-source
+
+To run via docker
+~~~~~~~~~~~~~~~~~
+
+Install ``Docker`` and ``docker-compose``
+
+Run
+
+.. code:: sh
+
+   $ docker-compose build
+   $ docker-compose up
+
+Open ``http://localhost:8000`` to view it in the browser
+
+To run via docker and emulate production
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Though probably not necessary in most cases, if you want to
+test/modify/emulate how this will run in production you can:
+
+-  In ``.env`` set:
+
+::
+
+   DEBUG = 'False'
+
+-  Modify the app container's command in ``docker-compose.yaml`` to run:
+
+::
+
+   "gunicorn signal_documentation.wsgi:application --bind 0.0.0.0:8000"
+
+   *(Essentially you'll replace just the last line of the command, switching out the "runserver" line)
+
+Open ``http://localhost`` to view it in the browser. In this usage your
+request will be serviced by Nginx instead of the application directly.
+
+The primary use case for this will be when making changes to the Nginx
+container image that runs in production and hosts the static file
+content, or also if making changes to the Gunicorn config.
+
+Changes of this sort should be carefully evaluated as they may require
+interaction with systems managed by devops folks.
+
+
+`Django admin <https://docs.djangoproject.com/en/4.1/ref/contrib/admin/>`__ web interface
+-----------------------------------------------------------------------------------------
+Django admin is a web interface for managing the project web application. It is available at the following URL.
+
+``http://localhost:8000/admin``
+
+The user should have ``is_staff`` or ``is_superuser`` permissions to access the admin interface.
+
+
+Read the docs (Sphynx)
+----------------------
+Auto generated documentation for the project web appplication is available at the following URL.
+
+``http://localhost:8000/<MAIN_PAGE>/docs/index.html``
+
+To clean the documentation, run the following commands:
+
+.. code:: sh
+
+   $ cd ./docs
+   $ make clean
+
+
+To generate the documentation, run the following commands:
+
+.. code:: sh
+
+   $ cd ./docs
+   $ make html

src/docs/build/doctrees/index.doctree

@@ -20,7 +20,7 @@
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
     <link rel="next" title="base.migrations package" href="base.migrations.html" />
-    <link rel="prev" title="src" href="modules.html" />
+    <link rel="prev" title="Signal Documentation Modules" href="modules.html" />
 </head>
 
 <body class="wy-body-for-nav">
@@ -44,7 +44,10 @@
         </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
               <p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
 <ul class="current">
-<li class="toctree-l1 current"><a class="reference internal" href="modules.html">src</a><ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="overview.html">Overview and installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="data_import.html">Import data from admin interface</a></li>
+<li class="toctree-l1"><a class="reference internal" href="deployment.html">Deployment</a></li>
+<li class="toctree-l1 current"><a class="reference internal" href="modules.html">Signal Documentation Modules</a><ul class="current">
 <li class="toctree-l2 current"><a class="current reference internal" href="#">base package</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="#subpackages">Subpackages</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="base.migrations.html">base.migrations package</a></li>
@@ -96,7 +99,7 @@
           <div role="navigation" aria-label="Page navigation">
   <ul class="wy-breadcrumbs">
       <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
-          <li class="breadcrumb-item"><a href="modules.html">src</a></li>
+          <li class="breadcrumb-item"><a href="modules.html">Signal Documentation Modules</a></li>
       <li class="breadcrumb-item active">base package</li>
       <li class="wy-breadcrumbs-aside">
             <a href="_sources/base.rst.txt" rel="nofollow"> View page source</a>
@@ -480,7 +483,7 @@ <h2>Submodules<a class="headerlink" href="#submodules" title="Link to this headi
            </div>
           </div>
           <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
-        <a href="modules.html" class="btn btn-neutral float-left" title="src" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+        <a href="modules.html" class="btn btn-neutral float-left" title="Signal Documentation Modules" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
         <a href="base.migrations.html" class="btn btn-neutral float-right" title="base.migrations package" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
     </div>
 

src/docs/build/doctrees/modules.doctree

@@ -44,7 +44,10 @@
         </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
               <p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
 <ul class="current">
-<li class="toctree-l1 current"><a class="reference internal" href="modules.html">src</a><ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="overview.html">Overview and installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="data_import.html">Import data from admin interface</a></li>
+<li class="toctree-l1"><a class="reference internal" href="deployment.html">Deployment</a></li>
+<li class="toctree-l1 current"><a class="reference internal" href="modules.html">Signal Documentation Modules</a><ul class="current">
 <li class="toctree-l2 current"><a class="reference internal" href="base.html">base package</a><ul class="current">
 <li class="toctree-l3 current"><a class="reference internal" href="base.html#subpackages">Subpackages</a><ul class="current">
 <li class="toctree-l4 current"><a class="current reference internal" href="#">base.migrations package</a></li>
@@ -80,7 +83,7 @@
           <div role="navigation" aria-label="Page navigation">
   <ul class="wy-breadcrumbs">
       <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
-          <li class="breadcrumb-item"><a href="modules.html">src</a></li>
+          <li class="breadcrumb-item"><a href="modules.html">Signal Documentation Modules</a></li>
           <li class="breadcrumb-item"><a href="base.html">base package</a></li>
       <li class="breadcrumb-item active">base.migrations package</li>
       <li class="wy-breadcrumbs-aside">

src/docs/build/doctrees/overview.doctree

@@ -44,7 +44,10 @@
         </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
               <p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
 <ul class="current">
-<li class="toctree-l1 current"><a class="reference internal" href="modules.html">src</a><ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="overview.html">Overview and installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="data_import.html">Import data from admin interface</a></li>
+<li class="toctree-l1"><a class="reference internal" href="deployment.html">Deployment</a></li>
+<li class="toctree-l1 current"><a class="reference internal" href="modules.html">Signal Documentation Modules</a><ul class="current">
 <li class="toctree-l2 current"><a class="reference internal" href="base.html">base package</a><ul class="current">
 <li class="toctree-l3 current"><a class="reference internal" href="base.html#subpackages">Subpackages</a><ul class="current">
 <li class="toctree-l4"><a class="reference internal" href="base.migrations.html">base.migrations package</a></li>
@@ -80,7 +83,7 @@
           <div role="navigation" aria-label="Page navigation">
   <ul class="wy-breadcrumbs">
       <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
-          <li class="breadcrumb-item"><a href="modules.html">src</a></li>
+          <li class="breadcrumb-item"><a href="modules.html">Signal Documentation Modules</a></li>
           <li class="breadcrumb-item"><a href="base.html">base package</a></li>
       <li class="breadcrumb-item active">base.tests package</li>
       <li class="wy-breadcrumbs-aside">
@@ -106,7 +109,7 @@ <h2>Submodules<a class="headerlink" href="#submodules" title="Link to this headi
 <p>A factory for the Link model.</p>
 <dl class="py attribute">
 <dt class="sig sig-object py" id="base.tests.factories.LinkFactory.link_type">
-<span class="sig-name descname"><span class="pre">link_type</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'question_text'</span></em><a class="headerlink" href="#base.tests.factories.LinkFactory.link_type" title="Link to this definition"></a></dt>
+<span class="sig-name descname"><span class="pre">link_type</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'wave_11_revision'</span></em><a class="headerlink" href="#base.tests.factories.LinkFactory.link_type" title="Link to this definition"></a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">