Merge pull request #32 from nexB/31-integration-docs

tdruez · web-flow · commit 853fc3d9adda · 2024-01-16T19:38:48.000-07:00
Improve documentation for integrations setup #31
diff --git a/docs/application-settings.rst b/docs/application-settings.rst
@@ -63,34 +63,6 @@ responsible to provide your own validation of the Host header.
 
     ALLOWED_HOSTS=*
 
-.. _dejacode_settings_purldb:
-
-PURLDB
-------
-
-Provide the URL and API key of your `PurlDB <https://github.com/nexB/purldb/>`_
-instance.
-
-::
-
-    PURLDB_URL=https://your-purldb-domain/
-    PURLDB_API_KEY=apikeyexample
-
-.. _dejacode_settings_vulnerablecode:
-
-VULNERABLECODE
---------------
-
-You can either run your own instance of
-`VulnerableCode <https://github.com/nexB/vulnerablecode/>`_
-or connect to the public one.
-
-Authentication is provided using an API key that you can obtain by registering at
-https://public.vulnerablecode.io/account/request_api_key/ ::
-
-    VULNERABLECODE_URL=https://public.vulnerablecode.io/
-    VULNERABLECODE_API_KEY=apikeyexample
-
 EMAIL
 -----
 
@@ -187,11 +159,11 @@ to provide some progress about pipeline run execution.
 
 Default: ``INFO``
 
-The ``DEBUG`` value can be provided to this setting to see all ScanCode.io debug
+The ``DEBUG`` value can be provided to this setting to see all DejaCode debug
 messages to help track down configuration issues for example.
 This mode can be enabled globally through the ``.env`` file::
 
-    SCANCODEIO_LOG_LEVEL=DEBUG
+    DEJACODE_LOG_LEVEL=DEBUG
 
 .. _clamd-settings:
 
@@ -222,6 +194,77 @@ default the ``US/Pacific`` time zone is used::
     You can view a detailed list of time zones `here.
     <https://en.wikipedia.org/wiki/List_of_tz_database_time_zones>`_
 
+.. _dejacode_settings_aboutcode_integrations:
+
+AboutCode integrations
+======================
+
+To **integrate DejaCode with other applications within the AboutCode stack**,
+you have the flexibility to configure and set up integrations using the following
+application settings.
+
+It's important to understand that employing application settings will make these
+integrations **globally accessible across all Dataspaces** within your DejaCode
+instance.
+
+Alternatively, if you wish to tailor the availability of these features to a specific
+Dataspace, you can define and set those values directly within the :ref:`dataspace`
+configuration. This can be done through the Dataspace admin UI, allowing you to scope
+the availability of these integrations exclusively to the designated Dataspace.
+
+.. _dejacode_settings_scancodeio:
+
+SCANCODEIO
+----------
+
+Provide the URL and API key of your `ScanCode.io <https://github.com/nexB/scancode.io>`_
+instance.
+
+.. code-block:: python
+
+    SCANCODEIO_URL=https://your_scancodeio.url/
+    SCANCODEIO_API_KEY=insert_your_api_key_here
+
+.. note::
+    You have the option to define and set those settings directly on your Dataspace.
+    For detailed instructions, refer to :ref:`dejacode_dataspace_scancodeio`.
+
+.. _dejacode_settings_purldb:
+
+PURLDB
+------
+
+Provide the URL and API key of your `PurlDB <https://github.com/nexB/purldb>`_ instance.
+
+.. code-block:: python
+
+    PURLDB_URL=https://your-purldb.url/
+    PURLDB_API_KEY=insert_your_api_key_here
+
+.. note::
+    You have the option to define and set those settings directly on your Dataspace.
+    For detailed instructions, refer to :ref:`dejacode_dataspace_purldb`.
+
+.. _dejacode_settings_vulnerablecode:
+
+VULNERABLECODE
+--------------
+
+You can either run your own instance of
+`VulnerableCode <https://github.com/nexB/vulnerablecode>`_
+or connect to the public one https://public.vulnerablecode.io/.
+
+.. note:: Providing an API key is optional when using the public VulnerableCode instance.
+
+.. code-block:: python
+
+    VULNERABLECODE_URL=https://public.vulnerablecode.io/
+    VULNERABLECODE_API_KEY=insert_your_api_key_here
+
+.. note::
+    You have the option to define and set those settings directly on your Dataspace.
+    For detailed instructions, refer to :ref:`dejacode_dataspace_vulnerablecode`.
+
 LDAP Integration
 ================
 
diff --git a/docs/dataspace.rst b/docs/dataspace.rst
@@ -1,7 +1,39 @@
+.. _dataspace:
+
 =========
 Dataspace
 =========
 
+The Dataspace serves as a pivotal mechanism within DejaCode, facilitating the
+segregation of data for each organization while maintaining a unified storage
+structure in the same database, schema, or table.
+Within a given installation, multiple "Dataspace" organizations can be defined,
+but there exists only one reference.
+
+This concept is a crucial element employed across DejaCode to effectively separate
+**reference data provided by nexB** from the data utilized in a specific DJE
+installation.
+Essentially, it introduces the notion of a "tenant" within a DJE installation,
+enabling the isolation of organization-specific and/or private records.
+This segregation supports both multi-tenancy and the coexistence of nexB-provided
+reference data and organization-specific or customized data.
+
+The key purposes of this separation include:
+
+1. **Orderly and Simplified Data Updates**: Facilitates smooth and streamlined updates
+   from nexB reference data, ensuring efficient data synchronization and exchange
+   across Dataspaces.
+2. **Dataspace-Specific Customizations**: Allows for customization of Dataspace-specific
+   data, such as configurations for license tags or specific preferences, tailoring
+   the installation to the unique needs of each organization.
+3. **Support for Multi-Tenancy**: Enables the sharing of the same DJE instance among
+   different organizations, each operating within its distinct Dataspace,
+   promoting multi-tenancy while maintaining data segregation.
+
+In summary, the Dataspace concept in DejaCode plays a pivotal role in maintaining data
+integrity, enabling efficient updates, accommodating customization, and supporting
+multi-tenancy for a diverse range of organizations within a DJE installation.
+
 Setting up your License Library
 ===============================
 
@@ -121,7 +153,7 @@ you do not need.
 Assigning Usage Policies to licenses
 ====================================
 
-There are more than **1,300 Licenses** in your License list from Reference Data, so
+There are more than **2,000 Licenses** in your License list from Reference Data, so
 you will want to develop a strategy that works for your business requirements to
 assign Usage Policies efficiently. The two basic techniques are:
 
@@ -130,7 +162,7 @@ assign Usage Policies efficiently. The two basic techniques are:
 2. Mass update: Select a group of licenses on the **Browse License** form and use Mass
    Update to assign a Usage Policy to that group of licenses.
 
-As an example of the first technique, locate and select the license with the Key
+As an example of the first technique, locate and select the license with the key
 of **apache-2.0** in your list. Your organization probably already has a policy
 regarding this very common license; you may simply allow engineering to use
 components under that license or you may require additional review of how they
@@ -278,6 +310,8 @@ The basic techniques to use in your own Dataspace are:
    **Set usage policy from licenses** option in the dropdown list in the lower
    right hand corner of the form and follow the prompts to complete that action.
 
+.. _dejacode_dataspace_scancodeio:
+
 Enable package scanning with your ScanCode.io server
 ====================================================
 
@@ -295,9 +329,16 @@ You can:
 * Download the scan results to a JSON-formatted file to integrate with other
   analysis and reporting tools.
 
+Install and configure ScanCode.io
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
 .. warning::
-    The ScanCode.io server **should not be installed on the same server** (virtual or
-    physical) as the DejaCode one.
+    If you plan to run ScanCode.io **on the same server** (virtual or  physical) as
+    the DejaCode instance, **ensure that the host machine has sufficient resources**
+    to handle both applications.
+    Also, you will have to provide custom network ports for the ScanCode.io application
+    as the ports 80 and 443 will be already used by the DejaCode application.
+    See https://scancodeio.readthedocs.io/en/latest/installation.html#use-alternative-http-ports
 
 1. Install a ScanCode.io server following instructions at
    https://scancodeio.readthedocs.io/en/latest/installation.html
@@ -321,29 +362,21 @@ You can:
    DejaCode instance:
    https://scancodeio.readthedocs.io/en/latest/command-line-interface.html#scanpipe-create-user-username
 
-4. Set the ScanCode.io server URL and the API key in your local DejaCode settings file
-   ``.env``.
-
-  .. code-block:: python
-
-      SCANCODEIO_URL=https://<your_scancodeio.url>/
-      SCANCODEIO_API_KEY=<api_key>
-
-  **Restarting the services is required following any changes to .env:**
+4. Set the ScanCode.io Server URL and API key in your Dataspace Configuration:
 
-  .. code-block:: bash
-
-      docker compose restart web worker
-
-5. Enable package scanning on your Dataspace from the **Administration dashboard**.
-
-   Select **Dataspaces** and open your Dataspace definition.
-   In the **Application Process Settings** section, check the
-   **Enable package scanning** option and save.
+ - Access your DejaCode web application **Administration dashboard**.
+ - Navigate to the **Dataspaces** section and select your Dataspace name.
+ - Within the **Application Process Settings** section, enable the
+   **Enable package scanning** option.
+ - Update the values for the **ScanCode.io URL** and **ScanCode.io API key** fields
+   located in the **Configuration** panel at the bottom of the form.
+ - Click the **Save** button.
 
 You can now access the **Scans** section from the **Tools** menu and request package
 scans from this view.
 
+.. _dejacode_dataspace_purldb:
+
 Enable PurlDB service
 =====================
 
@@ -359,47 +392,44 @@ and perform an asynchronous query of the PurlDB to find relevant data.
 
 You can:
 
-* Browse and search from a list of over **14 millions Packages**.
+* Browse and search from a list of over **21 millions Packages**.
 * Get extra information on your local Packages from the **"PurlDB" tab**.
-* Create local Packages automatically from entries found in the PurlDB.
+* **Create local Packages automatically** from entries found in the PurlDB.
 * Enhance the **Global search** results with Packages from the PurlDB.
 * Check for **new Package versions** from your Products inventory
 
-1. Get in touch with nexB to request your credentials for the **PurlDB** service.
-
-2. Set those credentials in your local DejaCode configuration file ``.env``.
-
-   See :doc:`/application-settings` for more details on the configuration system.
+PurlDB service
+^^^^^^^^^^^^^^
 
-.. code-block:: python
+A public instance of **PurlDB** is available at
+https://public.purldb.io/api/packages/
 
-    PURLDB_URL=https://purldb.url/
-    PURLDB_USER=PROVIDED_BY_NEXB
-    PURLDB_PASSWORD=PROVIDED_BY_NEXB
+Alternatively, you have the option to run your instance of PurlDB by
+following the documentation provided at https://purldb.readthedocs.io/
 
+Set the PurlDB Server URL and API key in your Dataspace Configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-**Restarting the services is required following any changes to .env:**
-
-.. code-block:: bash
-
-    docker compose restart web worker
-
-3. Enable **PurlDB access** on your Dataspace from the **Administration dashboard**.
-
-   Select **Dataspaces** and open your Dataspace definition.
-   In the **Application Process Settings** section, check the **Enable PurlDB access**
-   option and save.
+ - Access your DejaCode web application **Administration dashboard**.
+ - Navigate to the **Dataspaces** section and select your Dataspace name.
+ - Within the **Application Process Settings** section, enable the
+   **Enable PurlDB access** option.
+ - Update the values for the **PurlDB URL** and **PurlDB API key** fields
+   located in the **Configuration** panel at the bottom of the form.
+ - Click the **Save** button.
 
 You can now access the **PurlDB** section from the **Tools** menu and browse package
 from this view.
 
+.. _dejacode_dataspace_vulnerablecode:
+
 Enable VulnerableCodeDB service
 ===============================
 
 DejaCode integration with the **VulnerableCodeDB** service authorizes DejaCode to access
-the VulnerableCodeDB using a Package URL (purl) to determine if there are any reported
-vulnerabilities for a specific Package and return the Vulnerability ID and related URLs
-to a Vulnerabilities tab in the Package details user view.
+the VulnerableCodeDB using a Package URL (purl) to determine if there are any
+**reported vulnerabilities for a specific Package** and return the Vulnerability ID
+and related URLs to a **Vulnerabilities tab** in the **Package details** user view.
 
 DejaCode displays a Vulnerability icon next to the Package identifier in the user view
 list, and also in any Product Inventory list using that Package.
@@ -414,31 +444,25 @@ You can:
 * Review and edit your Product Package assignments to record your analysis, the actions
   you have taken, and the current status of your usage of that Package.
 
-1. Get in touch with nexB to request your credentials for the **VulnerableCodeDB**
-   service.
-
-2. Set those credentials in your local DejaCode configuration file ``.env``.
-
-   See :doc:`/application-settings` for more details on the configuration system.
-
-.. code-block:: python
-
-    VULNERABLECODE_URL=https://vulnerablecodedb.url/
-    VULNERABLECODE_API_KEY=PROVIDED_BY_NEXB
-
-**Restarting the services is required following any changes to .env:**
-
-.. code-block:: bash
+VulnerableCodeDB service
+^^^^^^^^^^^^^^^^^^^^^^^^
 
-    docker compose restart web worker
+A public instance of **VulnerableCodeDB** is available at
+https://public.vulnerablecode.io/api/
 
+Alternatively, you have the option to run your instance of VulnerableCodeDB by
+following the documentation provided at https://vulnerablecode.readthedocs.io/
 
-3. Enable **VulnerableCodeDB access** on your Dataspace from the
-   **Administration dashboard**.
+Set the VulnerableCodeDB Server URL and API key in your Dataspace Configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-   Select **Dataspaces** and open your Dataspace definition.
-   In the **Application Process Settings** section,
-   check the **Enable VulnerableCodeDB access** option and save.
+ - Access your DejaCode web application **Administration dashboard**.
+ - Navigate to the **Dataspaces** section and select your Dataspace name.
+ - Within the **Application Process Settings** section, enable the
+   **Enable VulnerableCodeDB access** option.
+ - Update the values for the **VulnerableCode URL** and **VulnerableCode API key**
+   fields located in the **Configuration** panel at the bottom of the form.
+ - Click the **Save** button.
 
 You can now see Vulnerabilities in the Packages user view.
 The availability of the services can be checked by clicking on your user name in the
diff --git a/docs/index.rst b/docs/index.rst
@@ -9,8 +9,8 @@ Welcome to the very start of your DejaCode journey!
     :caption: Getting Started
 
     installation
-    application-settings
     dataspace
+    application-settings
 
 .. toctree::
     :maxdepth: 1
diff --git a/docs/installation.rst b/docs/installation.rst
