Complete usage documentation

[fmigneault]

Complete todo in weaver/docs (see also #101)
(use make fixme-list to find them)

• Introduction (the basics)

  • Foreword about WPS-1/2/3, XML/JSON, and Transactional/Quote/Billing additions.
  • Foreword about EMS/ADES.
  • Basic notes/pointers about informative endpoints
    • /api
    • /version
    • frontpage
  • Interfaces (ie: PyWPS-1/2 route vs WPS-REST / OGC-API)
  • Replace sphinx-redoc #732

• Type of Processes

  • What are builtin processes, and which ones are available? (Provide "builtin" apps as process building blocs #20)
  • What are WPS-1/2 processes (remote/provider processes pointer).
  • What are ESGF-CWT processes? (Esgf cwt process #23) [@matprov maybe can fill better at a later date]
  • CWL Application Package (docker app)
  • Other remote providers? ([Feature] Support other service types (remote providers) #130)
  • WPS endpoints (WPS-1/2 XML/JSON vs WPS-REST JSON).
    Maybe even bidirectional support? ([Feature] Handle Accept header for JSON/XML response in WPS 2.0 #125, [Feature] Handle WPS-1.x/2.x execute request as WPS-3 'job' #126)
  • EMS-specific execution using EOImage with AOI/TOI/CollectionId for OpenSearch

• CWL-related (Some kind of tutorial My first CWL for WPS Applcation Package)

  • How are WPS I/O mapped/merged with CWL I/O + some important differences
    • add note about arrays (most confusing since CWL are a list under the same ID but WPS repeat the ID for each individual entry of the list)
    • variants of enum/supported values/formats and to which type they apply
    • WPS Complex/Literal vs CWL File/others
  • Add details/example about typical steps: deploy app, update visibility, execute, get status
  • Add details related to CWL Application within WPS (where to put the definition - many places, variations supported - href vs json unit, ...)
  • Add details related to CWL Workflow. How are steps related to WPS processes IDs?

• Configuration (see: configuration)

  • Update docs with confval definitions #714
  • how does this run? (gunicorn/pserve/celery)
  • details about each setting in weaver.ini
    • comment about each 'group' of settings (e.g. path/url/dir of various wps config)
    • details about what is the result of any item
    • add any missing ones?
  • details about data_sources.json and wps_processes.yml
    • what do they do?
    • their respective schemas (one of them here)
    • how to provide them via weaver.ini (ie: none, single file, directory auto-listing)
    • default behavior (copy .example) + supported formats (full path vs relative vs name only looking in app/config directory)
  • Request Caching settings, invalidation, regions, etc.
    Current docs only describe generic Request Options but not specific INI settings for caching which are mostly different, but could also be combined with cache=True|False via request_extra handling.
  • Example Request Options
    • Fix broken link of Request Options in documentation #731
    • Embed example within the page
    • Provide more parameters details

• Process Execution (in https://github.com/crim-ca/weaver/blob/master/docs/source/processes.rst, see already defined sections with .. todo:: notes):

  • BoundingBox Inputs
  • Collection Inputs (relates to Support OGC-API Processes - Part 3: Collection Inputs #682)
  • Collection Outputs (relates to Support OGC-API Processes - Part 3: Collection Outputs #683)
  • Nested Processes (relates to support the Nested Processes as process: <uri> construct #747)
  • Multiple Inputs
  • Chaining Step Processes (combined with Nested Processes?)

• Quotation/Billing
  (see quote-estimator integration #531)

  • fill details of all "todo" notes
  • provide examples

• Utils and Use-Cases

  • Using provided CLI tools? (eventual CLI tools for basic CWL package definition #106)
  • How to define a notebook to be ADES/EMS compatible? (eventual Process Package generation from Jupyter Notebook definition #63)
  • Sc1 demo pavics-jupyter-images#20
  • Document reference presentation for CWL and Application Package use cases #595

• Other

  • Add extra OGC-TB15 & TB16 reference notes to README (bottom).

[fmigneault]

@dbyrns @tomLandry
FYI, list of items to document over time.
If anybody can help doing this during DACCS/OGC, that would be great (although I know most of it is knowledge that few but me are aware of).
I will try to setup the main "docs" files with todos so they are easier to fill by anyone.

[dbyrns]

Wow that's an impressive list!
Could we link some of those points to the appropriate OGC's ER?

[fmigneault]

@dbyrns
Introduction parts: could be added, if anyone feels poetic writing those ?
Type of Processes: points 2 & 3 are directly climate data services, 4 for docker app (ie: ML in this ER)
CWL-related: 1,2,3 could be useful for other participants (similar to docs made by Christophe)
Others are not that critical for OGC, more for other devs comprehension and server setup.

[tomLandry]

@fmigneault I've just finished the TOC for the pilot engineering report. See here: https://gitlab.ogc.org/ogc/eoappspilot/-/blob/master/ER_CRIM/er.adoc
It is based on recently-shared sample TOC by other participants, plus material found here. So I should be able to mesh back Weaver documentation, as well as doc found on our Confluence, back into the ER. And vice versa as applicable.