Design for ESGF usage data collection

[lliming]

We need a design document that covers how we will gather and report usage metrics for data in the ESGF-1.5 system. The usage metrics must include:

• Globus transfers from ESGF guest collections at ORNL, LLNL, ANL, and NERSC (if applicable)
• HTTPS accesses from ESGF guest collections at ORNL, LLNL, ANL, and NERSC (if applicable)
• Use of the ESGF-1.5 catalog (either the transition API or the underlying Globus Search index)

[jpnavarro]

Designs and Plans

Globus transfers from ESGF guest collections on US GCS data nodes:

• Design: collection admins can retrieve transfer logs from an existing Globus API
• Plan: collecting and doing something with these logs will not be part of the ESGF-1.5 transition, but could be prioritized separately in the future

HTTPS accesses from ESGF guest collections on US GCS data nodes:

• Design: In GCS >= v5.4.82 a new globus_access_log feature help isolate and access Globus HTTPS file accesses
• Plan: collecting and doing something with these logs will not be part of the ESGF-1.5 transition, but could be prioritized separately in the future

**Use of the ESGF-1.5 catalog (either the transition API or the underlying Globus Search index)

• Design: Globus Search doesn't provide access to query logs
• Design: The transition API could be instrumented to log queries
• Plans: instrumenting the transition API and then doing something with generated logs will not be part of the ESGF-1.5 transition, but could be prioritized separately in the future