Initial metadata load into the ESGF2-US wide Globus Search index

[jpnavarro]

Initial load of all Solr metadata into the Globus Search index:

1. Type=Dataset entries from LLNL
2. Type=File entries from LLNL, ORNL, and ANL
3. Only Projects identified in Identify projects to be transferred and their read-write status #30
4. Read-only projects CMIP3, CMIP5, CREATE-IP, E3SM-supplement, GeoMIP, LUCID, TAMIP should be transferred directly to the Public Index
5. Read-write projects CMIP6, CMIP6Plus, DRCDP, E3SM, input4MIPs, obs4MIPs should be transferred to their respective Staging indexes.

See ESGF2-US-1.5 Indexes.

For audit and validation purposes the transfer should track Solr source index, Solr query parameters, timestamp, Globus Search ingest results, total metadata items stored.

Recommend forking or enhancing this tool to assist in the audit and validation:

• https://github.com/esgf2-us/migration-tools

Other useful tool repositories:

• https://github.com/esgf2-us/solr2globus