Reduce opensearch index update script log output

[jsjiang]

The opensearch-update command produce large amount of log outputs. On 12/15, it produced 17GB log files on ezid-stg in 7 hours. Review the script and configurations to see if we can reduce log output.

[root@uc3-ezidui-stg01 logs]# du -sh * | grep G
9.1G    ecs_json.log
4.0G    request.log
4.1G    trace.log

[jsjiang]

EZID's default settings has 3 file type log handlers. All are configured at the DEBUG level. For updating opensearch indexes, there is no need to keep keep these logs.

• get a copy of the EZID code base
• modify the settings/setting.py file to only keep the request log
• change log level from DEBUG to ERROR
• modify the opensearch_update.py script to reduce log entry output
  • change SPLIT_SIZE = 5 to SPLIT_SIZE = 100 so it create a log entry for every 100 items instead of one entry for every 5 items.

'loggers': {
            '': {
                #'handlers': ['request_log', 'trace_log', 'ecs_json'],
                'handlers': ['request_log'],
                'propagate': True,
                'level': 'ERROR',
            },