[checkpoint_harmony_endpoint] Fix cursor data not updating

[chrisberkhout]

Proposed commit message

[checkpoint_harmony_endpoint] Fix cursor data not updating

When a single event is returned in order to indicate an error that should be
logged and update fleet health status, cursor data updates will be ignored[1].

Without successful cursor updates, this integration was never able to clear an
expired task ID (returning 404) from the cursor data.

The fix is to always return an array of events when cursor data needs to be
updated. The error event will continue to be ingested. It will not be
logged or update the fleet status.

All `cel.yml.hbs` files in this integration are identical.

[1]: https://github.com/elastic/beats/blob/11da96ab9ad2c3ca110ed99d16bbab0b89ddf582/x-pack/filebeat/input/cel/input.go#L492-L493

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: ba32ae0

cc @chrisberkhout

[ShourieG]

LGTM, only one question, should we ingest or drop the error events if we are just recovering ?

[chrisberkhout]

LGTM, only one question, should we ingest or drop the error events if we are just recovering ?

Recovery isn't guaranteed. If there was logic here for having limited retries and then skipping a time range if there are persistent errors there, then I'd probably only want to keep the final failure event. For now there's no change in what will be ingested, it'll just actually have a chance to recover.

Regarding how to do the failure events, the only thing I see in Fleet Package Code Review Comments is Use a terminate processor with inputs that produce failure events (but this integration allows ES before the terminate processor). Perhaps we could expand on that.

[elastic-vault-github-plugin-prod]

Package checkpoint_harmony_endpoint - 1.0.1 containing this change is available at https://epr.elastic.co/package/checkpoint_harmony_endpoint/1.0.1/