Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Entity Analytics] Adding changes for event.ingested in riskScore and…
… assetCriticality (elastic#203975) ## Summary This pull request introduces changes to the asset criticality and risk score data clients to utilize a new ingest pipeline for adding event timestamps. The changes include the addition of utility functions for creating and retrieving the ingest pipeline, updates to the field mappings, and modifications to the data clients to integrate the new pipeline. ### Ingest Pipeline Integration: * [`x-pack/plugins/security_solution/server/lib/entity_analytics/utils/create_ingest_pipeline.ts`](diffhunk://#diff-0011b86f0b91d8a6bb1c91ea0ff59830905e90436af01f5893b14d054b4e69f5R1-R50): Added new utility functions `getIngestPipelineName` and `createIngestTimestampPipeline` to manage the ingest pipeline for adding event timestamps. ### Asset Criticality Data Client: * [`x-pack/plugins/security_solution/server/lib/entity_analytics/asset_criticality/asset_criticality_data_client.ts`](diffhunk://#diff-31b32ff8816e16c97f0d702225b9e13d7417331850c88b33435079419db94b62R26-R29): Imported the new utility functions and updated the `init` method to create the ingest timestamp pipeline. Additionally, modified the index settings to use the new ingest pipeline. ### Risk Score Data Client: * [`x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/risk_score_data_client.ts`](diffhunk://#diff-5a33102890d8bc4948e5d3d7df3901c23146bde3dee7bd15563bd1169358e43aR43-R46): Imported the new utility functions, updated the `init` method to create the ingest timestamp pipeline, and modified the index settings to use the new ingest pipeline. ### Field Mapping Updates: * [`x-pack/plugins/security_solution/server/lib/entity_analytics/asset_criticality/constants.ts`](diffhunk://#diff-d0e75953a3b6d040a296cb4cd7513428a18b152808231819f28d7329dc86a92cL20-R20): Added the field mapping `event.ingested` for asset criticality. * [`x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/configurations.ts`](diffhunk://#diff-43b70e77669c1f7c9608f8d26095db18f6fa0380beeb5990701656ae920602d7L102-R102): Added the field mapping `event.ingested` for risk score. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Testing steps : - Checkout main branch - Setup and start kibana - Enable Risk Engine - Execute below query, result should not have event.ingested ``` GET /*asset-criticality.asset-criticality-*/_mapping GET /*risk-score.risk-score-latest-*/_mapping ``` - Add data using document generator - Execute below query ``` GET /*asset-criticality.asset-criticality-*/_search { "_source": ["event.ingested", "@timestamp"], "query": { "exists": { "field": "event.ingested" } } } ``` ### Expected output ``` { "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 0, "relation": "eq" }, "max_score": null, "hits": [] } } ``` - Same output as above for below query too ``` GET /*risk-score.risk-score-latest-*/_search { "_source": ["event.ingested", "@timestamp"], "query": { "exists": { "field": "event.ingested" } } } ``` - The below query should give results but `event.ingested` should not be present in the results ``` GET /*asset-criticality.asset-criticality-*/_search { "_source": ["@timestamp", "event.ingested"] } GET /*risk-score.risk-score-latest-*/_search { "_source": ["@timestamp", "event.ingested"] } ``` ### Expected output ``` { "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 3, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": ".asset-criticality.asset-criticality-default", "_id": "user.name:user-001", "_score": 1, "_source": { "@timestamp": "2025-01-09T14:20:24.221Z" } }, { "_index": ".asset-criticality.asset-criticality-default", "_id": "user.name:user-002", "_score": 1, "_source": { "@timestamp": "2025-01-09T14:20:24.221Z" } }, { "_index": ".asset-criticality.asset-criticality-default", "_id": "host.name:host-001", "_score": 1, "_source": { "@timestamp": "2025-01-09T14:20:24.222Z" } } ] } } ``` ### - Checkout this PR and restart Kibana (Try running the Risk Score engine using the Run Engine option if you have added data after enabling the Risk Engine) All the above queries should contain data/results with `event.ingested` as below : ``` { "took": 1, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 11, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": "risk-score.risk-score-latest-default", "_id": "X19B5MlF3Loy86u-U-mC6BrCwAAAAAAA", "_score": 1, "_source": { "event": { "ingested": "2025-01-10T07:51:30.757784Z" }, "@timestamp": "2025-01-10T07:51:30.363Z" } }, { "_index": "risk-score.risk-score-latest-default", "_id": "X19DYvlD0CQ6h1VE9n-ScWnjqwAAAAAA", "_score": 1, "_source": { "event": { "ingested": "2025-01-10T07:51:30.757971Z" }, "@timestamp": "2025-01-10T07:51:30.363Z" } }, { "_index": "risk-score.risk-score-latest-default", "_id": "X19DQLgfYH-Zr4z01uVnAImoTgAAAAAA", "_score": 1, "_source": { "event": { "ingested": "2025-01-10T07:51:30.758039Z" }, "@timestamp": "2025-01-10T07:51:30.363Z" } }, { "_index": "risk-score.risk-score-latest-default", "_id": "X19IqrXmM5aDk2qno3rUL5TI3gAAAAAA", "_score": 1, "_source": { "event": { "ingested": "2025-01-10T07:51:30.758108Z" }, "@timestamp": "2025-01-10T07:51:30.363Z" } }, { "_index": "risk-score.risk-score-latest-default", "_id": "X19K9okuf9lAZcd2Y7t-QFWJAQAAAAAA", "_score": 1, "_source": { "event": { "ingested": "2025-01-10T07:51:30.758163Z" }, "@timestamp": "2025-01-10T07:51:30.363Z" } }, { "_index": "risk-score.risk-score-latest-default", "_id": "X19K95CQyZSvT-ZQVwx_6jJTzgAAAAAA", "_score": 1, "_source": { "event": { "ingested": "2025-01-10T07:51:30.758222Z" }, "@timestamp": "2025-01-10T07:51:30.363Z" } }, { "_index": "risk-score.risk-score-latest-default", "_id": "X19LMkPHJ-L99JamiiYkt9WB1wAAAAAA", "_score": 1, "_source": { "event": { "ingested": "2025-01-10T07:51:30.758272Z" }, "@timestamp": "2025-01-10T07:51:30.363Z" } }, { "_index": "risk-score.risk-score-latest-default", "_id": "X19M4c0tojXVhK5aOwVA46RNVgAAAAAA", "_score": 1, "_source": { "event": { "ingested": "2025-01-10T07:51:30.758462Z" }, "@timestamp": "2025-01-10T07:51:30.363Z" } }, { "_index": "risk-score.risk-score-latest-default", "_id": "X19M7j9nZmY4g5bEDPJc20zNHgAAAAAA", "_score": 1, "_source": { "event": { "ingested": "2025-01-10T07:51:30.758573Z" }, "@timestamp": "2025-01-10T07:51:30.363Z" } }, { "_index": "risk-score.risk-score-latest-default", "_id": "X19TVbTGATHGj2iG_rFIUx2_1QAAAAAA", "_score": 1, "_source": { "event": { "ingested": "2025-01-10T07:51:30.758629Z" }, "@timestamp": "2025-01-10T07:51:30.363Z" } } ] } } ``` ``` { "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 3, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": ".asset-criticality.asset-criticality-default", "_id": "user.name:user-001", "_score": 1, "_source": { "@timestamp": "2025-01-10T07:50:19.522Z", "event": { "ingested": "2025-01-10T07:50:19.532122Z" } } }, { "_index": ".asset-criticality.asset-criticality-default", "_id": "user.name:user-002", "_score": 1, "_source": { "@timestamp": "2025-01-10T07:50:19.523Z", "event": { "ingested": "2025-01-10T07:50:19.535465Z" } } }, { "_index": ".asset-criticality.asset-criticality-default", "_id": "host.name:host-001", "_score": 1, "_source": { "@timestamp": "2025-01-10T07:50:19.523Z", "event": { "ingested": "2025-01-10T07:50:19.535536Z" } } } ] } } ``` The ingest pipeline should also be visible as below ``` GET /_ingest/pipeline/entity_analytics_create_eventIngest_from_timestamp-pipeline* ``` ![image](https://github.com/user-attachments/assets/42d4167b-575c-43ea-9219-34b31ded12fb) --------- Co-authored-by: kibanamachine <[email protected]>
- Loading branch information