context engineering - ingest documentation, architecture diagrams, source code, service tickets to improve AIOps performance

[aspanner]

Context engineering - ingest documentation, architecture diagrams, source code, service tickets to improve AIOps performance.

[aspanner]

Construct comparison runs
0) Setup:
-->OTEL demo app with real faults (check https://github.com/lfedgeai/AIOps/tree/main/research/agentic_aiops_architectures for code examples eg run_harness.py)
--> MLFlow for experiment tracking in order to compare performance (accuracy & MTTD/MTTR):
--> define prompt

1. LLM (raw)
2. LLM + system prompts + RAG (documentation, architecture digarma, source code, service tickets with resolution)

Output: A table containing different runs against the same system-under-test (=k8s cluster with an application, which a fault is being inject; the fault injection time is recorded in order to calculate MTTD/MTTR)
The table shows a comparison of MTTD/MTTR timing based on the same fault injected but with different contexts.

For example:

• Context A includes documentation
• Context B included documentation & source code
• Context C includes only sourc code
• Context D includes only architecture diagrams
  --> run all permutation of different context information sources