[feature request]: Auto-Instrumentation for Microsoft Semantic Kernel in Arize Phoenix

[farzad528]

Is your feature request related to a problem? Please describe.
Currently, integrating Arize Phoenix with Microsoft Semantic Kernel requires manual OpenTelemetry (OTEL) instrumentation. While Phoenix supports OTEL, there is no built-in support for tracing Semantic Kernel interactions, making it challenging to track AI agent workflows, tool invocations, and retrieval operations efficiently. This adds complexity for developers using Semantic Kernel in production environments.

Describe the solution you'd like
I propose adding native auto-instrumentation for Microsoft Semantic Kernel in Arize Phoenix, allowing seamless tracing and evaluation of AI workflows. Specifically, this could include:

• Automatic tracing of Semantic Kernel pipelines, including planner execution, tool calls, and memory retrieval.
• Out-of-the-box support for monitoring Azure AI Search queries used within the Semantic Kernel framework.
• Prebuilt OTEL instrumentation for logging execution latency, failure rates, and quality metrics across agent workflows.
• Compatibility with Azure Agent Service to track orchestration performance in multi-agent systems.

Describe alternatives you've considered

• Manually instrumenting Semantic Kernel workflows using OTEL, which is complex and requires deep customization.
• Using generic OTEL tracing for LLM interactions but missing Semantic Kernel-specific insights (e.g., planner decisions, memory retrieval effectiveness).

Additional context