feat(stt): add Azure AI Speech as an optional cloud speech-to-text provider#354
Open
amartya-dev wants to merge 1 commit into
Open
feat(stt): add Azure AI Speech as an optional cloud speech-to-text provider#354amartya-dev wants to merge 1 commit into
amartya-dev wants to merge 1 commit into
Conversation
…ovider Add an Azure AI Speech transcription tool. It is opt-in: when AZURE_SPEECH_KEY is configured the agent may prefer it for cloud STT, while the local faster-whisper `transcriber` stays the default offline path. Shared pipeline manifests are intentionally left unchanged, so no default provider selection is altered for existing users. - tools/analysis/azure_stt.py: new `azure_stt` tool (capability=analysis, provider=azure) calling the Fast Transcription REST API. The local file is uploaded via multipart and transcribed synchronously with word-level timestamps and optional diarization — no Blob storage or async polling. Output schema mirrors `transcriber` exactly, so it is a drop-in for `subtitle_gen` and other transcript consumers. Follows the existing provider-tool conventions (env-var status check, `_transcribe` helper, cost_usd/model on the result, fallback="transcriber"). - Auto-discovered by the registry; no registry or selector changes. - tests/tools/test_azure_stt.py: contract, discovery, status, response mapping, execute guardrails, and a mocked-network success path (no live API calls). - .agents/skills + .claude/skills: azure-speech-to-text Layer-3 skill. - docs/PROVIDERS.md: Azure AI Speech setup, API notes, and pricing. - .env.example, skills/INDEX.md, AGENT_GUIDE.md: document the optional cloud STT path alongside the default whisper transcriber.
4e0f807 to
a2a0d8c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Azure AI Speech as an optional cloud speech-to-text provider. It is opt-in: when
AZURE_SPEECH_KEYis configured, the agent may preferazure_sttfor cloud transcription, while the local faster-whispertranscriberremains the default offline path. Shared pipeline manifests are intentionally left unchanged, so no default provider selection is altered for existing users.The tool uses Azure's Fast Transcription REST API, which accepts a local audio file directly (multipart upload) and returns a synchronous result with word-level timestamps and optional speaker diarization — no Azure Blob storage, SAS URLs, or async job polling. Its output schema mirrors
transcriberexactly, so it is a drop-in forsubtitle_genand every other transcript consumer.Related issue
No tracking issue — small additive provider integration.
Changes
tools/analysis/azure_stt.py— newazure_stttool (capability=analysis,provider=azure,runtime=API). Calls the Fast Transcription endpoint, maps the response onto the existing transcript schema (segments+words[]+word_timestamps+language+duration_seconds). Follows the existing provider-tool conventions used byelevenlabs_tts: env-varget_status(), a_transcribe()helper wrapped for error handling,cost_usd/modelon theToolResult, andfallback="transcriber".analysiscapability has no selector).tests/tools/test_azure_stt.py— contract, registry discovery, status behavior, the response→transcript mapping,execute()guardrails, and a mocked-network success path. No live API calls..agents/skills/azure-speech-to-text/+.claude/skills/azure-speech-to-text/— Layer-3 skill referenced by the tool'sagent_skills.docs/PROVIDERS.md— Azure AI Speech setup, API notes, and pricing; env-var summary entry..env.example,skills/INDEX.md,AGENT_GUIDE.md— document the optional cloud STT path alongside the default whispertranscriber.Testing
make test→ 373 passed, 8 skipped locally (18 new tests).make test-contracts→ 259 passed, 6 skipped.make lint→ clean (py_compile).en-USauto-detected, andcost_usd/modelpopulated. Confirmed the output flows throughsubtitle_gento produce valid SRT.Checklist
make test-contracts/make test).docs/PROVIDERS.md,AGENT_GUIDE.md,skills/INDEX.md,.env.example).Notes for reviewers
azure_sttis additive and opt-in; pipelines and shipped config are untouched. Preference is expressed only when the user setsAZURE_SPEECH_KEYlocally.probability(documented in the skill).