Chore: AssemblyAI add dynamic keyterms + return language code + normalize preemptive generation validation #3802
Conversation
|
|
||
| # Normalize text by lowercasing and removing punctuation for comparison | ||
| def normalize_text(text: str) -> str: | ||
| return text.lower().translate(str.maketrans("", "", string.punctuation)) |
There was a problem hiding this comment.
should we just use text.strip() to remove start and end spaces, different punctuation like ? or ! may have different TTS results?
There was a problem hiding this comment.
@longcw in my testing, the same input to an LLM but changing between blank, ?, ! has no effect on the LLM response
There was a problem hiding this comment.
IMO we shouldn't normalize the punctuations in the agents framework, maybe you can add it to the specific (validated) STT to reuse the latest transcript sent to the preemptive generation as the final version if the normalized text is the same.
There was a problem hiding this comment.
Done, thanks @longcw!
Let me know if there's anything else needed for this PR or all good to go
|
Hi @longcw @theomonnom may we have this updated PR reviewed? The type checks and tests are failing due to OpenAI plugin |
2 participants
Hi team!
Added dynamic keyterms updating without closing the stream, sends a message with type "UpdateConfiguration" https://www.assemblyai.com/docs/api-reference/streaming-api/streaming-api#send.sendUpdateConfiguration
Language code is returned when "Language_detection" is True (set now by default), and we pass this in the preflight and final transcripts, its only returned with utterances so we store it when we get it
https://www.assemblyai.com/docs/api-reference/streaming-api/streaming-api#receive.receiveTurn.language_code
Also normalized the preemptive generation text in agent_activity before validation, so we only compare if there are new words and not any changes in casing or punctuation.