Skip to content

Conversation

GuittenyMartin
Copy link
Collaborator

@GuittenyMartin GuittenyMartin commented Sep 22, 2025

add names in diarization and summary

Add a livekit agent to send metadata on s3
Add logic to map names to the right person
Add the formatting of the summary

Martin Guitteny added 3 commits September 19, 2025 17:32
Add a visible agent that joins the room and
writes in a json all the active speaker informations.
Change the saving of the json metadata to the minio bucket_s3
to prepare the changes to map the name in the diarization
Add logic to map name to participant by aligning whisper
output and metadata, calculating jaccard similarity for
each speaker and participant
Cleaning metadata extractor to pass ruff checks
Martin Guitteny added 2 commits September 23, 2025 21:27
get the name in the conversation and convert id
to name in the metadata. Preference order:
name attribute, metadata display_name, identity
Add metadata-agent container in compose to setup metadata
agent dev
Add run agents in makefile "make run"
Add agent name in settings
Setup livekit webhook in liverkit-server to start agent
and recording simultaneously
@GuittenyMartin GuittenyMartin force-pushed the add-name branch 3 times, most recently from c83b1bd to d6cd005 Compare September 26, 2025 08:52
Add the call of metadata agent when webhook from
egress_started is received.
Add dispatch id in recording model to be able to
shutdown agent when recording end.
Still need to make the transcription start after
metadatas are uploaded on minio.
Martin Guitteny added 3 commits September 26, 2025 14:00
Change the formatting of the formatting to use the names
instead of SPEAKER_00
Update start recording test because start_room_recording
output changed.
Martin Guitteny added 3 commits September 29, 2025 10:27
Change FeatureFlag class to accept featureflag from posthog.
add metadata_agent featuer flag to check if the feature flag
is enabled for the creator of the recording.
This feature flag will be used to allow some chosen users to test
the named diarization feature.
Adding a condition on feature flag to get metadata in bucket
Fixing the format_segments function to work without metadatas

This is important because metadatas for names diarization
will not be enabled for everyone at first
Fix the error created by trying to close agent when it is
disabled.
Make the agent hidden so users do not see it.

It makes the diarization invisible for users.
Make the agent tolerant to no known owner

In the tests we do not always have an owner so
it is necessary to make the pipeline tolerant to
that special case.
@GuittenyMartin GuittenyMartin marked this pull request as ready for review October 6, 2025 09:34
Martin Guitteny added 4 commits October 10, 2025 11:26
Add the possibility of getting egress manifest by sending the
worker id in the notification.
This will allow us to get the starting time of the recording
to be able to align perfectly the metadatas and the transcription.
Copy the metadata agent from "Metadata agent #741" branch
to have the VAD agent and the right output format for metadatas
Add pandas and numpy to use dataframe to calculate
the speakers's names in diarization and improve performances.
Add livekit-pluging-silero for VAD agent in compose
(it's not supposed to be started like this and will be changed)
Martin Guitteny added 2 commits October 17, 2025 13:42
format code with ruff to pass checks
Change the algorithm to map speakers to their name to use
dataframe to improve performance.
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Change in the transcription from SPEAKER to the names of the people in the video call.

1 participant