-
Notifications
You must be signed in to change notification settings - Fork 74
Add name #715
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
GuittenyMartin
wants to merge
21
commits into
main
Choose a base branch
from
add-name
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Add name #715
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a visible agent that joins the room and writes in a json all the active speaker informations.
Change the saving of the json metadata to the minio bucket_s3 to prepare the changes to map the name in the diarization
Add logic to map name to participant by aligning whisper output and metadata, calculating jaccard similarity for each speaker and participant Cleaning metadata extractor to pass ruff checks
get the name in the conversation and convert id to name in the metadata. Preference order: name attribute, metadata display_name, identity
Add metadata-agent container in compose to setup metadata agent dev Add run agents in makefile "make run" Add agent name in settings Setup livekit webhook in liverkit-server to start agent and recording simultaneously
c83b1bd
to
d6cd005
Compare
Add the call of metadata agent when webhook from egress_started is received. Add dispatch id in recording model to be able to shutdown agent when recording end. Still need to make the transcription start after metadatas are uploaded on minio.
d6cd005
to
e3267f9
Compare
Change the formatting of the formatting to use the names instead of SPEAKER_00
Update start recording test because start_room_recording output changed.
255e9ca
to
34716b6
Compare
Change FeatureFlag class to accept featureflag from posthog. add metadata_agent featuer flag to check if the feature flag is enabled for the creator of the recording. This feature flag will be used to allow some chosen users to test the named diarization feature.
Adding a condition on feature flag to get metadata in bucket Fixing the format_segments function to work without metadatas This is important because metadatas for names diarization will not be enabled for everyone at first
a948779
to
b4201e7
Compare
31aef7a
to
64269c5
Compare
Fix the error created by trying to close agent when it is disabled. Make the agent hidden so users do not see it. It makes the diarization invisible for users.
5547d2c
to
fd6f933
Compare
Make the agent tolerant to no known owner In the tests we do not always have an owner so it is necessary to make the pipeline tolerant to that special case.
88f6c49
to
a02bea4
Compare
Add the possibility of getting egress manifest by sending the worker id in the notification. This will allow us to get the starting time of the recording to be able to align perfectly the metadatas and the transcription.
Copy the metadata agent from "Metadata agent #741" branch to have the VAD agent and the right output format for metadatas
Add pandas and numpy to use dataframe to calculate the speakers's names in diarization and improve performances.
Add livekit-pluging-silero for VAD agent in compose (it's not supposed to be started like this and will be changed)
format code with ruff to pass checks
Change the algorithm to map speakers to their name to use dataframe to improve performance.
d105d1c
to
29ad484
Compare
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
add names in diarization and summary
Add a livekit agent to send metadata on s3
Add logic to map names to the right person
Add the formatting of the summary