You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Is there still anyone working on dynamic embedding now, and when will it be in the official release? #2735
Dynamic Embeddings (ie. collision-less embeddings), are supported in a few different ways as a Beta / Prototype; and is area of active investment.
Curious, what specific use cases did you have in mind? most OSS uses case we see can manage today using Managed Collision Collection modules and mapping from a large raw id range down to physical memory space. [ie. whats your trainer hardware / memory budget vs number of unique sparse ids]
Curious about it too. I find there is very few introduction about MCH in torchrec, so I took a quick glance at the code.
Looks like it trades the low-freq features to save memory, treating them as default. How does it deal with the situation when it grows full?
Also I wonder, some frameworks already support models with TB level embeddings, is MCH enough in such case? How does it compare to others like HKV?
Dynamic Embeddings (ie. collision-less embeddings), are supported in a few different ways as a Beta / Prototype; and is area of active investment.
Curious, what specific use cases did you have in mind? most OSS uses case we see can manage today using Managed Collision Collection modules and mapping from a large raw id range down to physical memory space. [ie. whats your trainer hardware / memory budget vs number of unique sparse ids]
Thanks for replying. What should I do if I encounter a change in user behavior during inference and embedding needs to change?
Curious about it too. I find there is very few introduction about MCH in torchrec, so I took a quick glance at the code. Looks like it trades the low-freq features to save memory, treating them as default. How does it deal with the situation when it grows full?
Also I wonder, some frameworks already support models with TB level embeddings, is MCH enough in such case? How does it compare to others like HKV?
I agree, MCH (aka ZCH) examples could be better, thats good feedback; and the tutorials tend show our more advanced features (in part because hard to demonstrate on public datasets). Let me chat to team (its probably combination of more industry papers + examples to demonstrate).
Thanks for replying. What should I do if I encounter a change in user behavior during inference and embedding needs to change?
In practice, you learn a mapping of raw sparse id -> physical memory address in Training. As you hit on, during online training you may encounter a new id and it will trigger a insert/re-sort of remapping vector. So to propagate this change to the inference tier you would need tensor replacement; but generally most parameters need sync'ed so this isn't really any different.
Ie typical pattern: run distributed training on online data -> call online_training_model.state_dict() -> quantize relevant params/buffers (optional) -> load into predictor/inference tier.
Thanks for replying. What should I do if I encounter a change in user behavior during inference and embedding needs to change?
In practice, you learn a mapping of raw sparse id -> physical memory address in Training. As you hit on, during online training you may encounter a new id and it will trigger a insert/re-sort of remapping vector. So to propagate this change to the inference tier you would need tensor replacement; but generally most parameters need sync'ed so this isn't really any different.
Ie typical pattern: run distributed training on online data -> call online_training_model.state_dict() -> quantize relevant params/buffers (optional) -> load into predictor/inference tier.
I see, thanks for the reply, does torchrec have a tutorial document on the online training update model?
Thanks for replying. What should I do if I encounter a change in user behavior during inference and embedding needs to change?
In practice, you learn a mapping of raw sparse id -> physical memory address in Training. As you hit on, during online training you may encounter a new id and it will trigger a insert/re-sort of remapping vector. So to propagate this change to the inference tier you would need tensor replacement; but generally most parameters need sync'ed so this isn't really any different.
Ie typical pattern: run distributed training on online data -> call online_training_model.state_dict() -> quantize relevant params/buffers (optional) -> load into predictor/inference tier.
Sorry to bother, I have another question, can torchrec update the model only in full during online training, or can it replace some embedding vectors. Thank you
I agree, MCH (aka ZCH) examples could be better, thats good feedback; and the tutorials tend show our more advanced features (in part because hard to demonstrate on public datasets). Let me chat to team (its probably combination of more industry papers + examples to demonstrate).
Great, so the MCH is broadly used and satisfy your industrial requirements? It would be great to just utilize existing components in torchrec and not incorporate any new 3rd-party library.
Interesting, the dynamic embedding is powerful function for Recommendation,Search System and even in Ads system
Why we need dynamic embedding?
It's about the Recommendation,Search, Ads System, just image we have tens of millions of feed news, short videos, of course we can make a large embedding table, 100 million * 64 dims, try to fit all the content in to this embedding table, like news1: "NASA launched a rocket to Mars Today" map to index 1 in this large embedding table, but the news, short video would expired,
the news1 "NASA launched a rocket to Mars Today" would just stay in the recommend system maybe 7days or 14days, after that , the news1 expired, and news X may map to index 1 of embedding table, but it's chaos, that's why we need dynamic embedding table.
There are some sophisticated implements in tensorflow, like https://github.com/tensorflow/recommenders-addons, which helps a lot in this aspect.
Activity
dstaay-fb commentedon Feb 10, 2025
Dynamic Embeddings (ie. collision-less embeddings), are supported in a few different ways as a Beta / Prototype; and is area of active investment.
Curious, what specific use cases did you have in mind? most OSS uses case we see can manage today using Managed Collision Collection modules and mapping from a large raw id range down to physical memory space. [ie. whats your trainer hardware / memory budget vs number of unique sparse ids]
WhoisZihan commentedon Feb 11, 2025
Curious about it too. I find there is very few introduction about MCH in torchrec, so I took a quick glance at the code.
Looks like it trades the low-freq features to save memory, treating them as default. How does it deal with the situation when it grows full?
Also I wonder, some frameworks already support models with TB level embeddings, is MCH enough in such case? How does it compare to others like HKV?
freshduer commentedon Feb 11, 2025
Thanks for replying. What should I do if I encounter a change in user behavior during inference and embedding needs to change?
dstaay-fb commentedon Feb 12, 2025
I agree, MCH (aka ZCH) examples could be better, thats good feedback; and the tutorials tend show our more advanced features (in part because hard to demonstrate on public datasets). Let me chat to team (its probably combination of more industry papers + examples to demonstrate).
dstaay-fb commentedon Feb 12, 2025
In practice, you learn a mapping of raw sparse id -> physical memory address in Training. As you hit on, during online training you may encounter a new id and it will trigger a insert/re-sort of remapping vector. So to propagate this change to the inference tier you would need tensor replacement; but generally most parameters need sync'ed so this isn't really any different.
Ie typical pattern: run distributed training on online data -> call online_training_model.state_dict() -> quantize relevant params/buffers (optional) -> load into predictor/inference tier.
freshduer commentedon Feb 13, 2025
I see, thanks for the reply, does torchrec have a tutorial document on the online training update model?
freshduer commentedon Feb 14, 2025
Sorry to bother, I have another question, can torchrec update the model only in full during online training, or can it replace some embedding vectors. Thank you
WhoisZihan commentedon Feb 17, 2025
Great, so the MCH is broadly used and satisfy your industrial requirements? It would be great to just utilize existing components in torchrec and not incorporate any new 3rd-party library.
Ross-Fan commentedon Aug 21, 2025
Interesting, the dynamic embedding is powerful function for Recommendation,Search System and even in Ads system
It's about the Recommendation,Search, Ads System, just image we have tens of millions of feed news, short videos, of course we can make a large embedding table, 100 million * 64 dims, try to fit all the content in to this embedding table, like news1: "NASA launched a rocket to Mars Today" map to index 1 in this large embedding table, but the news, short video would expired,
the news1 "NASA launched a rocket to Mars Today" would just stay in the recommend system maybe 7days or 14days, after that , the news1 expired, and news X may map to index 1 of embedding table, but it's chaos, that's why we need dynamic embedding table.
There are some sophisticated implements in tensorflow, like https://github.com/tensorflow/recommenders-addons, which helps a lot in this aspect.