-
Notifications
You must be signed in to change notification settings - Fork 125
Open
Labels
feature requestNew feature or requestNew feature or request
Description
Hi,
My app requires streaming since I wan to stop the generation once a certain (complicated) condition is met.
My decoding method is beam_search with beam_width=2, using greedy decoding or sampling is just not good enough accuracy-wise.
Are you planning to add support to beam_search in streaming mode?
This is very important to me.
Metadata
Metadata
Assignees
Labels
feature requestNew feature or requestNew feature or request