It could be useful to give users flexibility over the user prompt used to generate the goldens in the synthesizer.
generate_goldens_from_contexts and a_generate_goldens_from_contexts may be overridden when inheriting from the Synthesizer class and rely on predefined prompt templates composed from specific arguments context, max_goldens_per_context, scenario, task, input_format
|
prompt = SynthesizerTemplate.generate_synthetic_inputs( |
|
context=context, |
|
max_goldens_per_context=max_goldens_per_context, |
|
scenario=self.styling_config.scenario, |
|
task=self.styling_config.task, |
|
input_format=self.styling_config.input_format, |
|
) |
which are passed into the synthetic inputs. Does it make sense to add an optional prompt builder argument and allow the user to construct the prompt for golden generation?
prompt_builder: Optional[Callable[..., str | Prompt]]
It could be useful to give users flexibility over the user prompt used to generate the goldens in the synthesizer.
generate_goldens_from_contextsanda_generate_goldens_from_contextsmay be overridden when inheriting from the Synthesizer class and rely on predefined prompt templates composed from specific argumentscontext, max_goldens_per_context, scenario, task, input_formatdeepeval/deepeval/synthesizer/synthesizer.py
Lines 661 to 667 in a740ca3
which are passed into the synthetic inputs. Does it make sense to add an optional prompt builder argument and allow the user to construct the prompt for golden generation?