confident-ai/deepeval

Feat request: optional user-defined prompt builder for synthesizer

オープン

#2,396 opened on 2025/12/26

 (15 件のコメント) (0 件のリアクション) (0 人の担当者)Python (1,677 件のフォーク)auto 404
enhancementhelp wanted

Repository metrics

Stars
 (16,939 個のスター)
PR merge metrics
 (平均マージ 4d 16h) (30d で 26 merged PRs)

説明

It could be useful to give users flexibility over the user prompt used to generate the goldens in the synthesizer.

generate_goldens_from_contexts and a_generate_goldens_from_contexts may be overridden when inheriting from the Synthesizer class and rely on predefined prompt templates composed from specific arguments context, max_goldens_per_context, scenario, task, input_format

https://github.com/confident-ai/deepeval/blob/a740ca31d6c32d130e93fc1fe0c90d4795a93dc0/deepeval/synthesizer/synthesizer.py#L661-L667

which are passed into the synthetic inputs. Does it make sense to add an optional prompt builder argument and allow the user to construct the prompt for golden generation?

prompt_builder: Optional[Callable[..., str | Prompt]]

コントリビューターガイド