Enable javascript in your browser for better experience. Need to know to enable it? Go here.
Last updated : Nov 04, 2026
Nov 2026
Trial ?

We put Promptfoo in Assess in 2023 for testing prompts. We still use it for evals, including agent tool calls and CI gates, but now also for red teaming. We recommend trialing it for both. To red-team an app, you describe what it's for, and Promptfoo generates attacks for the risks you choose, applies attack strategies and grades the responses.

We used Promptfoo to red-team a production conversational assistant with domain-specific safety rules. It turned ad hoc manual probing into repeatable tests rerun after every fix. It also works with stateful guardrails: the guardrails are the runtime control, and Promptfoo tests that they hold. Promptfoo is now owned by OpenAI, although it remains MIT-licensed and is model agnostic. Some of Promptfoo's more advanced simulated attacks rely on its hosted service, so make sure you're comfortable with what data leaves your organization.

Sep 2023
Assess ?

promptfoo 是一款测试驱动的 prompt engineering。在应用程序中集成 LLM 时,调整提示词为生成最佳回答并保证输出的一致性,往往会耗费大量时间。你可以将 promptfoo 作为 CLI 和库使用,根据预定义的测试用例对提示词进行系统测试。测试用例和结果断言则可通过简单的 YAML 配置文件完成设置。 这个配置文件包含需要测试的提示词、模型提供者、断言以及将会在提示词中被替换的变量值。promptfoo 支持多种断言,包括相等性、JSON 结构、相似性、自定义函数检查,甚至支持使用 LLM 对模型输出结果分级。如果你想对提示词和模型质量进行自动化反馈,请务必体验 promptfoo。

Published : Sep 27, 2023

Download the PDF

 

 

 

English | Português 

Sign up for the Technology Radar newsletter

 

 

Subscribe now

Visit our archive to read previous volumes