Promptfoo

What it is

You write test cases for prompts, models, and LLM apps, then run them on your machine or in CI. A red-team mode generates attacks such as prompt injection and jailbreaks against your own app.

What it helps with

- Comparing prompts and models side by side on the same test cases
- Red teaming LLM apps, RAG pipelines, and agents for prompt injection, data leaks, and unsafe tool use
- Running those checks in CI so regressions show up on pull requests

Who it's for

Developers and security teams testing LLM apps and agents they own or are authorized to assess.

Worth knowing

Promptfoo is joining OpenAI, and the command-line tool stays open source under MIT.
The free Community edition runs locally, and paid plans add team features.

Do I need a paid plan to red-team?
No. The free Community edition includes red teaming for local or self-hosted use.

Does it only work with OpenAI models?
No. It tests models from many providers, including ones you host yourself.