Open-source CLI for testing, comparing and red-teaming LLM prompts
Promptfoo is an open-source command-line tool for testing prompts and models before they reach users. You write a configuration file listing prompts, providers and test cases, then run it to see how each combination performs against assertions, model-graded rubrics and similarity checks. Because it lives in the repository, evaluations run in CI and a prompt change that regresses quality can fail the build the same way a broken unit test would. It also scans for jailbreaks, prompt injection and data leakage, and reports the results as readable tables rather than opaque scores.
Last updated: 2026-09-20. This site only provides an index; for exact features, pricing, and licensing, see the official website.
prompt regression testing, model comparison, llm red-teaming and ci checks
If you're comparing similar products, check the alternatives below, or browse all tools in the AI Prompt Tools category.
No. It is a local command-line tool and a configuration file. An optional hosted dashboard exists for sharing results with a team, but the evaluation itself runs on your machine or in your pipeline.
Anything you can express as a check: exact or partial string matches, valid JSON, a numeric threshold, a similarity score, or a rubric graded by another model. Custom JavaScript and Python assertions cover the rest.
It uses the same setup. You point a scan at your prompts and providers and it probes for jailbreaks, prompt injection and leakage, so safety checks live next to ordinary quality checks.
Community library of shared prompts with a chat interface to run them
Open-source tracing, evaluation and prompt management for LLM applications
Free educational resource teaching prompt engineering from basics to research topics
Searchable gallery of AI image prompts shown alongside the images they made