7.6 KiB
order, title, description
| order | title | description |
|---|---|---|
| 8 | Lab 8 - Evaluation and Red Teaming | Probe model defenses manually and with Promptfoo to evaluate security controls. |
Lab 8 - Evaluation and Red Teaming
In this lab, we will:
- Perform prompt injection against three layers of model protection
- Use Promptfoo to programmatically evaluate a model's security protections
Explore sections focus on manually probing a model, then scaling that same thinking into repeatable evaluation workflows.
Expect this lab to move from hands-on experimentation into structured testing.
To start this lab, one web service has been preconfigured:
- Promptfoo - {{service-url:promptfoo}}
Objective 1 Explore: Direct Prompt Injection
For the first part of this lab, we are going to explore direct prompt injection. There are three levels for this challenge:
- System Prompt Instructional Guardrail
- System Prompt + Regex
- System Prompt + LLM Evaluation
Each level will be more difficult than the last, based on how the protection interacts with the generated output.
Use the embedded widget below to probe each layer. The endpoint and model are already configured. Enter your API key, pick a level, and start testing.
As you test each protection level, pay attention to how the model responds. The goal is not just to trigger unsafe output, but to understand how each layer attempts to prevent it. For Levels 2 and 3, only the final result (or safety rejection) is shown.
Objective 2 Explore: Promptfoo
While manual interaction with a model is often required for a successful jailbreak, it is often unnecessary for a quick vulnerability-scan-style red team. More often, we want confidence that a model will not respond poorly during routine user interactions. For testing a wide set of prompts against a model or application, Promptfoo is an excellent open-source framework for generating and evaluating large sets of mutated prompts.
Explore: Promptfoo red-team workflow
Promptfoo is available on our lab machine at {{service-url:promptfoo}}. We can start by creating a new red-team configuration.
Promptfoo is designed to be approachable for both beginners and practitioners. Its wizard guides you through configuring the target, selecting datasets and mutation strategies, and tracking execution.
Once we select Start, Promptfoo handles the rest. Mutations, tests, and results are all tracked by the WebUI. Promptfoo runs can take a significant amount of time, but when they finish you will be presented with a new results screen.
Promptfoo is highly flexible. Anything that involves mass evaluation of prompts against a model can be performed with the framework. Likewise, we can run an evaluation against a direct Hugging Face dataset. Once again, Promptfoo provides a WebUI, but supplying the direct YAML is often easier.
Explore: Promptfoo evaluation workflow
Often, running an evaluation against a known public benchmark provides a more quantitative way to measure the precision loss in your local configuration. This can be especially useful when you are trying to squeeze the best possible performance out of limited hardware.
Conclusion
In this lab, we performed red team evaluations against a target model:
- Direct Injections - We explored different ways to bypass common LLM controls.
- Promptfoo Red Teaming - We used Promptfoo for red teaming a large number of prompts against our target model.
- Promptfoo Evaluation - We used Promptfoo for benchmarking a model against a popular public benchmark, giving us a local point of comparison.
We should now have a better sense of what our next round of fine-tuning should be, or if we need to explore additional protections for our model!