# Eval Case Generator (20 cases)
Generate 20 evaluation test cases for an LLM system.

_Given your system prompt and use case, get 20 eval cases — 5 happy path, 5 edge, 5 adversarial, 5 known failure modes. Each case has input, expected output, and the reason it tests something._

Used in: Lesson intermediate/08 — https://vetslearn.com/lessons/intermediate/08
Source: https://vetslearn.com/resources/templates/prompts#t-eval-case-generator

---

## Context
You are an AI evaluation engineer. You build test sets for LLM systems before they ship. You know that good eval cases are specific, scoreable, and cover the four categories: happy path, edge, adversarial, and known failure modes.

Here is the system prompt being evaluated:
[SYSTEM PROMPT]

Here is the use case:
[USE CASE DESCRIPTION]

Here is what success looks like:
[SUCCESS CRITERIA]

## Task
Generate 20 evaluation cases. 5 happy path, 5 edge cases, 5 adversarial, 5 known failure modes.

Format each case as:
- **Case ID:** e.g. HAPPY-01, EDGE-01, ADV-01, FAIL-01
- **Input:** the user message the system would receive
- **Expected output:** what a passing response looks like, scoreable in one sentence
- **Why it tests something:** the specific behavior this case probes

## Standards
- Inputs should be realistic. Write them the way a real user would type them, typos and all where useful.
- Expected outputs must be scoreable. "Refuses politely" beats "handles the request well."
- Adversarial cases should attempt prompt injection, jailbreaks, or out-of-scope requests relevant to this system.
- Known failure modes should reflect known weaknesses of LLMs: hallucination, stale knowledge, format drift, refusal of valid requests.
- No two cases should test the same thing.
- If a category does not apply to this system, say so for that category and reduce the count, do not pad.

---

**Why this works**

Evals fail when they only cover the happy path. The four-category split forces coverage of the failure surfaces that ship-blocking bugs hide in. Asking for the "why it tests something" field makes the AI justify each case, which kills near-duplicates. Realistic inputs (typos and all) are how you find the bugs that synthetic clean inputs miss.

## Check before you use it

- Run all 20 cases. Cut any the system passes trivially or that cannot be scored in one read.
- Check for near-duplicates. The "why it tests something" field reduces them; it does not eliminate them.
- Add at least five cases from real usage. Synthetic-only sets miss real bugs.