Evaluation & Testing

You can't improve what you can't measure. Evaluation and testing are the foundation of reliable AI systems — they tell you whether your model is good enough to ship, whether a change improved or degraded quality, and whether your system is drifting over time. This module covers industry-standard benchmarks, building custom evaluation suites, human evaluation frameworks, A/B testing, regression testing for prompts, and the tools that make eval-driven development practical.

Why AI Evaluation Is Hard

Traditional software has deterministic outputs: given the same input, you get the same output, and you can check it with an assertion. LLMs are stochastic — the same prompt can produce different (but equally valid) responses every time. This means evaluation shifts from "is the output exactly right?" to "is the output good enough along multiple dimensions?"

Evaluation dimensions for AI systems include correctness, helpfulness, safety, format compliance, latency, cost, and consistency. A change that improves correctness might degrade safety. A prompt tweak that fixes one edge case might break five others. Systematic evaluation is the only way to navigate these trade-offs confidently.

Industry Benchmarks for LLMs

Benchmarks provide standardized tests that allow comparison across models and over time. While no single benchmark captures real-world performance, together they paint a useful picture of model capabilities.

Building Custom Eval Suites

Custom evals are tests designed specifically for your application's use case. They are the single most important tool for making confident changes to your AI system.

Anatomy of an Eval

Every eval has three components: an input (the test case), an expected behavior (what "good" looks like), and a scoring function (how to measure success).

Types of Scoring Functions

• Exact match: The output must match exactly. Good for classifications, structured outputs, and extracted fields (dates, IDs, amounts).
• Contains/regex: Check whether the output contains specific strings or matches a pattern. Good for verifying citations, required disclaimers, or format compliance.
• Semantic similarity: Compare the output embedding to a reference answer embedding. Good for open-ended responses where wording can vary but meaning should be similar.
• LLM-as-judge: Use a powerful model to evaluate the output against a rubric. The most flexible approach and essential for subjective qualities like helpfulness, tone, and completeness.
• Code execution: For code generation tasks, run the generated code against test cases and check whether it produces correct outputs.

Building Your Eval Dataset

• Start with 50–100 representative cases: Cover the main paths your application handles. Include happy paths, edge cases, and known failure modes.
• Mine production logs: Real user queries are the best source of eval cases. Flag interactions where users gave negative feedback, retried their query, or escalated to a human.
• Bug-driven additions: Every bug report becomes an eval case. This ensures the same failure never ships twice.
• Adversarial cases: Include prompt injection attempts, off-topic queries, ambiguous inputs, and multi-language inputs to test robustness.
• Stratify by category: Ensure balanced representation across different query types, difficulty levels, and user segments. A suite dominated by easy cases will give inflated scores.

Human Evaluation Frameworks

Automated evals catch most issues, but human evaluation remains the gold standard for subjective quality — especially for tone, helpfulness, and naturalness. The challenge is making human eval scalable and consistent.

Designing Human Eval Protocols

• Clear rubrics: Define exactly what each score means. A 5-point scale with vague labels like "good" and "excellent" produces inconsistent ratings. Use behavioral anchors: "Score 5: Answer is factually correct, directly addresses the question, cites relevant sources, and is easy to understand."
• Blind evaluation: Evaluators should not know which model or prompt version generated the output. This eliminates bias toward the known "current" version.
• Inter-rater reliability: Have multiple evaluators rate the same samples and measure agreement (Cohen's Kappa or Krippendorff's Alpha). If agreement is low, your rubric needs refinement.
• Side-by-side comparison: Instead of absolute ratings, show evaluators two outputs and ask which is better. Humans are more consistent at relative comparison than absolute scoring. This is the approach behind LMSYS Chatbot Arena.

A/B Testing AI Features

A/B testing for AI features follows the same principles as traditional product A/B tests, with additional considerations for the stochastic nature of AI outputs.

What to Measure

A/B Testing Pitfalls for AI

• Sample size: Because AI outputs are variable, you need larger sample sizes than typical product A/B tests to detect meaningful differences. Plan for at least 1,000–5,000 interactions per variant.
• Confounding by topic: If variant A gets easy queries and variant B gets hard ones (by chance), the comparison is invalid. Ensure random assignment is truly random and stratified by query type when possible.
• Measuring long-term effects: A prompt that produces slightly lower satisfaction per-interaction might lead to higher overall task completion. Run tests long enough to capture downstream effects.

Regression Testing for Prompts

Prompt drift is one of the most insidious problems in AI applications. A small prompt tweak to fix one issue silently breaks five other behaviors. Regression testing catches this by running your full eval suite on every prompt change.

The Prompt Regression Workflow

Eval Tooling

LangSmith

LangSmith, built by LangChain, is a platform for tracing, evaluating, and monitoring LLM applications. On the evaluation side, it provides:

• Dataset management: Create and version eval datasets with inputs, expected outputs, and metadata. Import from CSV, JSON, or production traces.
• Custom evaluators: Define scoring functions in Python — from simple string matching to LLM-as-judge chains. Evaluators run automatically against datasets.
• Experiment tracking: Compare multiple prompt versions side-by-side with full scoring breakdowns. See exactly which cases improved and which regressed.
• CI integration: Run evals in your CI pipeline and gate deployments on eval scores.

Braintrust

Braintrust is purpose-built for AI evaluation with a focus on developer experience and speed. Key features include:

• Fast iteration: Evaluations run in parallel with real-time streaming results. See scores updating live as each case completes instead of waiting for the full suite.
• Built-in scorers: Pre-built scoring functions for common patterns — factuality, relevance, summarization quality, SQL correctness, and more.
• Diff view: Side-by-side comparison of outputs between experiments, highlighting exactly what changed and how scores shifted.
• GitHub integration: Automatically posts eval results as comments on pull requests, making prompt changes reviewable by the team.
• Online scoring: Run evaluations against production traffic in real-time to catch quality issues as they happen.

Eval-Driven Development Workflow

The best AI engineering teams practice eval-driven development — the AI equivalent of test-driven development (TDD). The principle is simple: write the eval first, then make it pass.

Resources