Reliable generative AI applications need more than a successful demo. This guide shows how to build a repeatable Databricks LLM evaluation process with representative test sets, quality and operational metrics, MLflow experiment tracking, human review, and regression testing. It also provides a practical way to estimate evaluation effort and decide when a prompt, model, or retrieval change is ready for release.
Overview
LLM evaluation is the process of measuring whether an AI application produces useful, safe, and operationally acceptable results. In a Databricks workflow, evaluation can cover a foundation model, a fine-tuned model, a retrieval-augmented generation (RAG) application, or the complete chain from user input to final response.
The most dependable approach treats evaluation as a product test suite rather than a one-time benchmark. Store representative inputs and expected characteristics in a managed dataset, run the same cases against each candidate version, record results in MLflow, and compare the new run with a known baseline. This makes prompt optimization and model selection measurable instead of dependent on anecdotal examples.
For a RAG application, separate the stages you evaluate. Retrieval can return the wrong documents even when the language model writes a fluent answer. Conversely, retrieval can be appropriate while the final response ignores the evidence. Test at least these layers:
- Input and routing: Does the application interpret the request and select the intended workflow?
- Retrieval: Are the returned chunks relevant, complete, and permitted for the user?
- Generation: Is the response correct, relevant, clear, and grounded in available context?
- Operations: Are latency, token usage, errors, and estimated cost within acceptable limits?
Evaluation results should support a release decision. A single aggregate score is rarely enough because a small improvement in helpfulness may not justify a regression in groundedness, safety, latency, or cost.
For background on building a broader test set and quality checklist, see the Databricks Model Evaluation Guide. If your application uses retrieval, pair this process with the Databricks RAG Tutorial.
How to estimate
Estimate evaluation effort and operating impact with a simple model. The exact values depend on your workload, so use your own observed measurements rather than assuming a universal benchmark.
Evaluation run effort can be approximated as:
run effort = test cases × candidate variants × evaluation passes
A pass might include automated scoring, human review, or both. If you have 200 test cases, three candidate configurations, and two passes, the run contains 1,200 case evaluations. This number helps you plan compute, reviewer capacity, and time before a release review.
For usage economics, estimate:
estimated model cost = (input tokens ÷ 1,000,000 × input rate) + (output tokens ÷ 1,000,000 × output rate)
Use the applicable provider or internal rate card for the input and output rates. If rates are unavailable or change frequently, report token volume separately and label the cost as an estimate. Add infrastructure, embedding, vector search, orchestration, and human-review costs when they materially affect the decision.
For quality, calculate each metric over the same test set:
metric result = successful cases ÷ applicable cases
Not every metric applies to every case. For example, a factuality or groundedness score requires source context, while a latency measure applies to every completed request. Keep the denominator visible so that a score cannot appear stronger merely because difficult cases were excluded.
Set release thresholds before comparing variants. A practical policy might require no meaningful regression in safety or groundedness, a minimum quality score for critical tasks, and a maximum latency or cost per request. Treat these as product decisions that should be agreed on by engineering and subject-matter owners, not as automatic defaults.
Inputs and assumptions
A useful evaluation dataset represents real usage, not only ideal prompts. Sample cases across common requests, ambiguous questions, long inputs, unsupported requests, multilingual inputs if relevant, and known failure modes. Include difficult examples deliberately; otherwise the test suite will reward a system for handling only its easiest workload.
Each record can include:
- A stable case identifier and the user request.
- Relevant retrieved context, document identifiers, or expected sources for RAG testing.
- An expected answer, answer rubric, or required facts where available.
- Task labels such as summarization, extraction, classification, or question answering.
- Risk labels for sensitive, high-impact, or adversarial cases.
- Metadata such as language, data version, prompt version, model name, and test-set version.
Choose metrics that match the task. Correctness measures whether the output satisfies the intended requirement. Relevance measures whether it addresses the request without unnecessary material. Groundedness checks whether claims are supported by the supplied context. Retrieval metrics examine whether useful evidence appears in the returned documents. Safety review looks for prohibited, harmful, or privacy-sensitive behavior according to your application requirements.
Also record operational measures: end-to-end latency, model latency, retrieval latency, error rate, input and output tokens, retries, and estimated cost. These measures often explain why a quality change occurred. For example, a larger context window may improve answer coverage while increasing latency and token usage.
Automated judges can scale review, but they should not be treated as unquestionable truth. Use deterministic checks for structured outputs, required fields, citations, or exact classifications where appropriate. Use rubric-based model evaluation for open-ended responses, then sample results for human review. Human reviewers should receive clear criteria and should be able to mark “uncertain” rather than forcing a binary judgment.
Track every run in MLflow with parameters, metrics, artifacts, and tags. Useful parameters include model identifier, system prompt version, temperature or equivalent settings, retrieval configuration, chunking version, and test-set version. Save representative inputs and outputs, failure examples, evaluator configuration, and aggregate comparisons as artifacts. This creates an audit trail for prompt changes and supports controlled model tuning.
Worked examples
Example 1: prompt change for a support assistant. A team has 120 labeled support questions and compares the current prompt with a revised prompt. Each version is run once through automated correctness, relevance, groundedness, latency, and token measurements. The evaluation effort is:
120 cases × 2 prompt versions × 1 pass = 240 case evaluations
The team then samples 30 outputs from each version for human review. The release decision should consider the full comparison: if the revised prompt improves correctness but produces unsupported claims in previously safe cases, it should not ship without further changes. Store both runs in MLflow and tag them with the same test-set version so the comparison remains reproducible.
Example 2: RAG configuration change. A knowledge assistant uses 180 cases, three retrieval settings, and two answer-generation models. The comparison contains:
180 cases × 3 retrieval settings × 2 models = 1,080 application runs
Measure retrieval relevance separately from final-answer quality. A configuration that retrieves more documents may improve recall but overwhelm the model with irrelevant context. Compare groundedness, response completeness, latency, and token volume together. Select the configuration that meets the minimum quality and operational requirements, rather than the one with the highest isolated score.
Example 3: monthly cost estimate. Suppose a test run uses 1,000 requests, an average of 2,000 input tokens, and an average of 400 output tokens. The run processes 2,000,000 input tokens and 400,000 output tokens. Apply the current provider rates to those quantities, then add any retrieval, embedding, compute, and review costs. Recalculate the estimate whenever traffic assumptions, context size, model choice, or rate-card inputs change.
When to recalculate
Re-run the relevant evaluation suite before releasing any change to the prompt, model, fine-tuning data, retrieval index, embedding model, chunking method, system instructions, output schema, safety filter, or orchestration logic. A change that appears local can alter downstream behavior, so keep a smaller smoke suite for every development iteration and a broader regression suite for release candidates.
Recalculate costs when provider pricing, token mix, traffic volume, context length, retry behavior, or infrastructure configuration changes. Recalculate quality thresholds when the business task changes, new failure modes appear, or subject-matter reviewers revise what constitutes an acceptable answer. Do not silently replace historical results: preserve the old test-set and evaluator versions, then add a new labeled version.
Review the suite on a regular operating cadence and after incidents. Add production failures, support escalations, and newly discovered adversarial cases after removing sensitive data or applying the required access controls. Use the Databricks AI Governance Checklist to connect evaluation with security and governance controls, and consult the Databricks Model Serving Guide when deployment and monitoring become part of the release process.
A practical next step is to create a versioned dataset of 50 to 200 representative cases, define task-specific pass criteria, run a baseline, and log it in MLflow. Add automated checks first, then schedule human review for ambiguous or high-risk cases. Before approving the next change, compare quality, groundedness, latency, token usage, and estimated cost against that baseline. This turns LLM regression testing into a repeatable engineering habit that remains useful as models, prompts, and pricing inputs evolve.
