Databricks Model Evaluation Guide: Metrics, Test Sets, and LLM Quality Checks

Overview

Model evaluation is the process of measuring whether an AI system is useful, accurate, safe, consistent, and efficient for a defined task. In Databricks, that system may include a base model, a fine-tuned model, a prompt template, a retrieval layer, business rules, and a serving endpoint. Evaluating only the model can therefore miss failures introduced elsewhere in the application.

Start by writing an evaluation contract before choosing metrics. Define the task, the expected output, acceptable failure modes, and the users who will rely on the result. For example, a customer-support assistant may need grounded answers with citations, while a classification workflow may prioritize label accuracy, predictable formatting, and low latency.

Keep evaluation data separate from training or prompt-development data. A practical test set should include common examples, difficult cases, ambiguous requests, empty or malformed inputs, long inputs, and examples that represent important business segments. Store each test case with an identifier, input, expected behavior, reference answer or label when available, metadata, and the version of the prompt or model used.

In a Databricks workflow, log evaluation inputs and outputs in a durable table or experiment run rather than relying on notebook output. Track the model identifier, prompt version, retrieval configuration, decoding settings, evaluation timestamp, and code version. This makes it possible to compare runs and identify whether a quality change came from a model update, new documents, altered chunking, or a prompt edit.

For implementation context, pair this process with a Databricks Mosaic AI RAG tutorial when evaluating a knowledge assistant, and review the Databricks Model Serving guide when moving a tested model into a serving workflow.

Checklist by scenario

Foundation model or prompt evaluation

• Define the task and output schema in plain language.
• Create representative examples before optimizing the prompt.
• Include adversarial, ambiguous, and out-of-scope requests.
• Check instruction following, factuality, relevance, completeness, and format compliance.
• Measure response length, latency, and approximate usage or infrastructure cost where those affect the application.
• Review a sample of outputs manually to validate that automated scores reflect user expectations.

For structured outputs, validate the response programmatically. A response that sounds correct but fails to produce valid JSON, required fields, or an allowed category should not pass the same way as a correctly formatted result.

Fine-tuned model evaluation

• Separate training, validation, and held-out test examples.
• Check that the test set covers the same label distribution and input conditions expected in production.
• Compare the fine-tuned model with the original model using the same prompts and test cases.
• Measure task-specific performance, such as precision, recall, F1 score, exact match, or error rate when labels or references exist.
• Inspect examples where the fine-tuned model regresses, not only its average score.
• Test behavior on inputs outside the fine-tuning data to identify memorization or narrow specialization.

A higher aggregate score is not automatically an improvement. A model may improve on frequent examples while becoming less reliable for rare but important cases. Segment results by language, product area, customer type, document category, or other operational dimensions.

RAG application evaluation

• Test retrieval separately from generation.
• Verify that relevant documents appear in the retrieved context.
• Measure context precision: whether retrieved passages are useful rather than merely similar.
• Measure context recall: whether the information needed to answer is present.
• Check groundedness or faithfulness: whether the answer is supported by the supplied context.
• Check answer relevance, completeness, citation accuracy, and refusal behavior when evidence is missing.
• Include stale, conflicting, duplicated, and access-restricted documents in the test set.

RAG evaluation metrics should be interpreted together. Strong retrieval does not guarantee a good answer, and a fluent answer does not prove that retrieval worked. If the application uses Databricks Vector Search, document the index, embedding configuration, chunking approach, filters, and retrieval settings so that a later comparison is meaningful. The Databricks Vector Search guide provides related implementation context.

Human review workflow

• Write reviewer instructions with examples of passing, borderline, and failing outputs.
• Use at least two reviewers for a selected sample when the decision is consequential.
• Randomize or blind model labels where practical to reduce preference bias.
• Capture a categorical rating and a short reason for each failure.
• Adjudicate disagreements and update the rubric when reviewers interpret criteria differently.

What to double-check

Before accepting an evaluation result, confirm that the test set matches real traffic. Synthetic examples can help expand coverage, but they should not replace observed inputs or carefully reviewed edge cases. Also check for duplicate or near-duplicate examples across development and test data; leakage can make a model appear more capable than it is.

Review the metric definition and direction. Accuracy, precision, recall, and F1 answer different questions. For generative tasks, reference-based similarity may be useful in some workflows but insufficient when multiple answers are valid. Combine automated metrics with rubric-based review for qualities such as helpfulness, tone, groundedness, and adherence to policy or business rules.

Double-check the evaluation environment. Keep model parameters, system instructions, prompt templates, retrieval settings, tool definitions, and preprocessing logic fixed when comparing two candidates. If multiple variables change at once, record the result as a new system version rather than attributing the difference to one change.

Finally, inspect operational quality. Track latency by percentile rather than only an average, identify timeout and error rates, and verify that logging does not expose sensitive input or output unnecessarily. Evaluation should cover the complete path from request to response, not just a batch inference function.

Common mistakes

• Using one score as the decision rule. A single average can hide severe failures in a small but important segment.
• Evaluating only successful requests. Include empty inputs, tool errors, retrieval misses, timeouts, and malformed outputs.
• Changing the test set after every prompt edit. Preserve a stable regression set and maintain a separate development set for experimentation.
• Confusing fluent writing with factual correctness. Require evidence, references, or reviewer verification where accuracy matters.
• Testing retrieval and generation only together. Isolate retrieval recall and context quality so the source of a failure is diagnosable.
• Ignoring regressions after deployment. Monitor representative production samples and feed reviewed failures back into the evaluation set.
• Failing to version evaluation artifacts. A score is difficult to interpret without the exact data, prompt, model, and configuration that produced it.

When to revisit

Run the full evaluation before a production launch, after changing the model or prompt, and whenever the retrieval index, embedding model, chunking strategy, filters, tools, or output schema changes. Repeat it before seasonal planning cycles if traffic, catalog content, policies, or customer behavior are likely to differ from the previous period.

Use a smaller smoke-test suite for frequent development changes and reserve the complete test set for release candidates. Revisit the rubric when reviewers identify a new failure mode, users change their expectations, or the application expands to a new language, region, document type, or business process.

A practical final checklist is:

1. Confirm the task definition and pass criteria.
2. Refresh representative and edge-case examples.
3. Run automated metrics and segmented analysis.
4. Review a meaningful sample of outputs by hand.
5. Compare against the last approved version.
6. Investigate regressions before release.
7. Log the result, configuration, decision, and follow-up actions.

For ongoing operations, connect evaluation results with your job and serving workflows so that changes produce traceable runs. Use the Databricks Jobs guide for scheduling and monitoring considerations. Repeating this checklist whenever inputs or tools change turns model evaluation from a one-time approval step into a durable quality practice.