This Databricks AI governance checklist gives business, security, data, and engineering teams a repeatable way to manage generative AI workloads. It covers Unity Catalog permissions, lineage, prompt and output handling, PII protection, model access, auditability, monitoring, and production review gates, with a practical implementation matrix and recurring audit schedule.
Overview
AI governance is the operating system around an AI application. It defines which data an application may use, who may access it, how prompts and responses are handled, how models are approved, and what happens when quality, security, or compliance conditions change.
On Databricks, governance should be designed across the full workflow rather than added after a model reaches production. A typical workflow may include source data, curated tables, retrieval indexes, prompt templates, foundation or fine-tuned models, evaluation data, model endpoints, application logs, and user feedback. Each component creates a control point.
Unity Catalog can serve as the central place to organize and govern data and AI assets, but the exact controls available depend on the workspace configuration, account setup, identity provider, networking, and enabled services. Treat this checklist as an operating framework. Confirm implementation details against your current Databricks environment and internal requirements.
Start by assigning owners. A useful minimum is a business owner for the use case, a data owner for source information, a technical owner for the application and model, and a security or privacy reviewer. Governance becomes difficult to maintain when responsibility is shared by everyone but assigned to no one.
What to track
1. Inventory and business purpose
Maintain an inventory of every production and pilot GenAI workload. Record its business purpose, users, data domains, model or endpoint, deployment environment, owner, risk classification, and approval status. Include internal assistants, retrieval-augmented generation applications, summarization workflows, classification services, and automated content generation.
Document what the system is allowed to do and what it must not do. For example, a knowledge assistant may answer questions from approved internal documents but should not make an employment, lending, medical, or legal decision without an appropriate human process. Clear boundaries make later testing and incident review more concrete.
2. Identity, permissions, and data access
Review access at the workspace, catalog, schema, table, volume, model, endpoint, and application layers that apply to your architecture. Use groups and service principals where possible, grant the minimum access needed, and separate development, test, and production permissions.
Check whether a user can retrieve information indirectly through an AI application that they could not access directly. Retrieval permissions should reflect the sensitivity of the underlying data, not merely the visibility of the application. Remove inactive identities, review privileged groups, and verify that service credentials are stored and rotated through approved mechanisms.
For broader security controls, pair this review with the Databricks Security Best Practices Checklist.
3. Lineage and data provenance
For every important response, determine whether the team can identify the source tables, documents, or records that influenced it. Track ingestion jobs, transformations, access policies, embedding or indexing steps, prompt templates, and model versions. Lineage should support both routine debugging and an investigation after an incident.
Where the application presents citations or source references, test whether they are accurate and sufficiently specific. A citation that points to a large document without identifying the relevant passage may not be useful for review.
4. Prompt, response, and PII handling
Decide what may be included in prompts and what may be retained from responses. Define rules for personal information, confidential business data, credentials, regulated records, and customer content. Apply filtering, masking, access restrictions, or exclusion before data reaches a model when appropriate.
Do not assume that a prompt is harmless because it is temporary. Prompts and outputs may appear in application logs, traces, evaluation datasets, notebooks, support tickets, or analytics tables. Create a retention schedule, restrict access to logs, and document whether content is used for troubleshooting, quality evaluation, or model improvement.
5. Model access and evaluation
Record which models are approved for which workloads, who can change the selected model, and how changes are tested. Evaluation should include task quality, factuality, retrieval relevance where applicable, refusal behavior, prompt injection resistance, sensitive-data leakage, latency, and cost or resource impact as relevant to the use case.
Use a stable test set that represents real user requests, including difficult and prohibited cases. Keep results by model, prompt version, retrieval configuration, and application release. The Databricks Model Evaluation Guide can complement this governance process with a more detailed quality-checking workflow.
6. Auditability and operational monitoring
Retain enough metadata to reconstruct a request without unnecessarily retaining sensitive content. Useful fields may include timestamp, application, authenticated identity or service, model and endpoint version, prompt-template version, retrieval configuration, outcome status, latency, and review result.
Monitor for access failures, unusual request volume, prompt injection indicators, sensitive-data exposure, declining evaluation scores, retrieval failures, endpoint errors, and unexpected changes in usage. Establish an escalation path and define who can pause a release, disable an endpoint, or investigate an incident.
Implementation matrix
Use the following matrix as a starting point. Adapt the owner and evidence columns to your internal control system.
| Control area | Minimum evidence | Suggested owner | Review cadence |
|---|---|---|---|
| Use-case inventory | Approved record with purpose, scope, risk, and owner | Business owner | Monthly |
| Unity Catalog access | Permission export, group review, and exception register | Data and security owners | Monthly |
| Data lineage | Source-to-response architecture and lineage checks | Data engineering | Quarterly or after a pipeline change |
| Prompt and output handling | Retention, masking, logging, and deletion rules | Application owner | Quarterly |
| Model approval | Evaluation results, version record, and release decision | ML or AI platform team | Before each release |
| Monitoring and incidents | Dashboard, alert history, and incident runbook | Operations and security | Monthly |
Cadence and checkpoints
A recurring schedule prevents governance from becoming a one-time project.
Before development
Confirm the business purpose, data classification, intended users, prohibited uses, owner, and initial risk level. Decide whether the workload needs retrieval, fine-tuning, external model access, or human review. Create a data-flow diagram before sensitive data is connected.
Before production
Verify permissions, secrets handling, network and endpoint controls, prompt and output retention, evaluation results, logging, monitoring, rollback steps, and incident contacts. Require a documented decision from the designated reviewer rather than relying on an informal approval in a chat or notebook comment.
Monthly
Review new and retired workloads, privileged access, failed requests, unusual usage, open incidents, policy exceptions, and changes in data sources. Sample real interactions where permitted and look for leakage, unsupported answers, or unintended use.
Quarterly
Reassess the risk classification, data flows, model and endpoint inventory, retention settings, test set coverage, access groups, and business outcomes. Re-run representative evaluations after material changes. Confirm that owners and escalation contacts are still correct.
After a material change
Trigger an out-of-cycle review when the model, prompt template, retrieval index, source data, user population, endpoint, logging design, or business purpose changes materially. A model replacement can alter behavior even when the application code remains unchanged.
How to interpret changes
Trend information is more useful when it leads to a defined action. A rise in access denials may indicate overly restrictive permissions, a changed group structure, or an application attempting to cross a data boundary. Investigate the pattern before simply granting broader access.
A decline in answer quality may come from stale source data, broken ingestion, changed chunking, a new prompt, a model update, or user questions outside the approved scope. Compare evaluation results by component so that the team does not treat every quality problem as a model problem.
An increase in sensitive-data detections should prompt a review of input sources, retrieval filters, redaction, and logs. It may reflect better detection rather than a new exposure, but it still deserves triage. Preserve evidence, limit access during investigation, and document the decision.
Repeated exceptions are a governance signal. If teams regularly bypass a control because it blocks a legitimate workflow, redesign the process with the control owner. If an exception has become normal practice, either formalize it with appropriate approval or correct the underlying architecture.
For retrieval-heavy applications, connect governance findings with technical checks from the Databricks Mosaic AI RAG Tutorial. For deployed models, use the Databricks Model Serving Guide alongside your endpoint review.
When to revisit
Revisit this checklist at least monthly for operational checks and quarterly for a fuller governance review. Return sooner when a new data domain is connected, permissions change, a model or endpoint is replaced, a prompt or retrieval strategy is redesigned, the application gains new users, an incident occurs, or internal compliance requirements change.
To make the next review efficient, keep a dated control record with the evidence examined, unresolved findings, assigned owners, due dates, and approval decisions. At the end of each review, select three concrete actions: one access or privacy improvement, one quality or evaluation improvement, and one operational improvement. Close the loop by recording whether each action reduced the observed risk.
Teams can use this article as a standing agenda for an AI governance meeting. Start with the inventory, sample a recent workload, inspect its data and model path, review open exceptions, and approve or reject the next change based on evidence. That routine turns Databricks AI governance from a static checklist into a maintainable part of enterprise AI delivery.
