AI Data Governance: What to Control First

2026-10-20 · Tommaso Maria Ricci

AI data governance is the part of an AI program that decides whether the rest of it survives. In a Gartner survey of 248 data management leaders run in the third quarter of 2024, 63% of organizations said they either did not have, or were not sure they had, the right data management practices for AI. Gartner's forecast from the same research is blunter: through 2026, organizations will abandon 60% of AI projects that are not supported by AI-ready data.

The security numbers point the same way. The IBM Cost of a Data Breach Report 2025, built with the Ponemon Institute on 600 breached organizations, found that 13% had a breach involving AI models or applications. Among those, 97% had no proper AI access controls in place.

So the question for a leadership team is not whether to govern the data that feeds AI. It is what to control first, with limited people and budget, so that the first useful AI systems can go live without creating a liability you discover a year later.

This guide answers that question. It is written for founders, COOs, CFOs and heads of data in companies that are past the experimentation stage and want AI to touch real customers, real money and real employee records.

What AI Data Governance Actually Means

Classic data governance answers four questions about data: who owns it, what it means, how good it is, and who may use it. Those questions still matter. AI adds new ones, because AI systems do three things that reports and dashboards never did.

They read data at a scale no analyst can review. A retrieval system can pull from 40,000 documents in a second. Nobody checks each one before it shapes an answer.

They create new data. Prompts, generated answers, summaries, embeddings and agent logs are all data. Most governance programs do not even list them as assets.

They act on data. An agent that can read a CRM and send an email turns a data permission into an action permission.

AI data governance is the set of decisions, rules and controls that cover those three behaviors: which data AI systems may use, for which purpose, at what quality, with which protections, and with what record of what happened.

Data governance vs AI governance

These terms get mixed up, and the confusion has a cost: each team assumes the other one owns the problem.

If you already have a data governance program, AI data governance is an extension of it, not a new bureaucracy. If you already have an AI policy, this is the part that makes the policy enforceable. I covered the policy side in the guide to AI governance for business, and the foundation in the guide on how to build a data governance framework. This article sits between the two.

Why Classic Data Governance Breaks Under AI

Most data governance programs were designed for structured data in a warehouse, consumed by humans through reports. AI breaks that model in five specific places.

1. Unstructured data becomes the main input. Contracts, emails, call transcripts, PDFs, tickets and wiki pages are where AI creates value. They are also where governance is weakest. Few companies have owners, classifications or retention rules for a shared drive.

2. Permissions get flattened. When you index documents into a search or retrieval layer, the original access rules often do not travel with them. A junior employee asks a question and gets an answer built from a board memo they could never have opened directly.

3. Purpose changes silently. Customer emails collected to answer support tickets get reused to train a sales assistant. Under privacy law, a new purpose needs its own legal basis. In practice, nobody asks.

4. Quality problems compound. A wrong value in a report misleads one reader. A wrong policy document in a retrieval index misleads every customer who asks about it, with confident wording.

5. Outputs leave no trail. If you cannot reconstruct which sources an answer used, you cannot fix it, defend it or prove compliance.

The Salesforce State of Service AI Agents Edition, a 2026 survey of 3,075 service professionals in 13 countries, shows how this plays out on the ground: 59% of service leaders call data readiness a major blocker to AI adoption, and 70% of respondents say data privacy concerns delay or limit their rollouts. The blocker is rarely the model. It is the data around it.

AI Data Governance: What to Control First

You cannot govern everything at once. These are the seven controls I put in place first, in this order, because each one prevents a failure that is expensive to undo.

1. An inventory of AI use cases and the data each one touches

Before policies, list what exists. For each AI use case, record the business owner, the data sources it reads, the data it writes, the users it serves and whether it can take actions. Include the tools people use on their own: the shadow AI problem is mostly a data governance problem.

A spreadsheet is enough for the first 90 days. What matters is that every row has a named owner.

2. A short list of approved data sources per use case

Every AI system should read from an explicit allowlist, not from "everything on the drive". This single decision prevents most of the embarrassing failures: outdated price lists, draft contracts, HR files and personal notes leaking into answers.

Start narrow. A customer support assistant needs product documentation, current policies and order data. It does not need the finance share.

3. Classification that AI systems can read

Sensitivity labels only work if the AI pipeline respects them. At minimum, mark four classes: public, internal, confidential and restricted, with restricted covering personal data of special categories, health data, payroll and anything under legal privilege. Then make the retrieval layer filter on those labels.

4. Permission-aware retrieval

The rule is simple: an AI answer must never use a document the user could not have opened. Enforce it at query time, against the source system's permissions, not with a copy that drifts. This is the control most often missing in early deployments, and it is the one IBM's 97% figure is really about.

5. A purpose register for personal data

For every use case that touches personal data, write down the purpose, the legal basis and whether the data was originally collected for that purpose. If the answer to the last question is no, legal review comes before go-live, not after.

6. Quality rules on the sources that matter

Do not try to clean all your data. Pick the sources your AI actually reads and define three or four checks for each: freshness, completeness, a single current version, and a named person who approves changes. The data quality management framework on this blog has the metrics.

7. Logging that lets you reconstruct an answer

For every AI response that matters, log the user, the time, the prompt, the sources retrieved, the output and any action taken. Set a retention period for those logs that matches your legal and audit needs, and treat them as sensitive data themselves.

If you only have budget for three of these in the first quarter, take 2, 4 and 7. They prevent leaks and let you investigate when something goes wrong.

Data Governance for AI Across the Lifecycle

Each stage of an AI system creates its own governance questions. Mapping them avoids the common pattern where the team governs training data carefully and ignores everything that happens after launch.

Most mid-sized companies do not train models. They use retrieval and agents on top of commercial models. For them, the retrieval, inference and action rows matter far more than the training row, and that is where governance effort should go first.

Embeddings and vector stores are data too

A vector database built from your documents is a copy of those documents in another form. It inherits their sensitivity. If a customer asks you to delete their data under privacy law, the embeddings built from their emails are in scope. If an employee leaves and their files are restricted, the index has to reflect that. Treat every vector store as a governed data asset, with an owner, a classification and a deletion process.

Prompts and outputs are records

When an AI assistant drafts a reply to a customer complaint, the prompt and the draft are business records. In regulated sectors they may be discoverable in litigation. Decide early which outputs are stored, for how long, and who can search them. The data retention policy guide covers the method.

Governing Agentic AI: When Data Access Becomes Action

Agentic AI changes the risk profile. A chatbot that reads a wrong document gives a wrong answer. An agent that reads a wrong document and has permission to issue refunds, update a CRM or send emails turns a data error into a business event.

Three rules keep agents governable.

Least privilege, per task. An agent gets the minimum data and tool access needed for one defined job. An agent that triages support tickets does not need write access to billing.

Separate read rights from action rights. Reading the order history and issuing a refund are two permissions. Grant the second only with limits: amounts, volumes per day, customer segments.

Human approval above thresholds. Define in writing which actions an agent completes alone and which require a person. A practical starting point: anything irreversible, anything above a monetary threshold, and anything touching a regulated decision goes to a human.

The Moffatt v. Air Canada case is the reminder that matters here. In February 2024 a Canadian tribunal held the airline responsible for wrong refund information given by its website chatbot, and rejected the argument that the chatbot was a separate entity responsible for its own statements. The award was small, about CA$812. The principle is not: what your AI says and does is your company speaking and acting.

For a broader view of how agents fit into operations, see the guide to AI agents for business.

Three Worked Examples of Data Governance for AI

Principles are easy to agree with. These three examples, typical of what mid-sized companies deploy first, show what the controls look like when applied to a real use case. They are illustrations of the method, not client stories.

Example 1: A customer support assistant

Use case. An assistant drafts replies to customer emails and answers questions on the website.

Approved sources. Current product documentation, the published return and warranty policy, the order system (read only, limited to the customer's own orders), and the last 12 months of resolved tickets with personal data removed.

Excluded. Internal pricing notes, supplier contracts, HR files, anything in draft status.

Controls that matter most. Freshness: when a policy changes, the old version leaves the index the same day. Permissions: the assistant can only see orders linked to the authenticated customer. Transparency: customers are told they are talking to an AI system. Logging: every answer stores the documents it used, so a wrong answer can be traced to its source in minutes.

Example 2: A finance close assistant

Use case. An internal assistant answers questions from the finance team about accounting policies and explains variances using ledger data.

Approved sources. The accounting manual, the chart of accounts, the general ledger for closed periods, and approved management commentary.

Excluded. Payroll detail at employee level, board materials, open-period data until it is reviewed.

Controls that matter most. Role-based access: a controller sees all entities, an analyst sees only their own. Versioning: answers cite the policy version that applied to the period. Human review: nothing the assistant produces goes into external reporting without a named reviewer. The AI in financial reporting guide covers the review step in detail.

Example 3: An HR policy assistant

Use case. Employees ask questions about leave, benefits, travel rules and internal procedures.

Approved sources. Published HR policies, the employee handbook, benefits documentation, and country-specific rules for each office.

Excluded. Individual employee records, performance reviews, disciplinary files, salary data.

Controls that matter most. Strict separation between policy content and personal records: the assistant answers "how many days of parental leave do I have under our policy" but never reads an individual file. Clear escalation to a person for anything about an individual case. Retention: conversation logs kept for a short, defined period and accessible only to the data protection lead, because employees will type personal details even when told not to.

What the three examples have in common

In each case, the most important decision was what to exclude. Every useful AI system I have seen is defined as much by the data it cannot see as by the data it can. Writing the exclusion list takes an afternoon. Discovering you needed one, after an employee sees a colleague's salary in a chatbot answer, takes much longer to repair.

Roles: Who Decides What

Governance fails when everyone is consulted and nobody decides. For AI data, I use five roles, and in companies under 500 people some of them sit with the same person.

Two rules make this work. First, the use case owner cannot approve their own use of restricted data: the data owner must sign. Second, one person, the accountable executive, can stop a deployment. If nobody has that authority, nobody will use it.

What the board should see

Boards do not need the catalog. They need four numbers each quarter: AI use cases in production, how many touch personal or restricted data, incidents and near misses, and open findings from reviews. If those four are not available, the governance program does not exist yet, whatever the policy documents say.

The AI Data Governance Self-Assessment Scorecard

Score each statement from 0 (not in place) to 2 (in place and working). Be strict: a policy nobody follows scores 0.

How to read the total.

• 0 to 8: exposed. AI is running on trust. Freeze new use cases that touch personal or confidential data until items 1, 3, 5 and 8 are fixed.
• 9 to 16: foundations in place. You can scale cautiously. Prioritize the items scored 0, starting with permissions and logging.
• 17 to 24: governed. Your constraint is now speed, not safety. Automate the controls so they do not become a queue.

If your score is under 12 and you plan to put AI in front of customers in the next two quarters, that is the moment to bring in someone who has done it before. A short advisory engagement costs far less than a rebuild, and you can request a consultation through the contact page on this site.

Regulation: What the Rules Actually Require

AI data governance is not only good practice. In Europe and in regulated US sectors, parts of it are legal requirements. Here is what applies, without the inflated deadlines that circulate online.

GDPR and privacy law

The GDPR applies to any AI processing of personal data of people in the EU. Four principles bite hardest in AI projects: purpose limitation (data collected for one purpose cannot be freely reused for another), data minimization, accuracy, and storage limitation. Article 22 restricts decisions based solely on automated processing that produce legal or similarly significant effects, and Article 35 requires a data protection impact assessment for high-risk processing.

In December 2024 the European Data Protection Board adopted Opinion 28/2024 on AI models. It covers when a model can be considered anonymous, how legitimate interest can serve as a legal basis, and what happens when a model was developed with unlawfully processed data. If you fine-tune on customer data, read it.

The EU AI Act

The EU AI Act, Regulation (EU) 2024/1689, entered into force on 1 August 2024. Its timeline was amended by Regulation (EU) 2026/1744, in force since 27 July 2026. What matters for data governance:

• Article 4, AI literacy, applies since 2 February 2025. As amended, it asks providers and deployers to take measures that support the AI literacy of their staff. It does not set a specific level of literacy for any person. Documented training on data handling is the easiest way to show you took those measures.
• Article 50, transparency, applies from 2 August 2026. People must be told when they interact with an AI system, and certain synthetic content must be marked.
• Article 10, data and data governance, applies to high-risk systems. It requires training, validation and testing data to be relevant, sufficiently representative and, to the best extent possible, free of errors and complete, with documented governance practices. The high-risk obligations apply from 2 December 2027 for Annex III systems (employment, credit, education, essential services) and from 2 August 2028 for Annex I systems embedded in regulated products.

The text of Article 10 is a useful checklist even if none of your systems is high-risk, because it describes what a defensible dataset looks like.

Frameworks that are not law but help

In the US, the NIST AI Risk Management Framework is the reference most auditors and enterprise buyers recognize. Its Generative AI Profile, published in July 2024, lists data privacy, information integrity and information security among the risks specific to generative systems. Mapping your controls to it is a cheap way to answer the security questionnaires that large customers send.

What It Costs and What It Returns

Nobody can quote you a universal price for AI data governance, and anyone who does is guessing. What I can give you is the structure of the cost and where the return comes from.

Where the money goes

1. People time. Owners and stewards spend a few hours a month each. In a mid-sized company, the first 90 days typically need one person close to half-time to drive the program.
2. Cleanup of priority sources. Usually the largest one-off cost. It is bounded if you limit it to the sources your AI actually reads.
3. Tooling. Catalog, classification, access management and logging. Many companies already own most of it inside their cloud or productivity suite and have not switched it on.
4. Legal review. Impact assessments and purpose reviews for use cases touching personal data.

Where the return comes from

The obvious return is avoided loss. IBM's 2025 report puts the global average cost of a data breach at 4.44 million dollars and the US average at 10.22 million. You do not need many avoided incidents to pay for a governance program.

The less obvious return matters more: speed. Teams with approved sources, clear owners and working permissions put new AI use cases live in weeks. Teams without them spend months in security reviews for every project, because every project reopens the same questions.

What I have seen in practice

The results I have seen from AI projects all came after the data questions were settled, not before:

• A sports distribution company grew sales by 30% with AI-driven marketing.
• A hotel moved revenue from 9 million to 10 million.
• A medical center increased its capacity by 20%.
• An agritourism business doubled its guests.

None of these started with a model choice. Each started with deciding which data the system was allowed to use, who answered for it, and what nobody could touch. The model was the easy part.

Tooling: What to Buy, and When

I will not give you a vendor list. Tool lists age in months and they push you to buy before you know what you need. Buy by capability, in this order.

The test for any purchase: does it enforce a decision you have already made? A tool bought before the decision exists becomes shelfware. The build vs buy decision guide applies here too.

Common Mistakes That Make AI Data Governance Fail

Starting with a 60-page policy. Nobody reads it, and it describes an ideal state instead of the next decision. Start with the seven controls above and write the policy around what you actually enforce.

Governing training data and ignoring retrieval. Most companies never train a model. Their risk sits in what the system retrieves at query time.

Indexing the whole drive. It is the fastest way to build a demo and the fastest way to leak a salary file. Use allowlists.

Treating it as an IT project. IT can build the controls. Only business owners can decide what data a sales assistant may use and what error rate is acceptable in a customer answer.

No owner for unstructured data. If nobody owns the policy folder, nobody updates it, and the AI will keep quoting last year's return policy.

Banning tools instead of providing approved ones. Bans push usage into personal accounts, where you see nothing. An approved tool with clear rules, set out in an AI acceptable use policy, beats a ban every time.

Measuring documents instead of outcomes. The measure is not how many policies exist. It is whether you can answer, within an hour, which data a given AI answer used.

A 30/60/90-Day AI Data Governance Roadmap

Days 1 to 30: See and stop the worst

• Appoint the accountable executive and name owners for the top five AI use cases.
• Build the inventory of AI use cases, including tools employees adopted on their own.
• Identify which use cases touch personal, confidential or restricted data.
• Pause or narrow any system that reads broad document stores without permission-aware retrieval.
• Publish a one-page rule for employees on what data may go into which AI tools.

Output: an inventory with owners, a list of high-exposure use cases, and a one-page employee rule.

Days 31 to 60: Put the controls in

• Define approved sources for each priority use case.
• Apply sensitivity labels to those sources and make the retrieval layer enforce them.
• Switch on logging of prompts, sources and outputs for production systems.
• Complete purpose and legal basis records for every use case touching personal data.
• Define quality checks and a steward for each approved source.

Output: priority use cases running on governed sources, with logs you can search.

Days 61 to 90: Make it repeatable

• Write the intake process: how a new AI use case gets approved, by whom, in how many days.
• Set agent permissions and approval thresholds in writing.
• Run a test: pick five AI answers at random and reconstruct their sources. Fix whatever fails.
• Deliver the first quarterly report to leadership with the four board numbers.
• Re-score the self-assessment and set targets for the next quarter.

Output: a governance process that approves new use cases in days instead of months.

If you want a structured way to run this roadmap with an outside view, you can request a consultation through this site. I work with leadership teams on exactly this transition, from scattered AI pilots to governed systems that scale, and the AI readiness assessment guide is a good companion to read before we talk.

FAQ

What is AI data governance?

AI data governance is the set of decisions, rules and controls that determine which data AI systems may use, for which purpose, at what quality, with which protections, and with what record of what happened. It extends traditional data governance to cover AI-specific behaviors: reading large volumes of unstructured data, creating new data such as prompts and embeddings, and acting on data through agents. Its goal is to let AI scale without leaks, unlawful processing or wrong answers at scale.

What is the difference between data governance and AI governance?

Data governance manages data assets: ownership, definitions, quality and access. AI governance manages AI systems: risk tiers, accountability, fairness and policies for use. AI data governance is the overlap. It decides whether a specific AI system may use specific data for a specific purpose, and whether the company can prove it afterwards. Most failures happen in that overlap, because the data team and the AI committee each assume the other one owns it.

In AI data governance, what should you control first?

Start with seven controls in this order: an inventory of AI use cases and the data each touches, an approved list of sources per use case, sensitivity labels the AI pipeline enforces, permission-aware retrieval, a purpose register for personal data, quality rules on the sources that matter, and logging that lets you reconstruct an answer. With limited budget, prioritize approved sources, permission-aware retrieval and logging, because they prevent leaks and make investigations possible.

Does the EU AI Act require data governance for AI?

Yes, for high-risk systems. Article 10 requires training, validation and testing data to be relevant, sufficiently representative and, as far as possible, error-free and complete, with documented governance practices. Following Regulation (EU) 2026/1744, these obligations apply from 2 December 2027 for Annex III systems and from 2 August 2028 for Annex I systems. Article 4 on AI literacy already applies and asks companies to take measures supporting staff literacy, without setting a specific level.

How do you govern data for retrieval-augmented generation?

Treat the retrieval index as a governed data asset. Feed it only from approved sources, carry sensitivity labels into the index, and filter results at query time against the user's original access rights. Set freshness rules so outdated documents are removed, keep version history, and log which chunks were retrieved for each answer. Remember that embeddings derived from personal data remain personal data for deletion and access requests.

How long does it take to set up AI data governance?

A mid-sized company can put the foundations in place in about 90 days: inventory and owners in the first month, source controls, labels and logging in the second, and a repeatable intake process in the third. Full maturity across all data takes longer, but you do not need it to start. You need governed data for the use cases you are actually running.

Who should own AI data governance in a mid-sized company?

One accountable executive, usually the COO, CFO or chief data officer, with authority to stop a deployment. Below that, each AI use case needs a business owner, each data source needs an owner and a steward, and the data protection lead covers legal basis and retention. IT builds the controls but should not decide what data a business process may use.

What This Means for Your Business

AI does not fail in production because the model was wrong. It fails because it read the wrong document, showed someone a file they should never have seen, or used customer data for a purpose nobody approved. Every one of those failures is a data governance failure, and every one is preventable with a small number of controls put in the right order.

Start with the inventory, the approved sources, permission-aware retrieval and logging. Add the purpose register and quality rules. Then make the approval of new use cases fast, because the real payoff of governance is not safety alone. It is the ability to say yes to the next AI project in days instead of months.