AI Operating Model: How to Structure Teams and Ownership

Only 25% of AI initiatives have delivered the return their sponsors expected, and only 16% have scaled across the enterprise. Those are the numbers 2,000 CEOs gave IBM in its 2025 CEO Study. In the same survey, half of them admitted that the pace of investment had left their company with disconnected technology. The models work. What is missing in most companies is an AI operating model: an explicit answer to who owns AI, who pays for it, who is allowed to say no, and who runs a system once the pilot team has moved on.

I have built and run companies for more than 20 years, and I now advise leadership teams on AI adoption. The pattern repeats with uncomfortable regularity. A company has a strategy deck, a handful of pilots, a tool budget and no clear answer on which executive is accountable for the result. This guide covers the structures that work, how to choose between them, what an AI center of excellence should and should not do, and a 90-day plan to put the model in place.

What an AI operating model is, and what it is not

An AI operating model describes how a company organizes itself to select, build, buy, deploy and run AI. It is the layer between strategy, which says where AI should create value, and execution, which is the daily work of teams. A strategy without an operating model is a list of intentions. An operating model without a strategy is an organization chart looking for a purpose.

In practice the model is the answer to six questions:

1. Ownership. Which executive is accountable for AI results, and which business leader owns each use case?
2. Structure. Where do AI skills sit: in a central team, inside business units, or split between the two?
3. Funding. Who pays for platforms, for pilots and for systems in production, and how are initiatives compared with each other?
4. Talent. Which roles does the company need, which does it hire, which does it train, and which does it rent from partners?
5. Governance. Who approves a use case, who can stop it, and against which risk rules?
6. Delivery. What path does an idea follow from intake to production, and who operates the system afterwards?

Three things are often confused with it. The first is the AI strategy, which sets priorities but rarely says who decides. The second is AI governance, which is one of the six components and not the whole. If you are working on that component specifically, the guide on AI governance for business goes deeper. The third is the AI center of excellence, which is one possible organizational unit inside the model and not a synonym for it.

A useful way to check whether a company has an operating model is to ask three people the same question: "If this AI system gives a wrong answer to a customer tomorrow, who gets the call?" If the three answers differ, the company has tools and projects. A model is still missing.

Why most AI programs stall without one

The data from the last two years points in the same direction, across different firms and different samples.

BCG surveyed 1,000 senior executives in 59 countries for its report Where's the Value in AI? and found that 74% of companies had yet to show tangible value from AI. Only 4% had built advanced capabilities across functions and were consistently generating significant value, with another 22% starting to see substantial gains. The most cited finding of that study is about where the difficulty sits: around 70% of the challenges come from people and processes, 20% from technology and only 10% from the algorithms.

McKinsey's global survey The state of AI: How organizations are rewiring to capture value tested 25 organizational attributes against bottom-line impact from generative AI. The redesign of workflows had the biggest effect. Yet only 21% of respondents whose organizations use generative AI said they had fundamentally redesigned at least some workflows.

Put the three studies together and the diagnosis is organizational. Value depends on redesigning how work gets done, most of the difficulty is people and process, and few companies have scaled anything. An operating model is the instrument a company uses to address exactly those three points.

The symptoms of a missing model are easy to recognize:

• Pilots that never end. Nobody has the authority to promote a pilot to production or to close it.
• Duplicate spending. Three departments buy three tools that do the same thing, each with its own contract and its own data exposure.
• Orphan systems. The team that built a model has moved on, and nobody monitors its quality.
• Governance as a brake. Legal and security see use cases at the end, so their only possible contribution is a late no.
• Shadow use. Employees solve their own problems with personal accounts because the official path is too slow. The guide on shadow AI risk management covers how to measure and contain it.

None of these is a technology problem. Each is a missing decision right.

AI Operating Model Options: Hub and Spoke, Centralized, Federated

There are three structures a company can realistically choose from. A fourth, fully decentralized, is where many companies start by accident and is worth describing only as a warning.

Centralized

One team holds the AI budget, the specialists, the platforms and the backlog. Business units submit requests and the central team delivers.

It works when the company is at the beginning, skills are scarce, and the first priority is to build shared foundations: data access, a secure model platform, vendor contracts, basic rules. It gives consistency and avoids duplicated spending.

It breaks when demand grows. The central team becomes a queue. Business units feel that AI is done to them, adoption suffers, and the specialists are too far from the process to redesign it. In my experience a purely centralized model has a useful life of 12 to 18 months in a company that takes AI seriously.

Federated

Each business unit or function has its own AI people, budget and priorities. A light central function sets a few common rules.

It works in groups whose businesses are truly different from each other, or in companies where the units already have strong digital teams. Speed and closeness to the process are its strengths.

It breaks on everything that should be common. Platforms multiply, the same vendor is negotiated five times, risk rules are interpreted differently in each unit, and specialists in small units have no peers and no career path. A federated model without strong shared standards drifts back into uncoordinated decentralization.

Hub and spoke

A central hub owns what must be consistent. Spokes inside the business units own what must be close to the work. The hub is usually called an AI center of excellence, an AI office or a platform team. The spokes are product owners, analysts and sometimes engineers who report to the business and work with the hub's standards and tools.

This is the hybrid that most mature companies converge on, and the McKinsey survey shows why. Its respondents report that risk and compliance and data governance are often run with a fully centralized model such as a center of excellence, while tech talent and the adoption of AI solutions are most often organized in a hybrid or partially centralized way. Companies centralize selectively, one element at a time, instead of choosing a single structure for everything.

The comparison

The accidental fourth option

Fully decentralized means every team buys and builds on its own with no common rule. It is what a company has when it has not chosen anything. It produces the fastest first results and the most expensive second year, because everything built has to be re-platformed, re-contracted or shut down.

How to choose: five questions that decide the structure

The right structure depends less on fashion and more on five facts about the company.

1. How similar are your business units? If the units share customers, data and processes, a strong hub pays off because what is built for one serves the others. If they share little, the hub should be thin and focused on rules, platforms and contracts.

2. Where is your data? If the data that matters sits in one or two core systems under a single owner, centralizing the technical work is natural. If each unit has its own systems, the people who know the data are in the units and the spokes need real capacity.

3. How scarce are skills? With fewer than five people who can build and evaluate AI systems, splitting them across units leaves each one alone and ineffective. Keep them together until there are enough to seed the spokes.

4. How regulated are you? In financial services, healthcare and other regulated sectors, the cost of inconsistent risk decisions is high. Central control of risk rules is not optional, whatever happens to the rest.

5. How strong are your business leaders on technology? A spoke works only if the unit's leader accepts ownership of the result. Where leaders still see AI as an IT matter, start centralized and move ownership outward as they are ready.

A practical rule follows from these questions. Companies under roughly 200 employees rarely need a formal structure: one accountable executive, one person or partner with technical competence and a short list of rules are enough. Between 200 and 5,000 employees, plan for hub and spoke even if you start centralized. Above that size, or in diversified groups, expect a federated structure with a hub that is strong on standards.

Whatever you choose, write down the date on which you will review it. The structure that fits the first year is rarely the one that fits the third. The broader sequence of stages is described in the enterprise AI adoption framework.

The AI center of excellence: what it should own and what it should hand off

The AI center of excellence is the most common form the hub takes, and the most commonly misdesigned. A good one is small, has a clear mandate and measures itself on what the business units achieve. A bad one is a large team that produces frameworks, slide decks and a growing backlog.

What the center of excellence should own

• The platform. Approved models, the secure environment in which they run, access to company data, monitoring and cost tracking.
• The standards. How a use case is assessed for risk, how a system is tested before release, how prompts, models and data sources are documented.
• Vendor and contract management. One negotiation per vendor, one set of data protection terms, one view of spending. The decision logic is covered in the guide on build vs buy for AI software.
• Scarce skills. Machine learning engineers, evaluation specialists, AI security. People who would be underused inside a single unit.
• The portfolio view. One list of every AI initiative in the company, with owner, stage, cost and expected value.
• Enablement. Training paths, internal communities, reusable components and examples.

What it should hand off to the business

• Use case ownership. The business leader whose numbers will change owns the use case, its target and its budget.
• Process redesign. Only the people who run a process can change it. The center can coach. It cannot do it for them.
• Adoption. Getting a team to change how it works is line management. The method is described in the AI change management framework.
• Day-to-day operation. Once a system is live, the unit that uses it monitors the outputs and handles exceptions, with the center as second-line support.

Three ways a center of excellence fails

The ivory tower. The center publishes principles and reference architectures, but ships nothing. Business units learn to work around it. The cure is to give the center delivery targets tied to business results, not to documents produced.

The bottleneck. Every request, however small, must pass through the center. The queue grows, and shadow use grows with it. The cure is a tiered path: low-risk uses of approved tools need no central approval, only registration.

The service desk. The center takes orders from the units and builds whatever is asked, without challenging the business case. It becomes an internal software house with no say on priorities. The cure is the portfolio process described below, with the center at the table when initiatives are ranked.

How big should it be?

Smaller than most people expect. For a company of 500 to 2,000 employees, a hub of three to six people is usually enough to start: a lead, one or two engineers, a data specialist, and a part-time contribution from legal or risk. What matters more than headcount is that each business unit names a product owner who dedicates real time to the work. A hub of four with six committed product owners in the business will outperform a hub of fifteen with none.

Roles and decision rights: who decides what

Structure tells you where people sit. Decision rights tell you who can commit the company. The second matters more, and it is the part most operating models leave vague.

The roles you need

Executive sponsor. A member of the leadership team who is accountable for AI results as a whole. In the McKinsey survey, 28% of respondents whose organizations use AI say the CEO is responsible for overseeing AI governance, and 17% say the board is. On average two leaders share the responsibility. Shared responsibility is workable only if the split is written down.

AI lead. The person who runs the hub. Some companies call this role Chief AI Officer, others Head of AI or Director of AI. The title matters less than the mandate: authority over the platform and the standards, a seat when priorities are set, and no ownership of the business results, which stay with the units.

Business product owner. One per use case, inside the business unit. Owns the target, the budget, the process change and the decision to continue or stop. This is the role most often missing, and its absence is the best predictor of a pilot that goes nowhere.

Data owner. The person accountable for the quality and the permitted use of each data source. Without named data owners every project begins with weeks of negotiation. The guide on AI data governance covers how to assign them.

Risk, legal and security. Involved at intake, not at release. Their job in the model is to classify the use case by risk and to state the conditions under which it can proceed.

Platform and engineering. The people who build, integrate and operate. They can sit in the hub, in IT or with a partner, as long as someone inside the company can evaluate their work.

The IBM study gives a sense of how fast this map is changing: 54% of CEOs say they are hiring for AI-related roles that did not exist a year earlier, and they estimate that roughly a third of the workforce will need retraining or reskilling over three years. Not every role requires a new hire, but every role requires a name next to it.

The decision table

A one-page table is the most useful artifact of the whole exercise. Adapt the rows to your company, and keep to one name per cell in the "decides" column.

Two tests tell you whether the table is real. First, each decision has one decider, not a committee. Second, the people named have seen the table and agreed to it. A table that lives only in a consultant's document changes nothing.

Funding and prioritization: run a portfolio, not a project list

How AI is funded shapes behavior more than any organization chart. Three funding patterns are common, and each produces a predictable result.

Central funding of everything. The hub pays for platforms, pilots and production. Business units have no cost and therefore no reason to prioritize. Requests multiply and the hub rations by queue.

Business units fund everything. Each unit pays for its own tools and projects. Shared platforms are nobody's priority, so they are not built, and every unit pays again for the same foundations.

Split funding. The hub funds the shared platform and a seed budget for pilots. Business units fund their use cases once they pass from pilot to production, and carry the running costs in their own budget. This is the pattern that works, because it makes the unit weigh the cost of a system against its benefit at the moment the commitment becomes serious.

IBM's survey found that 67% of CEOs say more budget flexibility is needed to take advantage of digital opportunities. The practical answer is a reserve that can be released in stages, against evidence, and never an unlimited fund.

Stage gates

Four gates are enough for most companies:

1. Intake. A one-page description: problem, owner, expected value, data needed, risk class. Cost to prepare: hours.
2. Pilot approval. A limited budget and a fixed duration, usually six to ten weeks, with a success threshold agreed before starting.
3. Production approval. The pilot met its threshold. The business unit commits budget, the process change is planned, monitoring is defined.
4. Scale review. Three to six months after release: did the value appear in the numbers of the business? Extend, hold or retire.

Each gate needs a kill criterion written in advance. A pilot without a stop condition will be extended indefinitely, because nobody wants to be the person who ends it.

Fewer initiatives, not more

The BCG study contains a finding that surprises most leadership teams. The companies that lead in AI value pursue, on average, only about half as many opportunities as their less advanced peers, and expect more than twice the return. They also report 1.5 times higher revenue growth over three years. Concentration is a feature of the model. A portfolio committee that approves everything is not doing its job.

To rank initiatives, three criteria are sufficient: value that can be measured in an existing business metric, feasibility with the data and skills the company has now, and risk. The method for the first is in the guide on AI ROI for business.

Governance inside the operating model

Governance is one component of the model, and it works only when it is built into the flow rather than added at the end.

The practical design has three elements.

Risk tiers. Classify use cases into three or four levels, from low (internal drafting support with an approved tool) to high (decisions that affect customers, employees or regulated activities). Low-tier uses follow a standard path with registration only. High-tier uses get a full review. The tiering is what keeps the center of excellence from becoming a bottleneck.

A reference framework. There is no need to invent one. The NIST AI Risk Management Framework, published in January 2023, organizes the work into four functions: Govern, Map, Measure and Manage. ISO/IEC 42001, published in December 2023, defines the requirements for an AI management system and can be certified. Either gives auditors, customers and your own board a recognizable structure.

A regulatory calendar. Companies that operate in the European Union need the dates of the AI Act in their plan. The prohibitions and the AI literacy provision have applied since 2 February 2025. The obligations for general-purpose AI models have applied since 2 August 2025. The transparency obligations apply from 2 August 2026. After the amendments adopted in July 2026, the obligations for high-risk systems listed in Annex III apply from 2 December 2027, and those for AI embedded in regulated products from 2 August 2028. On literacy, the regulation asks providers and deployers to take measures that support the AI literacy of their staff. It does not prescribe a specific level or a specific course.

Two documents make governance concrete for employees: an AI acceptable use policy that says what is allowed with which tools, and an inventory of AI systems with their owner and risk tier. If agents that act on systems are part of your plans, the additional controls are described in the guide on AI agent governance.

From pilot to production: the delivery process

The last component of the model is the path an idea follows. Most companies have the first half of it, the part that produces demos, and lack the second half, the part that produces running systems.

A complete path has six steps:

1. Intake and triage. Ideas enter through one door. The hub and the business owner check for duplicates, assign a risk tier and decide within two weeks whether the idea proceeds.
2. Design. The business owner describes the current process and the target process. This is where workflow redesign happens, and it is the step most often skipped.
3. Build or buy. The hub selects the approach using approved platforms and vendors.
4. Validation. The system is tested on real cases, with pass thresholds set before testing. Risk and legal confirm that the conditions set at intake are met.
5. Release and adoption. Training, updated procedures, a named person for questions. The unit's manager, not the hub, leads this.
6. Run and review. Monitoring of quality, cost and usage. A review date in the calendar. A retirement plan if the numbers do not hold.

The handover between step 4 and step 5 is where systems become orphans. The rule that prevents it is simple: no system goes live without a named business owner who has accepted, in writing, responsibility for its operation. If nobody will sign, the system is not ready, or it is not needed.

Skills are part of delivery, not a separate track. Every release should include the training of the people who will use the system, as described in the AI training for employees playbook.

What this looks like in mid-sized companies

The principles above can sound like something for corporations with a thousand people in IT. The opposite is true. The projects I have led in smaller organizations worked for the same reason: ownership was clear before any tool was chosen.

Four results from my own work make the point:

• A sports distribution company increased sales by 30% with AI applied to marketing.
• A hotel grew revenue from 9 million to 10 million.
• A medical center increased its capacity by 20%.
• A farm stay doubled its guests.

None of these companies had a center of excellence, a Chief AI Officer or a portfolio committee. My first request in each engagement was the same, and it is the request I would make of a company fifty times larger: a person in the business who owns the result, a budget that person controls, and one number that will tell us whether it worked. With those three in place, the choice of tools is the easy part. Without them, the best tool produces a demo.

For a company of that size, the whole operating model fits on one page. The discipline is the same as in a large group. Only the apparatus is lighter.

If you are unsure which structure fits your company, or you have pilots that are not turning into results, request a consultation. I start from your current initiatives and your organization chart, and we map who should own what.

Self-assessment: score your AI operating model

Score each statement from 0 (not true) to 2 (fully true and documented). Be strict: "we are working on it" is a 0.

How to read the result:

• 16 to 20. You have an operating model. Focus on scaling and on reviewing the structure once a year.
• 10 to 15. The foundations exist, with gaps. The lowest scores usually cluster in funding and in production ownership. Fix those first.
• 5 to 9. You have projects, not a model. Start with statements 1, 2 and 3 before approving anything new.
• 0 to 4. AI in your company is happening by accident. The 90-day plan below is written for you.

Statement 10 deserves a comment. A company that has never stopped an AI initiative is failing to decide, since no portfolio succeeds at everything. For a broader diagnostic that also covers data and technology, use the AI readiness assessment.

A 30/60/90-day roadmap

The plan assumes a company with some pilots in progress and no formal model. It needs an executive sponsor and about 20% of the time of one capable person for three months.

Days 1 to 30: see what you have

• Name the executive sponsor and announce it.
• Build the inventory: every AI tool, pilot and system in use, with who pays, who uses it and what data it touches. Expect to find more than you think.
• For each item, write the name of the business owner. Where there is none, mark it.
• Record current spending on AI tools and services across all budgets.
• Choose the target structure with the five questions above, and write a one-page rationale.

Output at day 30: an inventory, a spending figure, a chosen structure.

Days 31 to 60: decide who decides

• Appoint the AI lead and define the hub, even if it is two people.
• Agree the decision table with the leadership team. One name per decision.
• Define the risk tiers and the fast path for low-risk uses. Publish the acceptable use policy.
• Set the funding rule: what the hub pays for, what the units pay for, and when the cost moves.
• Run the first portfolio review on the inventory. Assign owners to orphan initiatives or close them. Aim to stop at least a quarter of what you found.

Output at day 60: a decision table, a funding rule, a shorter and owned portfolio.

Days 61 to 90: run the model on real work

• Take two or three priority use cases through the full delivery path, with gates and thresholds.
• Name product owners in the business units that have active use cases, and give them time for the role.
• Start a monthly portfolio meeting of 60 minutes with a fixed agenda: new intakes, gate decisions, incidents, spending.
• Define the metrics of the model itself and collect the first values.
• Set the date of the first structure review, six to nine months out.

Output at day 90: two or three initiatives moving under the new rules, a working monthly rhythm, baseline metrics.

Ninety days will not finish the job. They are enough to replace accident with intent, which is the step that most companies in the 74% have not taken.

Metrics that tell you the model works

Measure the model, not only the projects. Six indicators are enough.

• Share of initiatives with a named business owner. The target is 100%.
• Time from intake to decision. Under two weeks for the first answer.
• Pilot conversion rate. The share of pilots that reach production or are formally closed at the planned date. Open-ended pilots count as failures of the process.
• Share of production systems with active monitoring and a review date.
• Spending visibility. The share of total AI spending that appears in the portfolio view. Anything below 80% means shadow use is significant.
• Value realized against value promised. Measured at the scale review, in the business unit's own numbers.

Review them quarterly with the executive sponsor. If conversion is low and time to decision is long, the hub is a bottleneck. If visibility is low, the fast path is not fast enough.

Mistakes I see most often

Designing the structure before deciding the ownership. Companies spend months on where the team should report, and no time on who owns the results. Reverse the order.

Hiring a Chief AI Officer as a substitute for a model. A senior title without decision rights, budget or business owners to work with produces a frustrated executive and the same pilots.

Copying a structure from a company ten times larger. The hub and spoke design of a multinational, applied to 300 people, creates more roles than initiatives.

Treating governance as a document. A policy nobody has read, with no intake process behind it, protects no one.

Funding pilots and not production. The pilot is the cheap part. Integration, change management and operation cost more, and if no budget is foreseen the pilot is the end of the story.

Never revising. The model that suited ten use cases will not suit fifty. Put the review in the calendar.

Where to start tomorrow

Take a sheet of paper and list the AI initiatives you know about in your company. Next to each, write the name of the business owner, the annual cost and the metric it is supposed to move. The blanks on that sheet are your operating model's to-do list.

If the blanks are many, or the names are all in IT, a conversation can save months. Ask me for a consultation: I work with founders and leadership teams to set up the ownership, the structure and the first portfolio review, without a platform to sell.

FAQ

What is an AI operating model?

An AI operating model is the way a company organizes ownership, structure, funding, talent, governance and delivery for artificial intelligence. It defines which executive is accountable, where AI specialists sit, who pays for platforms and use cases, who approves or stops an initiative and how an idea moves from intake to a system in production. It sits between the AI strategy, which sets priorities, and the daily execution of teams.

Is a hub and spoke AI operating model better than a centralized one?

For most companies past their first pilots, yes. A centralized model gives consistency and works well for the first 12 to 18 months, but it turns into a queue as demand grows. In a hub and spoke AI operating model, a central hub keeps platforms, standards, risk rules and vendor contracts, while product owners inside the business units own use cases, process redesign and adoption. Small companies with few use cases can stay centralized for longer.

What does an AI center of excellence do?

An AI center of excellence is the central team in a hub and spoke design. It owns the AI platform, the technical and risk standards, vendor relationships, scarce specialist skills, the portfolio view of all initiatives and the enablement of business teams. It should not own business results, process redesign or daily operation of systems, which belong to the business units. Its performance is measured on what the units achieve.

Who should own AI in a company?

Two levels of ownership are needed. At the top, one member of the leadership team is accountable for AI results overall: the CEO in many companies, or a COO, CIO or Chief AI Officer with a clear mandate. At the use case level, a business leader whose numbers will change owns each initiative, including its budget and the decision to continue or stop. Ownership by IT alone is the most common cause of pilots that never reach production.

How big should an AI team be in a mid-sized company?

For a company of 500 to 2,000 employees, a central team of three to six people is usually enough to begin: a lead, one or two engineers, a data specialist and part-time support from legal or risk. More important than the size of the central team is that every business unit with active use cases names a product owner with real time for the role. Partners can cover specialist skills during the first year.

How long does it take to set up an AI operating model?

A first working version takes about 90 days: 30 days to inventory initiatives and spending and choose the structure, 30 days to agree decision rights, risk tiers and funding rules, and 30 days to run two or three use cases through the new process. Reaching maturity, with product owners in every unit and a stable portfolio rhythm, usually takes 12 to 18 months and at least one revision of the structure.

Do small businesses need an AI operating model?

They need the substance, not the apparatus. A company with 30 people can do without a center of excellence or a portfolio committee, provided it has one person accountable for AI decisions, a short list of approved tools and data rules, an owner and a metric for each use of AI that matters, and a budget line. That can fit on one page, and it prevents the duplicate spending and unowned tools that affect small companies as much as large ones.