Most enterprise AI programs die at the same place: not the pilot, not the model choice, but the org design decision the organization made without noticing it was making one.
Two large services companies I have worked with started their AI programs within about a quarter of each other. Both are in adjacent industries. Both had similar budgets. Both hired competent leaders and picked reasonable initial use cases. Two years in, one has AI woven into a dozen production workflows, a real internal community of practitioners, and a clean line item on the P&L. The other has a shelf of impressive pilots, a growing sense of internal fatigue, and a CFO who has stopped asking about the roadmap.
The variable was not talent. It was not the tech stack. It was not even the choice of vendor. Both programs picked competent versions of everything on those axes. What separated them was a decision they each made, quietly, in the first six months: how the AI capability was going to sit inside the organization.
That decision has a shape, and by 2026 the industry has enough field experience to name three of them. Every enterprise AI program eventually lands in one of these three shapes, or a hybrid that leans toward one. The shapes are not equal. They have different failure modes, different natural fits, and different half-lives. Picking the wrong one for your business does not just underperform — the friction compounds until the program dies.
The three shapes
The Center of Excellence. A single centralized AI team, typically five to fifteen people, sits inside the strategy, technology, or transformation function. This team owns tooling, standards, evaluation, model risk management, and often the actual deployments themselves. Business units bring problems to the CoE and receive solutions in return. The CoE is the answer to "who owns AI at this company?" It is the shape most enterprises reach for first, and it is the shape most consulting firms recommend by default. The industry calls it centralized, or a hub without spokes.
Embedded practitioners. AI-capable people are distributed across business units, each of which owns its own AI capability. Marketing has its analytics-and-AI team. Ops has one. Finance has one. There may be a small coordinating function — a community of practice, a shared tooling council — but each business unit is accountable for its own AI outcomes. The industry calls this federated or decentralized, and it is the natural shape for companies with strong business-unit autonomy and diverse workflows.
The platform team. A small technical team, typically ten to twenty engineers, builds and operates the internal substrate that everyone else uses: model gateways, retrieval infrastructure, evaluation harnesses, agent frameworks, observability, security controls. The platform team does not deploy AI applications itself. Business units and product teams do the deploying, using the platform. The industry calls this hub-and-spoke, or in its more mature form, an AI platform organization. It is the shape engineering-led companies gravitate toward, and it is the one most closely modeled on how modern software organizations built their data platforms over the last decade.
Every framework I have seen from an analyst or consultancy — from the large firms to the boutique AI practices — reduces to these three shapes with different labels. One useful public treatment, Assembly's 2026 operating-model guide, lays out the same centralized / federated / hub-and-spoke taxonomy and puts the cost of choosing wrong at twelve to eighteen months of rework. The vocabulary varies. The choice does not.
Where each shape actually fits
The Center of Excellence fits highly regulated environments where model risk management, auditability, and centralized approvals genuinely need to sit with one team. It fits organizations early in their AI adoption where the number of use cases is small and building shared literacy matters more than moving fast. It fits companies whose business units are structurally too small or too under-resourced to sustain their own capability. If you are a mid-market insurance company running your first eight or nine AI projects, the CoE is the correct choice.
Embedded practitioners fit companies with meaningfully different business units, each with its own P&L, its own workflows, and its own tolerance for experimentation. It fits organizations where the AI use cases are so domain-specific that routing them through a central team would only slow everyone down. It fits mature tech companies where every product team already ships software and adding AI capability to their existing craft is a smaller step than establishing a new central team.
The platform shape fits companies past the pilot stage that have identified enough repeatable AI patterns to justify building infrastructure around them. It fits engineering-first organizations that already know how to run internal platforms. It fits programs where the ambition is horizontal AI capability — retrieval, agents, evaluation, monitoring — that dozens of teams will consume, rather than a handful of monolithic AI applications maintained by one team.
How each one dies
The CoE dies of its own success. The team becomes the bottleneck it was supposed to prevent. Approval queues form. The intake process gets a triage step. Then a scoring rubric. Then a quarterly prioritization council. The teams that own the P&L stop bringing new use cases to the CoE because they cannot afford the wait, and start building AI capability inside their own group — but without the standards, the governance, or the model risk oversight the CoE was created to provide. This is what practitioners in 2026 are calling shadow AI, and it is the near-universal failure mode of the centralized model: the same structure meant to accelerate adoption ends up generating approval queues and documentation that business teams route around rather than through. The operating-model guidance converging in 2026 keeps landing on the same conclusion — choosing the wrong shape, and then refusing to change it, is what turns a portfolio of pilots into a program that never scales.
Embedded practitioners die of fragmentation. Every business unit re-solves the same problems from scratch. There are five different retrieval implementations, four different evaluation frameworks, three vendor contracts with the same model provider, and no way to know which team is doing what. When a governance concern arises — a data leak, a hallucination in a customer-facing surface, a regulatory question — nobody knows which team owns which risk. Learning does not compound; every business unit pays the AI learning cost fresh. BCG's 2025 research found that only about a quarter of companies are capturing real value from AI at scale — and the distance between those leaders and everyone else is far less about the technology than about how the AI work is organized. Fragmentation is a large part of that gap.
The platform team dies of over-engineering. A small team of thoughtful platform engineers builds a beautifully designed AI substrate that nobody uses, because business units cannot get help translating their actual problems into the platform's primitives. The platform ships features quarterly; adoption is measured in single-digit team counts. The team explains, correctly, that they are horizontal infrastructure and it is not their job to build products on top of the platform. Eighteen months in, executive sponsorship starts asking why the investment has not produced anything a customer can see. This is the failure mode nobody warns you about, and it is why platform-first AI organizations without an explicit adoption function tend to underdeliver for years before either pivoting or getting reorganized.
MIT's NANDA project put the industry-wide failure number at ninety-five percent of generative-AI pilots failing to produce measurable financial return, based on a study of over three hundred enterprise deployments. What that number obscures is the specificity of why. Every one of those failed programs picked a shape. The failure is not that AI does not work. The failure is that most companies picked the wrong shape for their stage, their culture, and their portfolio, and the mismatch compounded until the program lost its funding.
The default trap
Most companies pick the Center of Excellence not because it is the right shape but because it feels like the safest one. It gives an executive committee a single throat to choke. It gives procurement one vendor relationship to manage. It gives risk and compliance a single point of contact. It looks organized on a slide. And in the first year, when the use cases are few and the appetite for governance is high, it works.
The trap is that by the time a program has enough use cases and enough business-unit pull to need one of the other shapes, the CoE has become an entrenched function with headcount, career ladders, and political weight. Sunsetting it is a re-org. Federating its work back into business units is a headcount fight. Reshaping it into a platform team requires a different kind of engineer than the CoE hired for. So most companies do not reshape. They add a coordinating layer on top of the CoE, add a governance council, add a business-unit AI champion program, and slowly the org chart accumulates the shape it should have picked at the start — while the CoE remains the bottleneck.
There is nothing wrong with starting as a CoE. There is a great deal wrong with staying one longer than the stage of your program justifies. The operating-model guides and practitioner accounts converging on this in 2026 all describe the same evolution: a centralized CoE that, twelve to twenty-four months in, has to hand execution outward to the business units while keeping standards, governance, and model-risk oversight central. The most useful move for any enterprise CoE leader in 2026 is to plan that sunset before it becomes urgent.
The pilot-to-program lens
An earlier note argued that the transition from AI pilot to AI program is the moment most initiatives quietly die. The three-shapes question is the specific form that transition takes. Pilots can run inside any of the three shapes without much friction. Programs cannot. A program requires standards, repeated deployment patterns, cross-team learning, and governance that scales — and each shape scales those differently.
If your program is still deploying its first handful of use cases, a Center of Excellence is probably the right shape and will remain so for another twelve to eighteen months. If your program has deployed a dozen use cases and is starting to see repeated patterns across business units, you are at the moment where the CoE becomes a bottleneck and the platform shape starts to earn its cost. If your program has diverse business units each running their own AI initiatives semi-independently, and the friction you feel is coordination rather than execution, then embedded-with-a-lightweight-coordinating-function is where you already are, whether or not the org chart admits it.
The mistake is not picking the wrong shape for your first year. The mistake is refusing to change shapes when the program crosses a stage boundary. In practice, most enterprise AI programs need to change shape at least once — often from a CoE toward a platform or a federated model — within the first three years of serious investment.
Three questions worth asking
If you are trying to figure out what shape your AI program should be in, or whether it needs to change:
1. What is the shape of the demand? If AI demand is bunched into a few large use cases owned by a small number of stakeholders, centralize. If demand is diffuse — many small use cases across many business units — federate or platform. Do not pick a shape that fights the shape of your demand.
2. Who is accountable when an AI system misbehaves? If accountability lives centrally with the AI team, you have a CoE. If accountability lives in the business unit that deployed the system, you have embedded practitioners or a platform. Do not accept an org design where governance and accountability point at different teams.
3. What is the actual bottleneck? Programs die at their bottleneck, not their ambition. If your bottleneck is inconsistent evaluation practice, that is a platform problem. If your bottleneck is business units not knowing what AI is for, that is a CoE-with-solution-architects problem. If your bottleneck is coordination across too many small AI initiatives, that is a governance-layer problem on top of an embedded model. Match the shape to the bottleneck you actually have, not the one you had a year ago.
The bet you are making
The shape you pick is not just an org chart. It is a bet on what your business needs AI to be for you. A CoE is a bet that AI is a specialized capability a central team should own, at least for now. Embedded practitioners is a bet that AI is a general-purpose tool every function should master on its own. A platform is a bet that AI is infrastructure — foundational, horizontal, consumed by many teams — and that your competitive edge will come from what your teams build on top rather than from any single AI application.
All three bets can pay off. All three can also compound in the wrong direction. The winning move in 2026 is not to pick the "right" shape by industry benchmark. It is to name the shape you are in, name the shape your program is asking to become, and design the transition on purpose rather than by accretion. Most companies do not. That is a large part of why ninety-five percent of enterprise AI programs are producing nothing measurable — and why the five percent that are producing something quietly changed shape at least once on the way there.
Sources
- Assembly, AI Initiatives Operating Model Guide (2026): Hub-and-Spoke, Federated, or Centralized?, 2026.
- Boston Consulting Group, AI at Work 2025: Momentum Builds, but Gaps Remain, 2025.
- MIT Media Lab NANDA Project, The GenAI Divide: State of AI in Business 2025; coverage: Fortune, August 18 2025.
- Gartner Peer Community, Have you decided on a distinct operating model for AI? Centralized, distributed, or federated?, 2025.
- Adam Valine, Your AI Pilot Is Working. Your AI Program Isn't., Veil Consulting Notes, 2026.