TartanHQ Logo – Powering Seamless Enterprise Workflows with APIs and AI

Enterprise & Industry Insights

Enterprise & Industry Insights

The enterprise AI cold start problem: why agents fail in the first 90 days and what the context layer has to do with it

The enterprise AI cold start problem: why agents fail in the first 90 days and what the context layer has to do with it

The enterprise AI cold start problem: why agents fail in the first 90 days and what the context layer has to do with it

Rohan Mahajan

Rohan Mahajan

10 Min

10 Min

Build Connected Systems with Tartan

Automate workflows with integrated data across your customer applications at scale

The failure rate for enterprise AI agent deployments in 2026 is uncomfortably consistent across independent research. Between 86% and 89% of enterprise AI agent pilots never reach production at meaningful scale, according to studies from McKinsey, Gartner, and the AI Governance Institute published between January and March 2026. The number is not primarily a model quality failure. 

The models work. They perform their assigned tasks with measurable accuracy in controlled evaluation environments.

The failure is what happens between the controlled evaluation environment and the production environment - in the specific window where the agent encounters the actual enterprise it is supposed to operate in, with all of its data complexity, terminological ambiguity, system fragmentation, and operational dynamism.

This window - the first 90 days of production deployment - is where the enterprise AI cold start problem lives. Understanding it requires understanding what the cold start problem actually is, where it originates in the context layer, and what the implementation sequence looks like for closing it before it becomes the reason a promising deployment gets quietly shelved.

Defining the cold start problem precisely

The enterprise AI cold start problem has a specific technical definition that is distinct from general agent failure modes. Snowflake’s internal experiment, published in March 2026, provides the clearest empirical illustration.

Snowflake’s team added a plain-text “data ontology” to an AI agent that was already receiving structured, curated Snowflake semantic views - join keys, table grains, cardinality and fanout hints. The agent already had good data access. The ontology added organisational context: how the enterprise thinks about its own data, how entities relate, and how ambiguous terms should be resolved in this specific enterprise’s context.

The result of adding that organisational context layer was striking: final answer accuracy improved by 20% and average tool calls decreased by 39%. End-to-end latency improved as well. The model did not change. The data did not change. The infrastructure did not change. What changed was the agent’s access to the enterprise’s own understanding of itself - and that change produced a 20% accuracy improvement and a 39% efficiency gain from a single contextual addition.

The cold start problem is the absence of that organisational context at the point of initial deployment. An agent that is technically capable - correct model, correct data access, correct infrastructure - but missing the enterprise-specific context layer is an agent that will perform well on generic benchmarks and poorly on the specific, ambiguous, context-dependent decisions that enterprise operations actually require.

“The organisational cold start is harder because it cannot be resolved by tooling alone - it requires building a governed, machine-readable representation of the organisation’s entire data estate. Context infrastructure, not model quality, is the binding constraint on agent performance.” - Atlan, April 2026

The three dimensions of the cold start problem

The enterprise AI cold start problem has three distinct dimensions that compound on each other in the first 90 days of deployment. Each requires a different context infrastructure to address.

Dimension 1: Semantic cold start. The agent does not know what words mean in this enterprise’s specific context. “Customer” means one thing in the CRM, a different thing in the billing system, and a third thing in the compliance framework. “Revenue” may be calculated differently by the finance team and the product team. “Active account” may have different definitions in three different operational systems.

These ambiguities are invisible in a pilot where the data was prepared specifically for the evaluation. In production, the agent encounters them in every data retrieval, every entity resolution, every cross-system query. It resolves them using the training data’s generalised understanding of these terms - which may not match the enterprise’s specific definitions. The result is coherent-sounding outputs that are systematically misaligned with what the business actually means.

The fix is a knowledge graph and semantic definition layer that encodes the enterprise’s specific terminology - giving the agent an authoritative source for what entities and terms mean in this organisation’s context, overriding the generalised training data understanding. This cannot be built overnight. It requires domain knowledge, governance discipline, and iterative refinement as the agent surfaces ambiguities through its queries.

Dimension 2: Operational cold start. The agent does not know the current state of the enterprise. It knows the historical state - the data that was in the knowledge base at deployment time. But enterprise operations are dynamic: employees join and leave, customers change status, products launch and deprecate, policies are updated, organisational structures change.

An agent operating without real-time operational data connections is an agent reasoning from a picture of the enterprise that begins diverging from reality the moment it is deployed. For consequential decisions - credit assessments, insurance underwriting, compliance checks - this divergence is not a minor accuracy issue. It is a systematic error source that produces wrong decisions on cases where the current operational state differs from the deployment-time snapshot.

The fix is real-time operational data connections - live API access to the HRMS, CRM, ERP, and other systems that hold current operational state. These connections need to be established, validated, and monitored before or immediately at deployment, not added later when output quality issues surface.

Dimension 3: Access calibration cold start. The agent’s permission boundaries are miscalibrated for the actual decision patterns it encounters in production. This is the access governance dimension of the cold start problem - and it runs in both directions.

Permissions that are too broad allow the agent to access data it should not, producing two problems: outputs that incorporate unauthorised context (a compliance risk) and an agent that makes decisions that exceed its intended scope (an operational risk). Permissions that are too narrow prevent the agent from accessing data it needs, producing outputs that are incomplete or systematically wrong in cases where the missing data was the deciding factor.

The pilot environment rarely surfaces access calibration problems because pilot data is curated to fit the intended use case. Production data is not curated. It includes edge cases, unusual entity types, and decision scenarios that the permission model was not designed for. Access calibration requires a governed data access layer that can be adjusted incrementally as production usage reveals where the permission boundaries are set incorrectly.

The 90-day cold start implementation sequence

The cold start problem is solvable. It requires a specific implementation sequence that most enterprise AI deployments do not follow - because the natural pressure is to deploy the agent as quickly as possible and address context issues as they arise. The sequence below is designed for the opposite approach: address context issues before they become output quality failures.

Pre-deployment - Weeks 1–4 - Semantic foundation build

Map the 20–30 core entities the agent will reason about most frequently. For each entity, document the enterprise’s specific definition - how it is defined in each source system, what the canonical definition is for the agent to use, and what the resolution rules are when different systems disagree. 

Build this into the knowledge graph before the agent goes live. The Snowflake experiment shows this is the single highest-ROI pre-deployment investment.

Pre-deployment - Weeks 3–6 - Operational data connection validation

Establish and validate real-time API connections to the operational source systems the agent’s decisions depend on. For each connection, validate that the data fields being returned are current (not cached), that the field values match what the agent’s decision logic expects, and that the connection is stable under production load conditions. Document the validation results. This step is the one most commonly skipped - and the omission is the most common cause of production failures in the first 30 days.

Deployment - Weeks 5–8 - Shadowed production run with calibration feedback loop

Deploy the agent in shadow mode - making decisions against real production data but not acting on them. Compare the agent’s outputs against human decisions on the same cases. Where they diverge, investigate whether the divergence originates in semantic miscalibration (the agent misunderstood an entity), operational data staleness (the agent’s context was out of date), or access miscalibration (the agent lacked or had excess access to relevant data). Use the findings to refine the context layer before the agent goes live in action mode.

Production - Weeks 7–12 - Supervised production with active context refinement

Move to live production with human review of a defined sample - not all decisions, but a statistically sufficient sample to detect systematic errors. Establish a context refinement process: when a production decision is flagged as incorrect, root-cause it to a specific context layer failure and update the context layer to address it. Track the error rate by category (semantic, operational, access) and monitor for declining error rates as the context layer matures.

Post-cold-start - Week 12+ - Context maintenance handoff

By week 12, if the context refinement process has worked, the agent’s error rate should have declined substantially and the error pattern should have shifted from systematic cold-start errors to genuinely novel edge cases. This is the point at which the deployment transitions from active context calibration to ongoing context maintenance - the operational discipline that prevents cold-start errors from recurring as the enterprise evolves. This is not the end of context work. It is the transition from context construction to context maintenance.

The metrics that surface cold start failure early

Detecting cold start problems requires measuring the right things. Most enterprise AI deployments measure what is easy to measure - task completion rate, response time, user satisfaction scores. 

These metrics are important but they do not surface cold start problems early because they measure output volume and speed, not output correctness in the enterprise context.

The metrics that detect cold start failure early:

  • Entity resolution accuracy - for a sample of decisions, what percentage of entity identifications (this customer, this employee, this account) are correct? Systematic entity misidentification is the signature of semantic cold start failure.

  • Context staleness rate - for a sample of decisions, what percentage of the operational data the agent used was accurate as of the moment of the decision? High staleness rates signal operational cold start failure.

  • Permission boundary hit rate - how often does the agent attempt to access data outside its permission scope? Both a high rate (indicating over-narrow permissions) and a zero rate on complex decisions (indicating over-broad permissions where everything is accessible and no constraint is ever hit) are signals of access calibration cold start failure.

  • Human review escalation rate - what percentage of agent decisions are being escalated to human review? An escalation rate that is high and not declining over the first 90 days is a reliable signal that context calibration is not progressing as expected.

What the context layer has to do with it - specifically

The cold start problem is named as an enterprise AI problem. Its root cause is a context layer problem. Every dimension of the cold start - semantic, operational, access - is a failure of a specific context layer component to be established before production deployment.

Semantic cold start is a knowledge graph gap. Operational cold start is a real-time data connection gap. Access calibration cold start is a governed data access layer gap. The agent itself is not the problem - it is performing correctly given the context it has. The context is the problem, and the context is determined by the infrastructure built underneath the agent.

The 86–89% pilot failure rate is not primarily a model quality failure. It is a context infrastructure failure - deployments that went into production without the knowledge graph that resolves enterprise terminology, without the real-time data connections that provide current operational state, and without the governed access layer that calibrates what the agent can and cannot see. The model worked in the pilot. The context infrastructure was never built for production. The result was inevitable.

Building the context layer before deploying the agent is the intervention. The 90-day sequence above is the implementation. The outcome - an agent that passes its cold start window with a declining error rate and a context layer calibrated to the enterprise it is actually operating in - is an agent that can be scaled with confidence rather than shelved with regret.

One platform. Across workflows.

One platform. Across workflows.

Tartan helps teams integrate, enrich, and validate critical customer data across workflows, not as a one-off step but as an infrastructure layer.

Tartan helps teams integrate, enrich, and validate critical customer data across workflows, not as a one-off step but as an infrastructure layer.

Tartan helps teams integrate, enrich, and validate critical customer data across workflows, not as a one-off step but as an infrastructure layer.