The Production Readiness Gate: What an Enterprise AI Agent Must Pass Before Go-Live

A production-ready enterprise AI agent has passed seven gates with named owners and recorded evidence. It has demonstrated business value, authorized data access, measurable quality, security, governance, operational controls, and appropriate human oversight. This makes AI agent production readiness a repeatable go-live decision rather than a judgment based on a successful demonstration.

What does "production-ready" mean for an enterprise AI agent?

A production-ready AI agent performs a defined business task within explicit boundaries, with evidence that it is safe, measurable, governable, and operable. A successful demo alone does not establish readiness. An AI agent go-live checklist should cover value, data, evaluation, security, governance, operations, and human oversight.

The Production Readiness Gate is Brillio's proposed model, built on the public standards and surveys cited here. It is not an industry standard. Its purpose is to give enterprise leaders a consistent structure for deciding whether an agent has enough evidence to move from pilot to production.

Why do AI agents stall before go-live?

Agentic AI production readiness can break down when pilots demonstrate that an agent can act but do not establish that it should act, can act safely, or can deliver consistent results at production scale.

What Gartner predicts

Gartner's 25 June 2025 press release predicts that more than 40% of agentic AI projects will be canceled by the end of 2027 because of escalating costs, unclear business value, or inadequate risk controls. Gartner also estimates that only about 130 of the thousands of agentic AI vendors are real and says some use cases described as agentic may not require agentic implementations. These are predictions, not measured outcomes.

What MIT NANDA's sample shows

MIT NANDA's The GenAI Divide: State of AI in Business 2025 (preliminary report, July 2025) found that 95% of organizations in its research sample saw no measurable return from generative AI initiatives. This should not be presented as a universal failure rate. It describes MIT NANDA's preliminary sample of generative AI pilots and reinforces the need for measurable baselines and defined business outcomes.

What LangChain's practitioner survey shows

LangChain's State of Agent Engineering survey, conducted 18 November to 2 December 2025 among more than 1,300 professionals, reported that 57.3% had agents in production and that quality was the top barrier at 32%. It also reported 89% observability, 52.4% offline evals, and 37.3% online evals. Among enterprises with 2,000 or more employees, security was the second-largest barrier at 24.9%.

The Production Readiness Gate at a glance

An AI agent readiness checklist covers seven gates: Purpose and Value, owned by the business owner; Data and Knowledge, owned by the data owner or Chief Data Officer; Quality and Evaluation, owned by the AI engineering lead with QA; Security and Identity, owned by the CISO or security architect; Governance and Compliance, owned by risk and AI governance; Operability, owned by SRE or the platform lead; and Human Oversight and Adoption, owned by the business operations lead.

The seven gates of the Production Readiness Gate

Gate 1: Purpose and Value

Gate 1 checks whether the use case has measurable value and genuinely needs an agent rather than simple automation. The business owner is accountable. Evidence is a one-page charter covering the scoped task, KPI, measured baseline, cost ceiling, exit criteria, and rollback criteria. The review requires a measured baseline, signed success threshold and kill criteria, and a clear reason for agentic behavior. The evidence should make expected value and operating boundaries explicit before technical approval.

Pass if: the use case, value threshold, and exit conditions are approved.

Gate 2: Data and Knowledge

Gate 2 checks whether the agent can access the right knowledge without reaching unauthorized information. The data owner or Chief Data Officer owns the review. Evidence includes a source inventory, mapped permissions, versioned grounding sources, freshness expectations, and PII and PHI handling requirements. Retrieval quality must be tested on representative queries rather than assumed from a successful demonstration. The review should show that the agent receives appropriate information and respects the defined data boundaries.

Pass if: data access and retrieval tests meet the approved requirements.

Gate 3: Quality and Evaluation

Gate 3 asks whether the agent is accurate enough for its intended task. The AI engineering lead with QA owns an evaluation package containing edge, adversarial, and failure cases, offline evaluation results, an online evaluation plan, and a regression suite. Observability shows what happened, evaluation shows whether it was right. An LLM-as-judge evaluation uses one model to assess another model's output against defined criteria. LangChain reported 89% observability, 52.4% offline evals, and 37.3% online evals in its 2025 survey. The regression suite should run whenever the agent changes.

Pass if: task thresholds are met and online sampling is active.

Gate 4: Security and Identity

Gate 4 establishes AI agent security by checking whether the agent can be manipulated or given more authority than its task requires. The CISO or security architect owns the review. Evidence includes a threat model mapped to OWASP ASI01 through ASI10, a distinct identity for each agent, least-agency tool scopes, short-lived credentials, goal-hijack and prompt-injection tests, and a red-team report. A non-human identity is an identity assigned to an automated agent rather than a person. Critical security findings must be closed.

Pass if: the agent cannot exceed its declared permissions.

Gate 5: Governance and Compliance

Gate 5 checks whether the organization can identify, classify, approve, and reconstruct the agent's actions. Risk and compliance with the AI governance lead own the review. Evidence includes a registry entry covering owner, purpose, model, tools, data, and risk tier; an audit-trail design; policy mapping to NIST AI RMF, ISO/IEC 42001, and applicable sector rules; and an impact assessment. AI agent governance makes ownership, risk decisions, and accountability traceable throughout the agent's lifecycle.

Pass if: registration, risk tiering, approval, and action history are documented.

Gate 6: Operability

Gate 6 checks whether the agent can be operated safely when costs, traffic, dependencies, or behavior change. The SRE or platform lead owns the review. Evidence includes end-to-end tracing, cost, rate and step limits, fallback behavior, a rollback runbook, a tested AI agent kill switch, and an incident playbook. The organization must demonstrate shutdown and restoration rather than relying on an untested emergency procedure. A named on-call owner completes the operating model.

Pass if: rollback and shutdown drills pass and budget controls are enforced.

Gate 7: Human Oversight and Adoption

Gate 7 checks whether people know when to trust the agent, intervene, and escalate a problem. The business operations lead owns the review. Evidence includes human-in-the-loop control points by risk tier, escalation paths, user training, and a feedback loop. High-impact actions require human approval, while lower-risk actions can use defined sampling or exception review. The evidence should demonstrate that users understand both the agent's capabilities and its boundaries.

Pass if: consequential actions have tested human oversight.

How much evidence does each autonomy tier need?

Evidence should rise with autonomy. Tier 1, assistive, is read-only and leaves the action with the human. Tier 2, supervised, requires human approval for each proposed action. Tier 3, bounded autonomy, permits action within defined limits with sampling or exception review. Tier 4, high-impact autonomy, covers money, health, legal, or regulatory consequences and requires full evidence, continuous monitoring, and executive sign-off.

AI agent guardrails should become stronger as authority increases. A read-only policy assistant needs retrieval evidence, while an agent initiating a consequential transaction needs stronger identity, tool, rollback, monitoring, and human approval evidence.

Who signs off on a go-live decision?

The business owner is accountable for go-live, while the AI engineering lead is responsible for technical readiness. Security, risk, the data owner, and legal are consulted; operations and support are informed. Each gate requires a recorded decision, and incomplete evidence must be documented through conditions or a deferral.

How do you run a go-live review?

Start by assembling the evidence pack for every gate, including the charter, data controls, evaluation results, security testing, governance record, operating runbook, and oversight plan. Use the AI agent deployment checklist to walk through the gates in order and test whether the evidence supports each pass condition.

Record one of three outcomes: approve, approve with conditions, or defer. Record the decision, evidence gaps, and responsible owners, then schedule the next review based on the outcome and risk level.

What changes in banking, healthcare and insurance?

Banking

Banking requires attention to model risk, identity, auditability, and accountability. Federal Reserve SR 26-2, issued with the OCC and FDIC on 17 April 2026, revised model risk management guidance, superseding SR 11-7 and SR 21-8 and emphasizing a risk-based, proportionate approach. Banks should confirm with their model risk teams how agentic systems are treated.

Healthcare

Healthcare requires careful PHI handling and human review for consequential decisions. Permissions, audit trails, escalation paths, and oversight should be explicit before an agent acts on sensitive information or influences a consequential workflow.

Insurance

Insurance requires explainability around claims and underwriting decisions, along with attention to evolving state-level rules. A customer-facing AI Concierge also needs clear escalation and human oversight when its interactions can influence consequential customer decisions.

Which standards support the gates?

NIST AI RMF and AI 600-1

NIST AI RMF 1.0 was released in January 2023, while NIST AI 600-1, the Generative AI Profile, was released in July 2024. In Brillio's mapping, these frameworks support Gates 1, 3, 5, and 7 by helping structure risk identification, evaluation, governance, and oversight.

OWASP Top 10 for Agentic Applications

OWASP Top 10 for Agentic Applications 2026 was published on 9 December 2025 and built with more than 100 experts. Its ten risk categories include Agent Goal Hijack, Tool Misuse and Exploitation, and Identity and Privilege Abuse. In Brillio's mapping, it primarily supports Gate 4.

ISO/IEC 42001 and the EU AI Act

ISO/IEC 42001:2023 is a certifiable AI management system standard. In Brillio's mapping, it supports Gate 5. Regulation (EU) 2026/1744, the EU Digital Omnibus on AI, currently defers high-risk obligations to 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems. Organizations should confirm these timelines against the Official Journal of the EU.

NIST CAISI AI Agent Standards Initiative

NIST CAISI launched the AI Agent Standards Initiative on 17 February 2026 to advance industry-led standards, open protocols, and research on agent security and identity. In Brillio's mapping, it is a watch item for future security and identity practices, not a binding standard.

What are the most common go-live mistakes?

Common mistakes include treating observability as evaluation, leaving ownership unnamed, using only clean demo data, skipping tested rollback, and granting broad tool permissions. These gaps can make an agent appear ready without demonstrating that its actual production behavior is controlled. Each gate should use evidence from the intended task, data, permissions, failure cases, and approved autonomy level.

Why is go-live not the finish line?

Production evidence can become stale after a model change, tool change, meaningful drift, or incident. Re-certification keeps approval aligned with the system actually operating in production. Monitoring and evaluation therefore continue after launch rather than ending at deployment.

How can a control plane help enforce the gate?

A control plane can make the gate repeatable by connecting an agent registry, gateway, monitoring, and governance controls to production evidence. We offer ADAM, Agentic Data and Application Management, within our Enterprise AI Accelerator. Our Enterprise AI Accelerator solutions can support a structured approach to enterprise AI delivery without replacing accountable business and technical decisions.

AI accelerators for enterprise should connect ownership, permissions, evaluation, monitoring, and approvals. An AI product accelerator or AI business accelerator should ultimately be judged by whether agents survive go-live, operate within declared boundaries, and continue meeting their evidence requirements.

Frequently asked questions

What does production-ready mean for an AI agent?

Production-ready means an AI agent has passed gates for value, data, quality, security, governance, operations, and oversight. Each gate has an owner, recorded evidence, and a pass condition matched to the agent's autonomy and business risk.

How do you test an AI agent before production?

Test an AI agent with representative, edge, adversarial, and failure cases, then run offline and online evaluations against task-specific thresholds. Add retrieval, security, regression, operational, and human escalation tests before recording the go-live decision.

Why do AI agents fail in production?

AI agents fail when value is unclear, quality is insufficient, permissions are broad, data is poorly controlled, rollback is untested, or oversight is weak. Production can expose model, tool, traffic, and data changes that a limited demonstration did not reveal.

Who should approve an AI agent for go-live?

The business owner is accountable for go-live, while the AI engineering lead is responsible for technical readiness. Security, risk, the data owner, and legal are consulted, while operations and support are informed. Every gate needs a recorded decision.

What is the difference between AI agent observability and evaluation?

AI agent observability shows what happened, evaluation shows whether it was right. Observability captures actions and operational behavior, while evaluation compares results against defined criteria. Both matter because an observable result can still be incorrect.

Which standards apply to enterprise AI agents?

NIST AI RMF, NIST AI 600-1, OWASP Top 10 for Agentic Applications, ISO/IEC 42001, sector requirements, and EU AI Act timelines can support enterprise AI agent governance. NIST CAISI should also be monitored as an emerging initiative.

References

Gartner, "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027," 25 June 2025.

MIT NANDA, "The GenAI Divide: State of AI in Business 2025," preliminary report, July 2025.

LangChain, "State of Agent Engineering," survey conducted 18 November to 2 December 2025.

OWASP, "Top 10 for Agentic Applications 2026," 9 December 2025.

NIST, "AI Risk Management Framework," January 2023.

NIST, "AI 600-1: Generative AI Profile," July 2024.

NIST CAISI, "AI Agent Standards Initiative," 17 February 2026.

ISO/IEC, "ISO/IEC 42001:2023," 2023.

European Union, "Regulation (EU) 2026/1744, Digital Omnibus on AI," 2026.

Federal Reserve, OCC and FDIC, "SR 26-2: Revised Guidance on Model Risk Management," 17 April 2026.