Two AI Safety Veterans Launch Certification Standard for Enterprise Agents

The Paradox of Smarter AI

Rune Kvist spent his early career at Anthropic watching frontier models grow more capable. What surprised him wasn't the pace of improvement - it was the adoption curve flattening in regulated industries. Banks, hospitals, and government agencies weren't rejecting AI because it lacked intelligence. They were declining because no one could promise the systems would behave.

"AI is getting smarter at an increasingly rapid rate," Kvist observed. "The surprising thing is that it becomes harder to adopt and harder to control as AI gets smarter, not easier."

That tension led Kvist and his brother-in-law Rajiv Dattani - former chief operating officer at AI safety research organisation METR - to launch Artificial Intelligence Underwriting Company (AIUC) in early 2025. The startup has now raised $55 million across seed and Series A rounds to build what it describes as a certification layer for enterprise AI agents. Ribbit Capital led the $40 million Series A announced this week, with participation from First Harmonic. The earlier $15 million seed came from Nat Friedman's NFDG fund, Emergence, Terrain, and Anthropic co-founder Ben Mann.

AIUC names Cursor, Lovable, Harvey, and ElevenLabs among its early customers - companies building agents that write code, generate legal analysis, or produce synthetic voice at scale.

SOC 2 for Agents

The core product is a standard called AIUC-1, modelled explicitly on SOC 2, the cybersecurity compliance framework that became table stakes for enterprise software over the past decade. Where SOC 2 audits data handling and infrastructure controls, AIUC-1 examines agent behaviour under stress: jailbreak attempts, hallucination triggers, data exfiltration scenarios.

To define what "safe" looks like, AIUC convened a consortium of roughly 250 security and risk leaders - the buyers, not the builders, of AI systems. "These are the people who we meet with on a monthly basis," Dattani explained. "The question we ask them is: When you're buying agents from someone, what would you look for? What are the questions you'd want to ask, and what would you want to see addressed?"

That feedback loop shapes a test suite now comprising some 5,000 scenarios. An agent submitted for certification runs through the battery; AIUC's own AI agents execute the tests, analyse the outputs, and flag anomalies. Humans verify the final audit, which produces a report of around 100 pages detailing where an agent performs reliably and where it doesn't.

The result isn't a pass/fail seal. It's a map. "Here's where it passes and where you can trust it," Dattani said. "And here's where there's concerns. You should be aware of those before you make the decision to buy."

The METR Connection and Frontier Precedent

Dattani's background at METR - where he served as COO from 2024 to 2025 and remains a board member - lends credibility and raises questions about overlap. METR has conducted similar evaluations for frontier labs, though its work historically focused on capability testing: whether agents can reliably complete tasks, not whether they misbehave while doing so. OpenAI tapped METR as one of the independent research organisations investigating its Hugging Face incident earlier this year.

Anthropic chief executive Dario Amodei recently floated the idea of embedding third-party evaluators inside frontier labs to observe and verify safety commitments in real time, citing METR as a candidate. AIUC isn't proposing to embed staff at customer sites, but the structural parallel is clear: both models inject independent oversight into an ecosystem where self-assessment dominates.

The timing is deliberate. Anthropic researcher Jacob Coxon resigned publicly the day before AIUC's Series A announcement, warning that self-improving AI could pose existential risks within the decade. Amodei himself has called for the industry to slow frontier development in response to what he described as a rapid increase in bad-behaviour incidents.

At Opentechwire, we've tracked a pattern across the region: regulators in Singapore, South Korea, and Japan are drafting AI safety frameworks faster than the industry is converging on standards. AIUC's bet is that buyers - not regulators - will force the issue first.

Why Enterprises Are the Wedge

Kvist argues that the shift from research prototypes to production agents has inverted the risk calculus. "Banks, hospitals, governments, and militaries no longer decline to deploy AI because a model isn't smart enough," he said. "They decline because they've made commitments to their own customers about what a system will and won't do, and nobody can currently guarantee that."

That guarantee gap is acute in Asia. Financial institutions in Hong Kong and Singapore operate under strict data residency and fiduciary obligations. Healthcare providers in Japan face privacy regimes that make GDPR look permissive. Defence contractors across the region are navigating export controls on AI components while trying to deploy autonomous systems.

AIUC's model - third-party attestation that an agent has been stress-tested against a buyer-defined standard - maps onto procurement workflows that already exist. Enterprises buying SaaS products routinely demand SOC 2 Type II reports, penetration test results, and vendor risk questionnaires. Extending that expectation to agents is a short conceptual leap, even if the technical challenge is orders of magnitude harder.

The harder question is whether 5,000 test scenarios can keep pace with the adversarial creativity of jailbreakers, the subtlety of prompt injection attacks, and the emergent behaviours that appear only after agents interact with live data at scale. AIUC's answer is the consortium: a feedback loop that continuously surfaces new failure modes from the field.

The Audit-the-Auditor Problem

AIUC's reliance on AI agents to test AI agents introduces a recursion problem the startup acknowledges but hasn't fully resolved. If the testing agents themselves can be fooled or manipulated, the audit loses credibility. Kvist emphasised that humans verify the final output, but the volume of data - 5,000 tests producing 100-page reports - makes it unclear how much human oversight is substantive versus ceremonial.

The company hasn't disclosed the architecture of its testing agents, the models they're built on, or whether they're subject to the same AIUC-1 standard they're enforcing. That opacity may be necessary to prevent gaming, but it creates a trust bootstrapping problem: enterprises are being asked to rely on a black-box audit of a black-box system.

There's also the question of what certification actually prevents. A 100-page report that maps an agent's failure modes is useful for risk-aware buyers, but it doesn't stop a determined attacker from exploiting those weaknesses post-deployment. AIUC isn't offering runtime monitoring or kill switches - just pre-deployment transparency.

The Consortium as Moat

The list of investors and customers suggests AIUC has convinced a critical mass of the AI stack that certification will become mandatory. Ribbit Capital has deep ties to fintech; Emergence specialises in enterprise SaaS; Anthropic co-founder Ben Mann's involvement signals buy-in from at least one frontier lab.

Cursor and Harvey - both building agents that touch sensitive data - are early customers. Their participation indicates that agent builders see certification as a sales enabler, not just a compliance tax. If AIUC can establish AIUC-1 as the default standard before competitors emerge, the consortium itself becomes the moat: 250 enterprise buyers co-authoring the criteria makes it harder for a rival framework to gain traction.

But standards wars are rarely winner-take-all. SOC 2 coexists with ISO 27001, NIST, and a dozen sector-specific frameworks. AIUC will likely face competition from incumbents like BSI or emerging AI-native auditors. The question is whether first-mover advantage and the consortium lock-in are enough to establish AIUC-1 as the baseline, or whether enterprises will demand multiple overlapping certifications - creating audit fatigue before the market matures.

What Certification Doesn't Solve

Third-party audits can surface known risks and verify that an agent behaves predictably under test conditions. They can't predict emergent behaviour in production, adversarial attacks that post-date the audit, or the consequences of agents interacting with other agents in ways the test suite didn't anticipate.

The analogy to SOC 2 is instructive but imperfect. Cybersecurity audits assess controls - firewalls, encryption, access policies - that are static or change slowly. AI agents are stochastic, fine-tuned continuously, and often retrained on user data. A certification valid in September may be obsolete by November if the vendor ships a new model version.

AIUC hasn't disclosed whether AIUC-1 certification expires, how often re-audits are required, or what happens if an agent misbehaves in the field after passing. Those details will determine whether the standard becomes a living process or a one-time checkbox.

There's also the geopolitical dimension. A US-based auditor with ties to Anthropic and OpenAI may struggle to gain traction in China, where state-backed labs operate under different safety definitions. Even within allied markets, regulators in the EU and Asia may resist deferring to a private-sector standard without government oversight.

The Wider Safety Debate

AIUC's launch lands in the middle of a broader reckoning over who evaluates AI safety and what "safe" means. Frontier labs have historically self-assessed, publishing red-team results and safety cards with varying levels of rigour. External researchers like METR, Apollo, and the UK AI Safety Institute have begun conducting independent evaluations, but those efforts remain ad hoc and under-resourced.

Amodei's call for embedded evaluators at frontier labs reflects a growing consensus that self-regulation isn't sufficient as models approach AGI-level capabilities. AIUC's enterprise focus is narrower - certifying deployed agents, not research prototypes - but the accountability structure is similar: independent verification, transparent reporting, and consequences for non-compliance.

The difference is that AIUC is market-driven. Enterprises pay for audits because their customers demand proof of safety. Frontier labs, by contrast, face no comparable commercial pressure until governments impose liability regimes or insurance markets price AI risk.

If AIUC succeeds in making certification a precondition for enterprise sales, it could create a template for upstream regulation. Governments may eventually require frontier labs to undergo AIUC-1-style audits before releasing new models, much as medical devices must pass clinical trials before commercialisation.

The Path Forward

AIUC's $55 million war chest gives it runway to refine the standard, expand the consortium, and build out the testing infrastructure. The immediate challenge is execution: can the startup audit agents fast enough to keep pace with product release cycles, and can it maintain rigour as test volume scales?

The longer-term question is whether third-party certification becomes a pillar of AI governance or a footnote. If enterprises adopt AIUC-1 widely, regulators may codify it into law. If they don't - if buyers continue to deploy agents based on vendor promises and hope - AIUC will remain a niche service for the risk-averse.

Kvist and Dattani are betting that the gap between AI capability and AI control is wide enough, and painful enough, that buyers will pay to close it. The consortium of 250 enterprise leaders suggests they're not alone in that assessment. Whether the rest of the industry follows will determine whether AIUC-1 becomes the SOC 2 of AI - or another standard that never escaped the PowerPoint deck.