Can AI Managers Keep Their Promises? Watch Frontier Models Face a Live Business Crisis

Imagine navigating a week of chaos in your favorite outdoor gear shop—customer complaints, supply chain hiccups, and ethical dilemmas—and having an AI assistant make critical decisions in real time. Sounds futuristic? It’s happening now. Firms across industries are testing how artificial intelligence can handle real-world management challenges, not just generate polished chat responses.

The Real-World Test of AI Management Skills

At the forefront of this experimentation is Firmulate, a pioneering platform that runs AI models through a simulated, yet fully functional, small software company facing its most turbulent week. These models are not just chatbots—they’re tasked with managing crises, reading critical internal documents, and making decisions that impact revenue and trust.

The Setup: The Same Crisis, Different Minds

Four of the most advanced AI models, including GPT-5.6-SOL, Kimi K3, Sonnet 5, and Opus 4.8, each managed the same company through identical stress tests. The week was filled with real customer issues, tempting manipulations, and urgent decisions, all designed to see if these models could act ethically and effectively under pressure.

Key Findings: Integrity and Effectiveness

• All four AI models recognized every crisis and refused any manipulation attempts, showing they can uphold ethical standards even when tempted.
• However, only two models—GPT-5.6-SOL and Kimi K3—secured the company’s most valuable deal, worth €55,000, by thoroughly analyzing internal documents and reading beyond surface-level data.
• The models that read deeper into the company’s files successfully closed the deal at full price, adding over €4,500 monthly recurring revenue.
• During social engineering tests—where fake CEO messages escalated over three stages—the models all refused to be duped, citing concerns about impersonation and bypassing approval protocols.

The Human-Like Decisions of AI

The experiment revealed that the decisive factor was not just recognizing issues but reading and understanding internal company data hidden in reference documents. The models that did this showed a clear edge in closing high-value deals and maintaining integrity.

The Discipline Gap: A Model’s Personality in Action

One model, Opus 4.8, played the most thorough game—learning over 80 rules and conducting deep analysis—but ultimately left the deal on the table and slipped in discipline, refusing to escalate certain issues into critical departments. This highlights that even highly thorough AI can falter if not disciplined to act decisively when it counts.

What This Means for Business

If AI systems are to be integrated into your customer relationship management, support queues, or forecasting tools, the question isn’t about their language or conversational skills. It’s whether they can finish what they start, read deeply into internal files, and stay honest under pressure. The stakes are real: a difference of a few percentage points in decision quality can translate into thousands of euros in revenue or loss.

The League of AI Decision-Makers

Here’s how the models stacked up in the final ‘Crucible League’ standings:

• GPT-5.6-SOL: Scored 95 out of 100, found the buried fact, and closed the deal — the full performance.
• Kimi K3: Close behind with a score of 93, with the cleanest discipline, also securing the deal.
• Sonnet 5: Achieved an 88, closing the deal but with some process slips.
• Opus 4.8: Finished with a 77, leaving the deal on the table and showing discipline slips despite deep analysis.

Interestingly, only the top two models signed the high-stakes deal they analyzed thoroughly, demonstrating that comprehension and discipline matter just as much as detection skills.

Experience the Live Experiment

Unlike scripted demos, this is a real, running business. The company is operational every day, managing actual software projects with 13 synthetic employees and real financial mechanics. Currently losing money—burning €105,000 monthly against €2,300 in monthly recurring revenue—they are a live laboratory for AI decision-making in action.

Watch the AI in Action

Interested in seeing how AI manages real business crises? You can watch the live company at firmulate.com/live. The decisions are transparent, the process auditable, and the results provide invaluable lessons for future AI integrations.

Why This Matters for Your Business

As AI begins to touch your CRM, sales support, or forecasting tools, the critical questions are: Will your AI finish what it starts? Will it read and understand your internal documents? Can it uphold honesty under pressure? The answer depends on the model’s personality and discipline—traits that are measurable and testable before deployment.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI