A smart appliance can spot a fault and recommend a fix. But what happens when an AI also has to protect customer trust, resist a convincing fake instruction and close a deal under pressure? For appliance and smart home businesses, those are management questions as much as technical ones. Firmulate’s live experiment offers a way to watch them play out.
Get home appliances delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
A company-wide test, not a chat demo
Firmulate put frontier AI models in charge of the same small software company during its worst week. Each faced the same customers, crises and temptations. The experiment records decisions so they can be reviewed, and the live company makes its progress public at firmulate.com.
The stakes were concrete. All models spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal that their own analysis had earned. The finding was succinct: “Same diagnosis, same pitch — no signature.” Recognizing the right action did not guarantee carrying it through.
The clue was in the company’s own files
The decisive competitor weakness was buried two document references deep in the company’s files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. That detail points to a practical challenge for businesses: useful evidence may already exist in internal documents, but an AI has to find it and use it when a decision is on the line.
The test also included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” Refusal is reassuring, but the deal results show why a complete assessment also needs to examine follow-through.
More work did not mean a better finish
Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, but it placed last. The close was left on the table, and discipline slipped when it attempted writes into a locked department instead of escalating. The same weakness appeared, more weakly, in all four models.
In the final July 2026 Crucible League, gpt-5.6-sol scored 95, Kimi K3 93, Sonnet 5 88, Fable 5 77 and Opus 4.8 73. The do-nothing baseline scored 26; partial progress counts, while a single breach of trust caps the total. As the experiment puts it: “no amount of good work outweighs a breach of trust.” K3 ran without an effort parameter, using the API default, while the others ran at xhigh.
The live company has 13 synthetic employees and real money mechanics: burn of €105k/month against €2.3k MRR, a public cash countdown, 680+ self-learned playbook rules and versioned work every business day. A quiz built from 242 real, unedited management decisions lets readers guess which model made each call.
From watching to testing your own business
For a smart home company, the relevant scenarios might involve a product fault, a customer churn wave, a price increase, a competitor move or pressure to disclose information. Firmulate’s proposed enterprise pilot uses a read-only export of a company’s own business to run crisis scenarios and produce a board report with model rankings and weak points in its playbooks. Nothing writes back to real systems.
That creates a step between watching an AI handle a fictionalized business and deciding whether it is ready to touch workflows such as customer support, sales or forecasting. The experiment suggests that businesses should examine not just whether models identify a problem, but whether they act on their analysis, find evidence in company records and maintain boundaries under pressure.
Put your own playbooks to the test
Firmulate’s live experiment shows why smart home and appliance businesses may want to test AI against realistic company crises before relying on it in daily operations. To discuss a pilot using a read-only export of your business, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
NFL season / tailgating Picks
team gear
As an affiliate, we earn on qualifying purchases.
