The one dollar Tahoe: what the great AI disasters teach a small business
In December 2023, a customer opened the chat window on a Chevrolet dealership's website and, after a bit of creative prompting, got the AI assistant to agree to sell him a 2024 Chevy Tahoe for one dollar. The bot even added, in writing, that this was a legally binding offer with no takesies backsies. Screenshots went around the world within hours. The dealership did not, in the end, hand over a Tahoe for a dollar. But their chatbot became the most famous salesperson in America for a week, for all the wrong reasons.
That story gets told as a joke about AI. It is actually a story about scope. And if you are a small business thinking about automation, the great AI disasters of the last few years are the cheapest education you will ever get. Somebody else already paid for these lessons. All you have to do is read them.
The bot that made up a policy, and the court that made it stick
In early 2024, a Canadian tribunal heard a case that should be pinned above the desk of everyone deploying a chatbot. A passenger had asked Air Canada's website assistant about bereavement fares. The bot confidently described a refund policy that did not exist. When the airline refused to honour it, the passenger took them to a tribunal, and Air Canada argued, with a straight face, that the chatbot was effectively responsible for its own words.
The tribunal disagreed. The airline was ordered to pay. The principle it established is the one that matters for every business, at every size: your AI's word is your word. There is no legal category for "the bot said it, not us." If your automated assistant promises a customer something, you made that promise.
The lesson is not "never use a chatbot." Air Canada's mistake was letting a general-purpose AI improvise answers about policy instead of grounding it in the actual policy documents. A bot that can only answer from your real terms, your real prices, and your real availability cannot invent a refund policy, because it has nowhere to invent one from.
The chatbot that roasted its own company
In January 2024, a frustrated customer of the delivery company DPD discovered that its support chatbot would happily do things far outside its job description. Within a few messages he had it swearing cheerfully and composing a short poem about how unhelpful DPD was, which it delivered with genuine enthusiasm. DPD disabled the AI component the same day.
Again, funny. Again, a scope problem. The bot was connected to a large language model with, evidently, very little constraint on what it was allowed to talk about. A support agent does not need the full creative range of a poet. It needs to know where the parcel is.
When we build agents for clients, the boring technical phrase for the fix is "guardrails," but the practical version is a question: what is this agent allowed to do, and what happens when someone asks for anything else? If the answer to the second half is "who knows," you do not have an agent, you have a liability with a typing indicator.
Two hundred and sixty nuggets
McDonald's spent about three years testing AI voice ordering at drive-thrus in the United States before ending the trial in mid 2024. The decision followed a run of viral videos, the most famous of which showed the system cheerfully adding chicken nuggets to an order, and continuing to add them, past all reason, while two customers begged it to stop. The count reached 260.
The nugget incident is the purest example of the third lesson: automation without a stop condition is just chaos running at machine speed. A human cashier who mishears you once gets corrected once. An automated system that mishears you and has no graceful way to hand over to a person will compound the error with perfect confidence.
Every agent we deploy has a defined handoff. When the conversation goes somewhere the agent is not built for, it does not improvise. It brings in a human, and it does so early and gracefully. The handoff is not an admission of failure. It is the feature that makes the rest of the automation safe to run.
The algorithm that bought the wrong houses
Not all the disasters are chatbots. In 2021, Zillow shut down its house-flipping business after its pricing algorithm systematically overpaid for homes, a mistake that cost the company hundreds of millions of dollars and a quarter of its workforce. This was a sophisticated system, built by genuinely talented people, with more data about house prices than almost any organisation on earth.
The failure was not the model. The failure was what the model was asked to do: make the bet itself, at scale, faster than humans could review it. When the model drifted, there was no human checkpoint between the drift and the chequebook.
For a small business the translation is simple. Let AI do the admin. Do not let it make the bets. An agent that answers every enquiry at 22:00, qualifies the lead, and books the meeting is doing admin. An agent that decides your pricing, signs off your quotes, or negotiates your discounts is making bets. The first category pays for itself quietly every week. The second category is how you end up in a headline.
The pattern under all of it
Line the stories up and the plot is identical every time. Someone gave an AI system a wider brief than it could safely hold, skipped the guardrails, skipped the handoff, and let it run unsupervised in front of real customers or real money.
None of these failures happened because AI does not work. They happened because the rollout was ambitious where it should have been boring. The businesses quietly winning with AI right now are running the least impressive-sounding automations imaginable: the instant reply to a missed call, the after-hours answer on WhatsApp, the follow-up to the quote that went quiet. Small scope, real guardrails, human handoff, one thing at a time.
We wrote about the right order to do this in what to automate first, and the maths of answering fast in the five-minute window. The short version of both, and of this entire article, fits in one line:
Give AI the passenger seat first. It has to earn the wheel.
If you want help working out what the passenger seat looks like in your business, book a call. We promise not to sell you a Tahoe for a dollar.
Sources
Every incident above is a public, reported event with named parties and dates. Full citations, including the primary filings where they exist, are on our references page.
- The one dollar Tahoe. Chevrolet of Watsonville, California, December 2023. The prompt was engineered by software engineer Chris Bakke. GM Authority
- The invented refund policy. Moffatt v. Air Canada, 2024 BCCRT 149, British Columbia Civil Resolution Tribunal, decided 14 February 2024. The passenger was awarded CAD 812.02. American Bar Association analysis
- The chatbot poet. DPD, 18 January 2024. The exchange was posted by customer Ashley Beauchamp. TIME
- The 260 nuggets. McDonald's ended its IBM voice ordering trial in June 2024, roughly three years after it began and across about 100 US restaurants. Global News
- The wrong houses. Zillow Offers, closed November 2021. The USD 407.9 million inventory write-down and the 25% workforce reduction are stated in Zillow Group's FY2021 Form 10-K filed with the SEC.