The production of soft ice cream in fast food is a huge business. This delicacy is produced by very sophisticated machines. They are installed in almost every McDonald's and other restaurants under exclusive contracts with the manufacturer. The price of such a car is $ 18 thousand. And they break down almost every month. Service maintenance also pours into a serious amount every month - and it is allowed to be carried out only by an official distributor.
This Wired story is about two hackers who hacked into the insides of a Taylor C602 ice cream machine, decrypted the undocumented service code (5-2-3-1-then 16 more digits) and made a self-diagnostic device. For this, the manufacturer arranged a real hunt for the guys.
Secret code
Taylor ice cream machines are installed in McDonald's restaurants around the world (13 thousand in the USA). They have a reputation for being extremely fragile and unreliable, as well as being difficult to maintain and completely unfriendly to the user.
Due to the many dubious engineering decisions, they fail so often that they have become a real meme on social networks .
But the most surprising thing is that despite constant breakdowns, McDonald's still buys these devices from food equipment giant Taylor for about $ 18,000, and the manufacturer still keeps the inside of the machines a secret from the buyer. What's more, Taylor maintains a network of approved distributors who charge franchisees thousands of dollars a year for costly maintenance contracts, and call technicians come in and enter this secret passcode into customer devices.
This secret menu is part of a distinct business model that goes beyond the question of eligibility for renovation. According to hackers, this is real extortion :
- Selling a complex and fragile car to a franchisee.
- Don't explain why it breaks down all the time.
- Receive a share of distributors' profits from repairs.
This is a big cash flow considering the tens of thousands of installed ice cream machines. According to hackers, McDonald's is also involved in this scheme, insisting on loyalty to its longtime supplier (if you dare to install another ice cream machine other than Taylor, the company can terminate the lease of the restaurant, in accordance with the franchise agreement).
Kytch device
The two who dared to challenge the corporation are 34-year-old Jeremy O'Sullivan and 33-year-old Melissa Nelson.
Photo: Gabriela Hasburn
They started a little gadget called Kytch. It fits into your Taylor car and connects to Wi-Fi. In essence, he hacks into the machine and provides access to a secret menu for diagnostics. Kytch acts as a surveillance device inside the machine, intercepting and eavesdropping on messages between components and sending them to a friendlier user interface than the manufacturer. The device not only displays all hidden internal data, but saves logs and even offers troubleshooting solutions - all via the Internet or a mobile application.
Kytch device
Very convenient for the restaurant owner. He can save a lot of money if he himself starts troubleshooting some of the machine's problems.
But once O'Sullivan and Nelson began selling the hacking gadget, McDonald's sent all franchisees a warning that Kytch's device was "copying confidential information" from machines and could cause "serious human injury." This destroyed their business: now customers are simply afraid to buy a device, and old subscribers refuse it.
How it all happened
As soon as McDonald's and Taylor learned of Kytch's early success, a two-year Cold War began, which is now getting hot. At one point, Taylor even hired private detectives to procure these devices. Eventually, Taylor unveiled its own competing monitoring gadget with an internet connection. And McDonald's began to put pressure on franchisees by sending threatening letters.
After watching McDonald's and Taylor's efforts to destroy their business within five months of these letters, the boys have now launched a counter-attack. A pair of entrepreneurial hackers are planning to sue some McDonald's franchisees who they believe have colluded with Taylor by handing over their devices to Kytch and allowing them to reverse engineer in violation of the franchisee's agreement with Kytch. The lawsuit is likely to be only Kytch's first step in a future protracted legal battle against Taylor and McDonald's.
At the center of the conflict is the sophisticated Taylor C602, which is sometimes compared in complexity to an Italian sports car.
Taylor C602 Machine
The main characteristic of an ice cream machine is performance. Hundreds of sophisticated components of this machine are designed by engineers to achieve this goal. Like other similar devices, the Taylor C602 picks up liquid ingredients through a hopper, quickly freezes them in a rotating drum, removing tiny sheets of frozen mixture from the cold metal surface of the drum using scraper blades, repeatedly stirring the mixture to dissolve as much ice as possible. crystals, and then pushes it through the nozzle into a waiting cup or waffle cone.
But what makes the machine special are the two hoppers and two drums, each of which works independently with precise settings to produce both milkshakes and soft ice cream at the same time. To accelerate the flow of ice cream through the nozzle, a special pump is used here, which competitors do not have. During peak minutes, the machine is capable of delivering up to 10 portions of ice cream per minute, which is impossible in other machines.
Taylor's notoriously finicky and fragile ice cream machines are used by nearly every major fast food chain, including most McDonald's in the US and tens of thousands of other restaurants around the world.
And while other ice cream machines have to be disassembled and cleaned daily and any remaining contents discarded, Taylor machines use a daily “heat-treat” process designed to raise the temperature of the contents to 66 ° C, pasteurize for at least 30 minutes, and then freeze again in the cycle once a night, which is a real miracle of hygiene and cost savings.
But like Italian sports cars, these cars are just as moody, fragile and ridiculously overloaded with details. They work great as long as everything is 100% perfect. If something went wrong, it crashes.
All of these Taylor components must be disassembled, cleaned and lubricated every two weeks.... If you forget to insert at least one gasket back, the machine will fail. It is not always successful to assemble a machine the first time.
Every two weeks, all of Taylor's precisely engineered components must be disassembled and disinfected. Some parts should be thoroughly lubricated. These are at least two dozen rubber and plastic O-rings of various sizes. Forget one thing and the pump may fail or liquid ingredients may leak out of the machine. The technical manager of one McDonald's franchisee said that he has assembled Taylor ice cream machines more than a hundred times - and on the first try it worked no more than 10 times. These are very finicky cars.
The automatic overnight pasteurization process, instead of making life easier, has become the biggest problem: if there is too much or too little mixture of ingredients in the hopper in the machine, if it is accidentally turned off at the wrong time, or if any of the many other trivial errors occurs, then the four-hour pasteurization process will fail and produce some kind of standard, incomprehensible error message. This means that the machine will not run until it has repeated all four hours of heating and freezing, often during the peak hours of ice cream sales.
Unsurprisingly, machines fail frequently.
According to the statistics of the website McBroken.com, at any given time in the past two months, anywhere from 5% to 16% of all American McDonald's cannot sell ice cream.
From Apple to John Deere, many companies are resisting the freedom movement to repair devices by anyone with open documentation: see US Farmers Hailing to Break into Tractors Just to Fix Them . But few of these products need to be repaired as often as McDonald's ice cream machines. The company itself admits the problem: "We have a joke about our ice cream machines, but we are afraid that this joke does not work":
This tweet received 29,000 likes. Everyone knows that McDonald's ice cream machines often break down . Hundreds of people joke that they are willing to spend their coveted grants on repairing an ice cream machine in their restaurant.
Inventor engineers
O'Sullivan and Nelson met at Bucknell University (Pennsylvania), started dating in the late 2000s, and then both worked as economists at different firms: Deloitte and Ernst & Young. But the job was extremely boring. A few years later, they brainstormed and drew attention to a new global American craze for frozen yogurt.
This business was essentially built around a variety of ice cream machines - mostly Taylor machines. It worked like making ice cream, only without a pasteurization step that kills the bacteria in the yogurt. But ice cream sellers paid for hundreds of square meters of real estate rent and employees' salaries. The economic engineers decided that the frozen yogurt industry was ripe for revolutionary automation.
So they began developing a device they called the Frobot: a bulky cabinet, like a Taylor machine cabinet, with its own touchscreen interface the size of a TV and a card reader. In other words, they set out to make an automatic frozen yogurt shop. The guys hoped to install Frobot in public places where it would generate passive income.
Frobot
It took them three years to build the first Frobot prototype using a privately advertised Taylor machine. And it took some help from outside engineers who were hired under the contract. After a trial run at a local medical school cafeteria, Nelson and O'Sullivan put Frobot in Washington, DC, in a coworking space, and the business started. The couple quit their jobs and moved to San Francisco to pursue their startup full-time. The next-generation Frobot was staged at the Palais des Beaux-Arts , where it started bringing in $ 500 a day.
But now the inventors had a problem: they wanted their machine to be completely autonomous and run with minimal human intervention. National Sanitation Foundation regulations required periodic monitoring of product temperatures. This temperature data was hidden in Taylor's car, where they could not access it. However, they accidentally saw a technician from a certified distributor enter the 5-2-3-1 secret code on the machine, which was not mentioned anywhere in the user manual.
Breaking into
Around the same time, at a seminar, O'Sullivan met the famous hacker Andrew 'Bunny' Huang, who was the first to hack the Xbox 20 years ago . O'Sullivan and Nelson asked him to conduct a technical examination of the device: is it possible to receive temperature data from the machine and send it in real time to a remote interface?
At the end of 2016, O'Sullivan and his contract engineer moved to Shenzhen to fully immerse themselves in the reverse engineering of the machine. They rented a room in a warehouse, above one of the city's large electronics markets, in order to quickly purchase the necessary components there in the course of work.
Andrew Huang figured out the machine and explained to the guys that the Taylor machine, like many devices in the food industry, is essentially an ancient technology that has not fundamentally changed in 50 years. This is really the Middle Ages by the standards of modern electronics.
Nonetheless, O'Sullivan and the engineer were pretty advanced and four months later they designed the device that would become Kytch, a hacking device that removes data from a machine and allows it to meet US sanitary requirements.
At the time, Taylor supported their work. The company even sent them its ice cream machine in Shenzhen. After all, Frobot represented not so much a competitor for Taylor as a new source of sales, or even a whole new automated marketplace for its machines.
In 2017, Frobot slots began to gain popularity. Tesla has installed two in the factory canteen. Six more were installed at the large Levi's Stadium, and the owners of the football team invested in the company of Nelson and O'Sullivan. Taylor was extremely pleased.
But over time, businessmen took notice of the persistent mysterious crashes and error messages that all of Taylor's customers complained about. The machines gave messages that the freezer mixture was too cold. Or too hot. Or too viscous. They soon found themselves constantly driving to the stadium to help staff troubleshoot and restore Taylor machines inside their vending machines. Andrew Huang was right: zhetio is an extremely unreliable and ancient technology from the Middle Ages.
The problems went so far that they had to install surveillance cameras in Frobot cabinets. They filmed a video of what might go wrong inside. It soon became clear that the business was not automated at all. In fact, no one at Levi's or Tesla was capable of supporting automata without the constant hands-on help of the Frobot founders. And the problem was the extremely unreliable Taylor machine.
It began to dawn on O'Sullivan and Nelson that they would have to turn around. And they had already unwittingly created a prototype for another product that offered a solution to this problem. Over the next year, they honed a small computer component called Frobot that listened to data from the Taylor ice cream machines to see and control all machine settings, including automatic 5-2-3-1 code bypass to access the service menu, software interface for diagnostics and troubleshooting. numerous failures. The device is based on the Raspberry Pi.
Kytch
New company
In April 2019, they relaunched their company, this time called Kytch . Nelson toured the suburbs of San Francisco looking for restaurants that use Taylor machines. All of these restaurants were offered a six-month free trial, followed by a $ 10 a month subscription. The first customers were found in Burger King and Super Duper Burgers. Gradually, businessmen reached out to potentially the largest customer who used the most sophisticated version of an ice cream machine: McDonald's.
Most restaurant owners have never had access or even heard of a service menuin car. The manufacturer deliberately kept it secret in order to prevent the owners of the cars from carrying out diagnostics on their own. Meanwhile, many McDonald's owners were paying thousands of dollars a month to Taylor distributors for service. Often the maintenance consisted of simple settings that were made through a secret menu. Therefore, the developers added a feature to Kytch called Kytch Assist that could automatically detect some of the common machine errors when they happened, and automatically tune these hidden variables to prevent some crashes before they happened. The device is installed independently in 20 minutes using a screwdriver.
One franchisee reported that a car in one of his restaurants broke down almost every week due to a mysterious failure during the pasteurization cycle. He studied the assembly of the car thoroughly over and over again, to no avail. The Kytch installation immediately showed that an overzealous employee was putting too much mixture into one of the silos. Another technician said Kytch “easily saves thousands of dollars a month,” even though the company then doubled prices and added $ 250 per activation.
Even if Kytch does not fix the problem, it will send an email notification to the owner describing the problem. This is very valuable information that allows you to quickly fix the car.
Word spread among McDonald's franchisees very quickly - and Kytch's sales began to double every quarter. O'Sullivan and Nelson hired a third staff member. By the fall of 2020, more than 500 devices have been installed and have infiltrated Taylor ice cream machines around the world. But the ice cream empire was already preparing a retaliatory strike.
Within two days of Kytch's launch in late April 2019, O'Sullivan and Nelson noticed that a top executive from Taylor had placed an order for the device. The guys were on their guard, canceled the order and returned the money to him.
Taylor starts a war
A couple of months later, they saw another bizarre order, this time from an employee of a third-party law firm that partnered with Taylor. During the following months, suspicious orders continued. The owners canceled all of these orders. Taylor shouldn't have been allowed to get the device.
One day an order came from an unknown person. A LinkedIn search identified him as an employee of Marksmen, a private detective firm that specializes in intellectual property matters. Apparently, Taylor hired private detectives: they used fake names to get their hands on a device that hacked into their cars.
Around the same time, Nelson and O'Sullivan received an email from Taylor asking them to stop using their brand at food trade shows. The days of friendship with Taylor are officially over.
The war began on November 2, 2020. One of the Kytch salespeople forwarded an email to Nelson and O'Sullivan, which McDonald's sent to all franchisees. It warns that a Kytch installation will void the Taylor machine warranties. In addition, the letter said that Kytch “gains full access to all equipment controllers and confidential data,” that this “poses a potential very serious safety risk for personnel or technicians trying to clean or repair the machine,” and that this could result to "serious human injury". McDonald's strongly recommended that you remove the Kytch device from all machines and stop using it.
The next day, McDonald's announced a new machine called the Taylor Shake Sundae Connectivity, which essentially duplicates many of Kytch's functions. The company again reminded all franchisees to remove and not use a third-party gadget.
Over the next months, McDonald's restaurant owners canceled hundreds of subscriptions, trials, and Kytch install commitments. It became impossible to find new clients. Kytch's only salesman quit.
Taylor now categorically states that it did not and did not intend to copy the Kytch device. According to them, a similar device has been developed over the years. None of the franchisees who spoke to Wired have ever seen or even heard of such a device.
After the bomb that McDonald's and Taylor dropped on the startup, Nelson and O'Sullivan concluded that somehow the companies had got their hands on the Kytch device. They investigated and found double agent Tyler Gamble, a friendly top manager from the largest McDonald's franchisee who confessed his love to Kytch and promised mountains of gold, was personally friends with the founders of the startup, and handed the device to corporations behind their back, although this was prohibited by the terms use. The friend committed cold-blooded betrayal and effectively killed the startup.
But now Nelson and O'Sullivan are planning revenge and preparing a legal action. It will be based on the fact that Gamble and some other users violated user agreement when they gave Taylor their Kytch devices in an attempt to curry favor with McDonald's and its corporate allies.
The lawsuit was filed on May 10, 2021 against Taylor and his distributor TFG on charges of stealing trade secrets.
At the same time, Taylor claims that she never had a Kytch device. Regardless of how the legal conflict unfolds, old Kytch technical advisor hacker Bunny Huang argues that McDonald's and Taylor's efforts to crush the tiny startup are a form of recognition:
« , , -, — , . — , Kytch , . , , ».
As for Nelson and O'Sullivan, they have no illusions. O'Sullivan admits that he views this Wired article as a posthumous description of his company after being killed by the fast food superpowers. “You’re kind of writing our obituary,” O'Sullivan told the article's author, Andy Greenberg.
But the battle is not really over yet. The company is still alive.
The tiny startup continues to fight a brutal battle. O'Sullivan once again reminds of what the battle is going on - around such a trivial thing as a cup of ice cream : "We want the world to know about this, because it is ... I mean, this is ice cream!" He says irritably.
O'Sullivan walks past a McDonald's with a broken ice cream machine and says grimly, “This car. The guys haven't found the secret code yet ... "