Why a Receipt Scanner Should Never Let the AI Do the Math

Why is reading a receipt a different job from computing the split?

Reading is pattern recognition: find the item names, find the prices, keep them on the right rows. Computing is executing a procedure exactly: multiply, carry, sum, and get the same answer every time. Language models are built for the first. They predict the next piece of text from patterns in their training data, and a long column of digits is just more text to them. That helps explain how the same system can transcribe a receipt perfectly and then stumble on the sum of a column it just transcribed.

The research teams that build these models say so in their own papers. The OpenAI group that built the GSM8K math benchmark wrote that “our models frequently fail to accurately perform calculations,” that larger models make fewer arithmetic mistakes but “this remains a common source of errors,” and that to mitigate it they trained every model to use a calculator that “will override sampling” whenever the model chooses to invoke it. A 2021 study by Nogueira, Jiang, and Lin found that a small pretrained transformer (T5-220M) trained on 1,000 addition examples “fails to learn addition of five-digit numbers when using subwords,” that is, when numbers are split into multi-digit tokens such as “32.” The authors add that “with enough training data, models can learn the addition task regardless of the representation.” The limit they found at every scale is a different one: “regardless of the number of parameters and training examples, models cannot seem to learn addition rules that are independent of the length of the numbers seen during training.”

The deeper reason is compositional. Dziri and colleagues formalized tasks like long multiplication as graphs of small steps and showed that transformer accuracy “decreases to near zero as task complexity increases,” in part “due to error propagation”: one wrong intermediate digit poisons everything downstream. A restaurant split is exactly this kind of task. Subtotal feeds tax, tax and tip feed the total, the total feeds every per-person share. Get the service charge wrong in step two and all five shares in step five are wrong, each by a plausible-looking amount.

Sources: Cobbe et al., “Training Verifiers to Solve Math Word Problems,” arXiv (2021); Nogueira, Jiang & Lin, “Investigating the Limitations of Transformers with Simple Arithmetic Tasks,” arXiv (2021); Dziri et al., NeurIPS 36 (2023).

What does it look like when a language model does the bill math?

It looks right until you check. Random Walk’s September 2024 test is worth walking through step by step, because its arithmetic failures are ones a reader can catch with the checks below. The bill was a five-person restaurant tab of ₹13,545.00 before charges, with a service charge, a 14.5% VAT that applied only to alcohol, and two 2.5% goods-and-services taxes that applied only to food and non-alcoholic drinks, for a printed total of ₹15,562.00.

The post’s own verdict: the model “required multiple prompts and follow-up queries to grasp the logic of adding and subtracting taxes and service charges when removing an item and splitting each person’s bill by percentage.” Notice the shape of the errors in Steps 3 to 5. None of them is a misread digit. Each is a computation that used the wrong base, skipped a dependent line, or mixed a pre-tax numerator with a post-tax denominator. Those are arithmetic-procedure failures, and a clean table does not protect you from them. Not every failure in the post was arithmetic, though: in Step 6 the model listed items that were not on the bill, and elsewhere the post says “the LLM confused food and beverages” and that ChatGPT “mistakenly excluded the cost of the Adana Kebab.” Getting the table right once is not the same as keeping track of it across a long conversation.

Source: Random Walk, “The Story of a Bill: How Well Can AI Models Handle Real-World Math,” 2024-09-27. All quoted phrases and figures are from the post; the 87.04% sum is computed from the five percentages it prints.

Check 1: Do the shares add up to the bill total?

Add every person’s share. The sum must equal what the table actually pays, to the cent: the printed total, plus any tip added at the table. Any tool that fails this has either dropped money or invented it. One easy way to drop it is the one Random Walk’s model hit: dividing each person’s pre-tax items by the post-tax total. The shares then add to the subtotal’s share of the total, and the overhead is orphaned.

Here is the same mistake on an illustrative US check. The numbers are chosen for clean arithmetic, not taken from any receipt.

The right direction is the proportional method: one multiplier, and each person’s items times that number. This example was built so the multiplier comes out to exactly 1.28 and every share lands on a whole cent. On a real bill the multiplier is usually a long decimal, so each share has to be rounded, and the rounded shares may not add back to the total exactly. A well-built tool has to place those leftover cents on purpose, and Check 1 is how you find out whether it did. The full method, including why it beats an even split of tax and tip, is in how to split tax and tip in proportion. One honest limit: a sum that reconciles proves the arithmetic closed, not that the right items landed on the right people. A shared plate assigned to one diner leaves the total untouched. The sum check is necessary, not sufficient.

Check 2: Does removing an item lower every line that depends on it?

Take one item off the bill and watch what moves. The subtotal must fall by the item’s price, and every percentage line whose base included that item must fall with it: sales tax on a taxable item, a percentage tip, an automatic service charge. A line whose base did not include the item, such as an alcohol-only tax when a food item comes off, should stay put. This check is about repricing the bill, as when a dish is voided or was never on your order; a splitter that keeps the printed charges while you reassign an item among diners is doing a different job, and Check 2 does not condemn it. Random Walk’s model got the subtotal right and the rest wrong in both directions: it cut the alcohol-only VAT when a food item was removed, and later left the service charge at its original amount after the base it was charged on had shrunk.

On the illustrative check above, suppose B drops a $10 dish. The subtotal becomes $74.00, tax becomes $5.92, the 20% tip becomes $14.80, and the total becomes $94.72. A tool that only subtracts the $10 and leaves tax and tip frozen reports $97.52, which is $2.80 too high: $0.80 of stale tax and $2.00 of stale tip. The error is small on one dish and compounds when several items come off the bill before it is settled.

Check 3: Does each tax and fee line recompute from its own base?

Every percentage on a receipt has a base, and the bases differ. Sales tax applies to taxable items. A tip may be figured on the pre-tax subtotal. A delivery service fee may be a percentage of the food only. On the Indian bill in Random Walk’s test, one tax applied only to alcohol and two applied only to food and non-alcoholic drinks, and the model’s first recomputation scaled all of them with the whole subtotal. That is the base error: right percentage, wrong column.

The check is simple. For each percentage line, divide the printed amount by the base the tool used and confirm the rate matches the receipt. Check 3 confirms each line is consistent with a base and a rate. It cannot by itself show which items were in that base, since different item sets can share a subtotal, and it does not show the line was then allocated to the right people: an alcohol-only tax can be computed correctly and still be spread across the whole table, so the proportional split reconciles while quietly charging the water drinker for part of the bar tab. On a bill like that, compare a drinker’s share and a non-drinker’s share by hand. If a tool cannot tell you what base it used, you cannot tell which of those it did. The anatomy of each line, and which base it legally sits on, is in your restaurant receipt, explained.

Why do the research labs hand the arithmetic to a calculator?

Because the model’s reasoning is often fine and its arithmetic is not, and the two can be separated. The PAL paper from Carnegie Mellon, published at ICML 2023, had a model write its reasoning as a short Python program and let the Python interpreter produce the answer. On the GSM8K word-problem benchmark the program-aided model solved 72.0% of problems against 65.6% for the same model reasoning in plain text. The more telling result came from GSM-Hard, a copy of the benchmark with the numbers replaced by large ones: the text-reasoning model’s accuracy fell from 65.6% to about 20% (the paper’s text says 20.1%; its Table 1 prints 23.1%), a relative drop the text puts at “almost 70%.” The program-aided version stayed at about 61%: the text says it “remains stable at 61.5%, dropping by only 14.3%,” and Table 1 prints 61.2%.

The authors then asked whether the big numbers confused the reasoning or just the arithmetic. In 16 of the 25 cases they examined, the model’s written-out reasoning was “nearly identical” with small and large numbers, “indicating that the primary failure mode is the inability to perform arithmetic accurately.” Their design conclusion applies directly to a receipt splitter: “since PAL offloads the computation to the Python interpreter, any complex computation can be performed accurately given the correctly generated program.”

Meta’s Toolformer, published at NeurIPS 2023, reached the same conclusion from the other side. The team trained a 6.7-billion-parameter model to decide for itself when to call a calculator. On three arithmetic word-problem benchmarks, allowing those calls “more than doubles performance for all tasks,” lifting the model from 14.8%, 6.3%, and 15.0% to 40.4%, 29.4%, and 44.0%, and past the much larger GPT-3. The reason, in the authors’ words: “across all benchmarks, for 97.9% of all examples the model decides to ask the calculator tool for help.” That is a measurement under the paper’s setup, which used a decoding setting of k = 10 “to increase the disposition of our model to make use of” its tools, not a claim that it knew its own limits; the direction is still the point: the arithmetic went to the calculator.

Sources: Gao et al., “PAL: Program-aided Language Models,” ICML 2023 (PMLR 202); Schick et al., “Toolformer: Language Models Can Teach Themselves to Use Tools,” NeurIPS 36 (2023); Cobbe et al., arXiv (2021). The PAL paper’s prose and its Table 1 print different GSM-Hard figures (20.1% vs 23.1% for text reasoning, 61.5% vs 61.2% for PAL); both are shown above.

How can you tell which architecture a bill splitter uses?

Run the three checks, and if you can, read the plumbing. Some products publish it. A custom GPT in the ChatGPT store called “Split Bill Calculator, No Hallucinations” advertises that you can “simply upload your bill and Billy will split for you … accurately with no hallucinations.” Its page source shows how it earns the name: the GPT is wired to an external “Split Bill Action” at a separate domain, described by an OpenAPI 3.1.0 schema. The model’s job, per the schema, is to produce structured input, a list of people, the subtotal, the total, and a list of items with prices. The backend returns the shares. The schema describes a division of labor, with the chatbot handing the math to something else; it is not a published accuracy result, and it does not show that every answer goes through that path. We probed the endpoint: it answered a bare GET with HTTP 405 and an unauthenticated POST with HTTP 403, which shows a separate service exists and responds. It shows nothing about whether that service computes correctly. The three checks are the reader’s best test of that, though they cannot reveal architecture and cannot rule out a wrong assignment on their own.

The schema also shows what the reading layer can lose. Its item field is documented as “list of individual item and its total price. The total price is regardless of quantity,” so a line like “2 × margarita” is documented to arrive as one priced item; whether the count survives somewhere else, the schema does not say. If it does not, clean arithmetic downstream cannot recover a quantity the extraction flattened. That is the other half of the architecture: the deterministic layer is only as good as the structure it is handed.

Competitors are starting to sell the distinction outright. A Splitwise explainer published on split-the-bill.app in September 2026 names “deterministic calculation versus AI estimation” as “another real point of differentiation,” and describes its own flow as working “with no AI estimation—it’s deterministic math, the exact same calculation every time.” The same post says that some apps use AI OCR to “guess” receipt items, which “can make mistakes on crumpled receipts or small print,” and advises that “it’s always worth double-checking the total before finalizing the split, whichever app you choose.” That is a competitor’s pitch aimed at other apps, and its example is a reading error rather than a computation error, but the distinction it sells, reading a receipt versus computing from it, is the one this article is about.

Sources: Split Bill Calculator, No Hallucinations, ChatGPT GPT Store (page metadata and embedded Action schema; endpoint probed 2026-10-05); Split The Bill, “Splitwise: What It Is, How It Works, and Pricing,” published 2026-09-12, updated 2026-09-18.

Does deterministic code guarantee the right answer?

No. Deterministic means repeatable, not correct, and money has a classic trap even for ordinary code. David Goldberg’s 1991 ACM survey explains that the decimal number 0.1, “although it has a finite decimal representation, in binary it has an infinite repeating representation,” so in standard binary floating point it “is exactly representable by neither” of its two nearest neighbors. Goldberg’s survey is about floating point in general, not receipts. Our own inference for bill splitting: code that stores dollars as binary fractions can pick up tiny rounding errors, and those can surface as a split that misses the total by a cent. A common engineering defense, which is general practice and not a claim from Goldberg’s paper, is to keep amounts in whole cents and round on purpose.

This is why Check 1 earns its place even against a tool that uses no language model at all. A sum that is off by a rounding slip is a different bug from a sum that misses the total by 13%, but both are caught by the same test. It is also why the mental math route fails for a different reason again: as that article’s sources argue, working memory rather than tokenization is the bottleneck when a human tries to carry twenty line items, a tax rate, and a tip percentage at once.

Source for the quoted passages on 0.1: Goldberg, “What every computer scientist should know about floating-point arithmetic,” ACM Computing Surveys 23(1), 1991. The application to receipts and the whole-cents practice are the author’s, not Goldberg’s.