AI receipt recognition: what it can do, what it cannot, and why guessing is dangerous

In short: AI receipt recognition automatically finds the amount, date, merchant and often the category on a photographed receipt, so you do not have to type them in. It is very reliable on clean, printed receipts and much less reliable on faded, creased or unusual ones. The danger starts when an app guesses uncertain values instead of leaving them blank, because wrong numbers then slip into your books unnoticed.

If you capture receipts with your phone, you do not want to type every one of them by hand. That is what AI receipt recognition is for: the software looks at the photo, locates the relevant fields and fills them in. It looks like magic, but it is a chain of text recognition, pattern matching and language models, and every link in that chain can fail.

This article explains what machine learning does with receipts today, where its limits are, and how to spot a recognition system you can trust. The key point up front: the best AI is not the one that always produces an answer, but the one that knows when it does not know.

What makes AI receipt recognition different from classic OCR?

Classic OCR turns pixels into letters; AI receipt recognition also works out what those letters mean. OCR (optical character recognition) gives you the text of a till receipt as one long string: "Coop Bahnhofstr Total CHF 47.85 VAT 2.6% ...". That is not yet an amount or a date, just text. The article on reading receipts automatically explains the mechanics in detail.

The AI layer comes next. It answers questions like: which of the five numbers on this receipt is the final total? Is "12.03.26" the 12th of March or the 3rd of December? Is the merchant "Migros", or is that only the name of the shopping centre? Models are trained on very large numbers of receipts until they know the typical layouts: total at the bottom, date at the top or bottom, merchant in the header, VAT in its own block.

The difference matters in practice. Plain OCR is enough if you only want searchable receipts. As soon as you need structured data, for example a CSV list for your accounting tool, you need the interpretation as well.

What can AI do reliably with receipts?

On legible standard receipts, good AI receipt recognition gets the amount, date and merchant right in the large majority of cases. That applies in particular to:

Till receipts from retailers with a clear "Total" line
Restaurant bills with date, amount and a tip line
Fuel receipts, parking tickets, public transport tickets
PDF invoices with machine-generated text
Receipts in several currencies, as long as the currency code is printed

The category often works well too, because the merchant name gives a lot away: a pharmacy receipt goes to health, a hardware store receipt to materials. If you want a sensible set of groups, read receipt categories that make sense.

A realistic example: you photograph a bakery receipt for CHF 23.40. The app returns merchant "Bäckerei Müller", date "04.09.2026", amount "CHF 23.40", category "Meals". You glance at it, tap save, done. That is the normal case, and it saves about a minute of typing per receipt.

Where does machine learning fail on receipts?

AI fails where a human would also need a second look: poor image quality, unusual layouts and ambiguous information. The typical error sources:

Faded thermal paper: when the digits have almost no contrast, an 8 becomes a 3, or 47.85 becomes 41.85.
Creases and shadows: a fold through the total line can cut the amount in half; the shadow of your phone can hide the date.
Several amounts: subtotal, total, cash given, change. On a receipt with "Cash CHF 50.00", a weak system happily picks the 50.
Date formats: "03/04/26" is 3 April in Switzerland and 4 March in the United States.
Handwriting: handwritten receipts are read unreliably, especially the amounts.
Foreign receipts and separators: a thousands separator written as a full stop instead of a comma easily produces a factor-of-1000 error.

How high the hit rate really is, and how to check it, is covered in how accurate is receipt OCR?. The biggest lever, by the way, is not the model but the photo: a flat, well-lit, complete shot of the receipt halves the error rate.

Why is guessing dangerous in receipt recognition?

Guessing is dangerous because a confidently wrong value goes unnoticed, while an empty field does not. An app that always returns something looks better at first glance. In reality it moves the work to the end of the process, where it is most expensive.

Imagine you capture 400 receipts a year. The recognition gets 3 percent of the amounts wrong and, in those cases, guesses a plausible-looking value. That is twelve wrong amounts that look like right ones. You do not catch them while capturing, but only when your accountant asks why the expenses do not match the bank statement, or never. A single transposition from CHF 128.50 to CHF 182.50 happens quickly and costs half an hour of searching later.

The opposite approach is more honest: whatever the recognition cannot read with confidence stays empty. You see an empty amount field immediately and type the value in five seconds, while you still have the receipt in your hand. Belego works exactly this way: the app never guesses. Amount, date, merchant and category are read automatically, uncertain fields are left blank, and everything can be overridden by hand. That is less spectacular than a system claiming 100 percent, but in bookkeeping the number matters, not the impression.

A second point about guessing: language models tend to "complete" missing information. If the year is missing on a receipt, a guessing system happily adds the current one. For a December receipt scanned in January that is simply wrong, and in a tax year it makes a difference.

What should you look for in an AI receipt scanner?

Pay less attention to advertised accuracy figures and more to how the app handles uncertainty. This checklist helps when choosing:

Blank instead of guessed: does the app show uncertain fields as empty, or does it always fill them?
Everything editable: can you correct every recognised value with a tap?
Original preserved: is the receipt stored as a PDF so you can check it later? The article on converting receipts to PDF explains why that matters.
Check while capturing: does the app show the photo and the recognised values side by side, so you can compare them in a second?
Privacy: where do the photos go for recognition, and what stays stored there? With Belego, receipts stay on your device; only the automatic reading needs a brief internet connection.
Direct export: do the recognised values end up in a CSV you can load into Excel or your accounting tool?

And a tip that costs nothing: always check the amount and the date. The merchant name is less critical and a wrong category is fixed in a second, but amount and date determine what ends up in your tax return or with your accountant. Those two fields take three seconds to verify, and after that you can leave the rest to the recognition.

Frequently asked questions

Does AI receipt recognition need an internet connection?

Usually yes, at least for the automatic reading. The models that interpret receipts reliably are too large for a phone and run on a server. Capturing, searching and exporting can work offline, as they do in Belego. Ask an app provider which steps need the internet and what is transmitted in the process.

Can AI read handwritten receipts?

Only to a limited extent. AI receipt recognition reads printed receipts very well, but on handwriting the hit rate drops sharply, especially for amounts and dates. With handwritten receipts, expect to enter amount and date yourself. What matters is that the app still stores the photo as a PDF so the original is preserved.

Does AI also recognise the VAT on a receipt?

On standard receipts with a clearly printed VAT block, often yes, but this is exactly where a check pays off. Several rates on one receipt, for example 8.1 percent and 2.6 percent, are easily mixed up. For receipts you need in a VAT return, verify the tax amount against the original rather than accepting it blindly.

Is AI receipt recognition accepted for tax purposes?

The tax authority cares about the receipt itself, not the method you used to capture it. A legible PDF of the original plus correct values in your list is normally enough. What counts is that the values are right and the original is preserved. A guessing system is therefore riskier than one that leaves uncertain fields blank.