# Testing the food scanner with recorded scans, for free

Canonical page: https://mouve.it/blog/testing-the-food-scanner-with-recorded-scans
By Mouve team, 5 Nov 2026. 2 min read. Tags: engineering.

A wrong scan is the most expensive bug in a food app. Someone who weighed their dinner and watched us miss it stops trusting the numbers. So we wanted a way to know that a change doesn't make an already-fixed case wrong again, without paying for an AI call every time we check.

## How does it work?

Each time a scan runs, the AI models answer a few questions: what's on the plate, what the label says, what the barcode reads. We keep those answers, along with what the meal or product really was. To test a change, we feed the stored answers back through the new code and see what it would have picked.

The question is narrow: given what the models already said, does the new code choose better? It costs nothing, takes about a minute, gives the same result every time, and runs on every change before it's merged.

## What does it measure?

Everything that happens after the models have answered:

- how we look things up in the food database;
- how the results are ranked;
- how several readings of one item are combined into one answer, and how we judge whether it's plausible;
- when a scan should ask the slower model for a second look;
- how duplicates are merged, and making sure a later reading never replaces a better earlier one.

Today the set holds 640 cases from real scan sessions where we know the truth.

## What can't it see?

- **A change to the AI model or its instructions.** Those change the stored answers themselves, so they need the slower test that calls the models for real.
- **Your own history,** like products you log often. The replay behaves like a brand new account.
- **Second looks recorded under older instructions.**
- **Steps that failed when the scan originally ran.**

We write these limits next to the scores, because a passing replay tells you less than it seems to if you forget them.

## Why does free matter?

A check that costs money gets run less often. Because this one is free and fast, it runs on every change to the scanner, and a regression shows up on the pull request with the exact case that changed, not in a user report a week later.