Extracting a recipe from a Reel looks like a transcription problem. It isn't. Here are the five things that actually go wrong, with numbers from our own production database of 376 recipes — including the bug that published a pumpkin loaf containing no sugar.
A cooking video rarely states every quantity out loud, so extraction is part transcription and part inference. Then the numbers have to survive being stored, scaled to a different serving count and rendered — and that arithmetic is where recipes quietly break. In our database, a single rounding decision distorted at least one ingredient in 76% of recipes and zeroed an ingredient outright in 24 of them, without producing a single error or failed test.
A creator tips salt from their palm and says "season it". A blogger shows a bowl and says "add the flour". Nothing in the footage names a quantity, and no amount of transcription accuracy will recover a number that was never spoken.
So a recipe extracted from video is always part inference: the model reads the visual scale of what's poured, the ratios implied by the rest of the ingredients, and the conventions of the dish. That works surprisingly well for staples — a pasta sauce has known proportions — and poorly for anything unusual. It is also the honest reason every app in this category, ours included, will occasionally give you a quantity the creator never intended. The fix isn't a better transcript; it is being clear about what's inferred and easy to correct.
A video says "serves four" and shows two chicken breasts. To let someone cook the same recipe for six, the app has to store an amount that can be scaled — so quantities are held per serving and multiplied back up on display.
This is the correct design and it is also where the trouble starts, because it introduces a division into a pipeline that used to be pure text. Every ingredient now goes through total ÷ servings on the way in and × servings on the way out, and any precision lost between those two operations lands directly in someone's mixing bowl.
Our stored per-serving amount was rounded to one decimal place. It reads like a harmless formatting choice. It isn't, because the display multiplies it back up: the rounding error scales with the serving count.
An audit across all 335 recipes in the database at the time found:
The threshold is unforgiving: an ingredient breaks once total ÷ servings drops below about 0.5. Spices and leaveners in a recipe for eight are almost guaranteed to hit it. Nothing about this shows up as an error — the recipe still renders, the steps still read correctly, the photos are still right. Only the numbers lie, which is precisely the part the app exists to get right.
The fix was to store six significant figures instead of one decimal place, verified by round-tripping every serving-count combination, and to stop truncating the same value again in the web page's data attribute. Then the damaged rows had to be repaired: for the 24 emptied recipes we reconstructed 37 quantities from the surviving proportions and the wording of the steps. Those numbers are a reconstruction, not the original — which is its own small lesson about how much cheaper it is to not lose the data.
A Reel has a share URL, a short URL, a URL with tracking parameters and a URL with a trailing slash. Treat them as four recipes and you generate the same dish four times, pay for it four times, and hand the user a cookbook with four copies of one pasta.
Deduplication has to happen on a canonical form of the link, resolved before any generation starts — and it has to hold for two people importing the same viral video within seconds of each other, which is a concurrency problem rather than a string problem. Getting this right removed a meaningful slice of our generation cost, because the cheapest recipe to produce is the one you already have.
A recipe page can outlive its source. Creators delete posts, switch accounts to private, or a link gets shared with a stray space in it that turns a working URL into a 404. The written recipe still exists and is still useful; the video behind it may not.
Two consequences worth designing for. The extracted recipe has to stand on its own, because the source is not a dependency you control. And the failure has to be explained honestly — "this video is private or has been removed" is a different message from "something went wrong", and users can act on the first one.
The interesting problems in video-to-recipe are not in the model. Transcription and extraction are the parts that work. The failures live in the boring arithmetic afterwards — precision, scaling, canonical identity, missing sources — and none of them announce themselves. A recipe with a wrong quantity looks exactly like a recipe with a right one, which means the only way to find these is to audit the database against reality rather than wait for a bug report.