AI Meal Recognition

A review-first approach to capturing nutrition data using large language models as a proposing layer for local deterministic validation.

What this feature does

  • Proposes food names and weight estimates from photos or text descriptions.
  • Matches AI suggestions against the app's local database of foods.
  • Calculates nutrition totals locally using matched product data.
  • Provides a manual review interface to edit or reject every entry.

What it does NOT do

  • It does not provide medical-grade nutrition analysis.
  • It does not automatically 'know' the caloric density of a specific restaurant dish.
  • It does not silently save data without your explicit review and confirmation.

The Architecture: Local-First & BYOK

Train Libre uses a "Bring Your Own Key" (BYOK) model. You choose a provider and model; the app handles the orchestration. Your data stays local, and the AI is only called when you trigger a capture.

Recognition is treated as a noisy proposal. Once the AI returns food names and grams, the app runs a deterministic validation pass. It attempts to repair common errors and flags low-confidence matches before you see the result.

Algorithmic Invariants & Local Matching Pipeline

1
BYOK Privacy Invariant
Your API key is stored securely using the operating system's Keychain (iOS) or Keystore (Android) via flutter_secure_storage. Raw image bytes or text descriptions are sent directly to your chosen AI provider (OpenAI, Anthropic, Gemini, Mistral, xAI) via TLS 1.3. No intermediate Train Libre server exists. Personal user profiles, weight history, and targets never leave your device.
2
Local Deterministic Matching
The AI operates only as a proposing layer. Once the model outputs food names and raw gram estimates, Train Libre matches these proposals against the local SQLite database (compiled from Open Food Facts and wger) using a tokenized Jaro-Winkler fuzzy matching scoring engine (validation threshold Dv ≥ 0.82). If the match confidence is low, a local warning is flagged.
3
3-Pass Self-Repair Retry Loop
To counter structural LLM hallucinations, the app runs a local 3-pass self-repair routine: (1) Structured Extraction: forces JSON output using strict schemas; (2) Quantity Normalization: standardizes qualitative descriptors (e.g., 'a splash of milk', 'one medium slice') into metric grams; (3) Syntactic Repair: attempts regex-based extraction of malformed JSON arrays before initiating up to 3 automated localized retries.
4
Strict Nutrient Computation Ban
The AI is programmatically banned from calculating calories, macros, or nutritional totals (enforced via system instructions). All macro arithmetic is calculated 100% locally and offline on your device using verified food catalog densities. This prevents your health status from leaking to LLMs and ensures perfect mathematical consistency (1g Protein = 4 kcal, 1g Carb = 4 kcal, 1g Fat = 9 kcal).

Scientific & Technical Limitations

Research into computer-vision-based nutrition estimation highlights several fundamental hurdles that make 100% accuracy impossible for consumer apps:

The Volume Problem: A 2D photo lacks depth information. Studies show that without a reference object or multiple angles, volume error rates typically range from 10% to 30%.
Hidden Ingredients: AI cannot 'see' the oils, butter, or sugar used in preparation. A grilled breast and a sautéed one may look identical but differ significantly in caloric density.
Mixed Dishes: Ingredients in dishes like stir-fries or burritos are often occluded. If the rice is under the curry, the AI will likely underestimate the portion.

Practical Guidance

Treat AI capture as a friction-reduction tool, not a ground truth. Always use the review screen to adjust gram estimates and ensure the matched foods align with what you actually ate.

Evidence & further reading