Skip to content
Syed Agha Shah Haider
← Work index

02Case study

WhatTheFood

AI nutrition analysis, photo to macros

Year
2025
Role
Product engineering, end to end
Status
Live
Live
whatthefood.io ↗

Nutrition apps ask people to weigh food and search databases. WhatTheFood asks for a photograph. The engineering problem is not the model call - it is being honest about uncertainty, and being fast enough that people keep using it.

2,000+
Registered users
1,200
Paid subscribers
10,000+
Images analysed
60%
Free-to-paid conversion

The constraint

What made it hard

A vision model will confidently return "grilled chicken, 240 kcal" for a photograph it half understood. Shipping that number bare is how a nutrition product loses trust in one screenshot. The second problem is cost: image inference on every upload, for every user, is the line item that decides whether the unit economics work.

Approach

  1. 01

    Per-item confidence, surfaced to the user

    The pipeline returns each detected item with its own confidence score rather than one figure for the plate. Low-confidence items are marked and editable, so the user corrects the model instead of silently distrusting it.

  2. 02

    Serverless image processing

    Uploads are normalised and compressed at the edge before inference, cutting payload size and keeping the analysis path off the main application server entirely.

  3. 03

    Embeddable widgets

    The analysis surface is packaged as a third-party embeddable widget, so a gym or a meal-prep service can drop the scanner into their own site - a distribution channel that costs nothing per install.

  4. 04

    Metered billing that maps to cost

    Stripe subscription tiers are drawn along the same axis as the actual cost driver - analyses per month - so a heavy user is a profitable user rather than a margin leak.

Architecture

WhatTheFood image analysis pipelineUploadclientNormaliseresize · strip · hashCachekeyed on hashGemini visionstructured jsonScoringper-item confidenceResultcache hit - no inference, no costpayload shrunkbefore it leaveslow-confidence itemsmarked editableevery analysis is metered against the subscription tier before it reaches the model
Upload path. Normalisation happens before inference; results are cached against an image hash.

Specification

Model
Google Gemini vision, structured JSON output
Storage
Supabase Postgres + object storage
Auth
Supabase Auth, row-level security
Caching
Response cache keyed on normalised image hash
Billing
Stripe subscriptions, usage-tiered
Distribution
Embeddable third-party widget

Stack

  • Next.js 14
  • TypeScript
  • PostgreSQL
  • Supabase
  • Stripe
  • Google Gemini

Where it landed

  • 01

    Over 2,000 registered users and 1,200 paying subscribers.

  • 02

    More than 10,000 images analysed through the production pipeline.

  • 03

    Analytics dashboard and embeddable widgets shipped as part of the paid tier.

Next case study

03

Kinetic Rehabilitation

AI physiotherapy with in-browser pose detection

Read →

Something that has towork in production?

I take on contract engineering and full-time roles. Most useful first message: what you are building, what is currently in the way, and when you need it live.