Skip to content
Syed Agha Shah Haider
← Work index

03Case study

Kinetic Rehabilitation

AI physiotherapy with in-browser pose detection

Year
2025
Role
Full-stack, client-side ML integration
Status
Live
Live
kinetic-project-eight.vercel.app ↗

Remote physiotherapy has an obvious architecture - stream the patient’s camera to a server, run pose estimation there. That architecture also means storing video of people exercising in their bedrooms. Kinetic does the inference in the browser instead, and the video never leaves the machine.

0
Video frames sent to the server
Real-time
In-browser pose inference
Live
Therapist video sessions

The constraint

What made it hard

Pose estimation on a server is straightforward but drags in the entire weight of handling clinical video: transport, retention, consent, jurisdiction. For a physiotherapy tool, the video is the most sensitive artefact in the system and the least useful thing to keep.

Approach

  1. 01

    Inference at the edge of the edge

    MediaPipe and TensorFlow.js run pose detection inside the patient’s browser. Keypoints - not pixels - are the only thing that ever crosses the network.

  2. 02

    Scoring against a reference movement

    Each exercise carries a reference joint-angle envelope. The client scores the live skeleton against it per rep and surfaces the specific joint that fell out of range, rather than a generic pass or fail.

  3. 03

    Live sessions layered on top

    Therapist and patient can drop into a LiveKit session for supervised work, with the same scoring overlay visible to both - a call and a measurement device at once.

  4. 04

    Progress that outlives the session

    Only derived metrics persist to Supabase: rep counts, form scores, range-of-motion trend. The clinical record is small, portable and free of video.

Architecture

Kinetic Rehabilitation client-side inference boundarynetwork edgepatient deviceserverWebcamraw framesTensorFlow.jspose keypointsForm scoringvs reference anglesOverlaycorrective cueSupabasederived metricsLiveKitopt-in sessionrep count · form scorerange of motionvideo never crosses
The dashed boundary is the network edge. Video stays left of it; only scores cross.

Specification

Pose model
MediaPipe / TensorFlow.js, WebGL backend
Video
Never transmitted for the scored path
Live sessions
LiveKit WebRTC, opt-in
Persistence
Derived metrics only - Supabase Postgres
Feedback
Per-joint angle deviation against reference envelope

Stack

  • Next.js
  • Supabase
  • TensorFlow.js
  • MediaPipe
  • LiveKit

Where it landed

  • 01

    Real-time form correction with no clinical video in server-side storage.

  • 02

    Therapist-supervised sessions and self-guided sessions share one scoring engine.

  • 03

    Progress analytics built from lightweight derived metrics.

Next case study

04

Uploadery

Cloud file hosting with signed delivery

Read →

Something that has towork in production?

I take on contract engineering and full-time roles. Most useful first message: what you are building, what is currently in the way, and when you need it live.