Zulqurnain Haider
Senior AI × Mobile Engineer — I ship intelligent products, not just apps.
Kuala Lumpur, Malaysia · Open to remote worldwide
A real LLM, running entirely on the phone.
No API keys. No server. No network at inference time. This is a runnable React Native demo — pick a small GGUF, load it once, then toggle airplane mode and it still answers, because the model lives in the app sandbox. The pattern behind “ship AI features without sending user data to the cloud.”
Measured on-device
Cold start
to first context, 1.5B Q4_K_M
Time to first token
on-device, no round-trip
Network egress
at inference time
Offline / airplane mode
works after first download
Representative figures from the live demo on a mid-tier Android device with GPU offload (1.5B Q4_K_M). Actual numbers vary by SoC, model size, and quant. Run the repo on your own hardware to benchmark.
The proof: offline, on camera
A 45-second proof: ask the app a question, turn on airplane mode, and it still answers — with the live latency line (cold-start ms, first-token ms, tok/s) on screen.
Loom walkthrough — 45 sec
Recording slot pending. The 45-second script (hook → offline toggle → live metrics → where to find me) is in the repo’s LOOM_SCRIPT.md (a longer 6-min version is also there). Drop the Loom embed iframe in this card once recorded.
Recording checklist (the money shot is the airplane-mode toggle)
- App running on a real phone via Expo Go (Android w/ GPU ideal).
- One small model already downloaded (Qwen2.5-1.5B Q4_K_M, ~1 GB) — no WiFi wait on camera.
- Phone screen mirrored so the device + metrics line are visible.
- Tap Generate, let tokens stream, then toggle airplane mode and Generate again.
- Open
src/llm/OnDeviceLLM.tsto highlightn_gpu_layers: 99+ the latency measurement. - Switch to a second model, load, generate, read the new tok/s live.
- Title: “Private LLM inside React Native — fully offline, sub-2s cold start.”
How it works (the 60-second version)
llama.cpp via llama.rn
Battle-tested native runtime, not a toy wrapper. Runs through JSI — no bridge serialization penalty.
INT4 / Q4_K_M quant
A 1.5B model drops to ~1 GB at minimal quality loss. Ships small, loads fast, fits mid-tier devices.
SoC layer sharding (n_gpu_layers)
Layers offload to the GPU/NPU when present; automatic CPU fallback on devices without it.
Prompt + KV-cache per context
The same completion() call accepts retrieved context — so context-only RAG stays on-device too.
Why engineering leaders should care
| Concern | On-device (this repo) | Cloud LLM API |
|---|---|---|
| PII leaving the device | Never | Always (prompt + context) |
| Latency floor (1B class) | 50–200 ms | + network round-trip |
| Serving cost | ~$0 (on-device) | per-token, recurring |
| Offline / spotty 4G | ✅ works | ❌ |
| Data localisation (GDPR, fintech) | ✅ by construction | needs extra controls |
Evaluating edge-AI on mobile?
Happy to walk through this in a technical read-through.