Sofi
Chat, calling and games app with a real-time voice agent
Sofi is a Hinglish chat app that talks back. You text it, send voice notes, play games with it, and now call it. Four surfaces built solo: a Flutter app on both stores, a Go backend running the model and memory pipeline, a Python voice agent for real-time calling, and a Next.js admin panel that changes prompts, models and pricing in production without a deploy.
The Problem
A conversational product has to feel personal and stay affordable at the same time. It must remember past conversations without blowing up context windows, handle voice naturally, survive one model provider going down, and monetise on Indian payment rails, all operable by one engineer on a startup budget.
The Solution
A Go and Gin backend orchestrates a configurable model chain with cross-provider fallback and prompt caching, backed by pgvector long-term memory that an extraction pass distils from conversations and a merge pass deduplicates. Replies are split into multiple bubbles so they land like a person texting rather than a chatbot. Voice runs end to end on one cloned identity: AssemblyAI streaming speech-to-text, a Cartesia voice for voice notes, and a LiveKit agent so the same voice answers the phone. Firebase Remote Config is the control plane, so prompts, pricing, model chains and notification copy are all editable live from the admin panel.
Architecture
- Flutter app on Android and iOS: chat with voice notes, switchable moods, in-app games and a photo gallery
- Go and Gin backend on AWS, with EC2 behind Caddy for TLS, RDS Postgres with pgvector, and Redis
- Model orchestration: a configurable chain with cross-provider fallback, prompt caching, session summarisation and a multi-bubble reply pipeline
- Memory: embedding-backed long-term extraction and merging, retrieved under a strict prompt token budget
- Voice: AssemblyAI streaming speech-to-text, a cloned Cartesia voice, and a Python LiveKit agent for real-time calls
- Payments: Razorpay UPI AutoPay subscriptions, one-off unlocks and prepaid wallets, metered server-side and billed by the second
- Next.js admin panel driving prompts, pricing, model chains and notification copy through Firebase Remote Config, with no deploy required
Challenges & How I Solved Them
Half of all first subscription debits were failing
Roughly 48% of first debits failed with nothing obviously wrong in the logs. Pairing production data against the payment provider's API exposed a race between mandate confirmation and the mandatory 24-hour pre-debit notification: the charge was being attempted before the notice window closed. A two-line timing change took first-debit failures to zero.
560 of 596 push notifications going nowhere
Almost every push was silently failing on dead FCM tokens, because tokens were only ever synced at onboarding and never refreshed, and a string-comparison bug in the delivery check was reporting the failures as successes.
97% of model spend went to notifications
Cost attribution showed that generating push copy, not talking to users, was consuming almost all model spend. It stayed invisible until per-feature spend was measured rather than assumed.
My Contributions
- Entire system: Flutter app, Go backend, Python voice agent, Next.js admin panel and AWS infrastructure
- Model orchestration with fallback chains, prompt caching, vector memory and multi-bubble replies
- End-to-end voice: streaming speech-to-text, cloned text-to-speech and a real-time LiveKit calling agent
- Razorpay UPI AutoPay subscriptions, one-off unlocks and per-second metered wallets
- Production instrumentation that turned payment, delivery and cost failures into measurable bugs
Impact
- Live on Google Play and the App Store, with a working subscription business on UPI AutoPay
- First-subscription debit failures cut from roughly 48% to zero (as of Sept 2026)
- 1,137 users and 27,154 messages analysed to drive product decisions (as of Sept 2026)
- Prompts, models and pricing all changeable in production without a deploy
Lessons Learned
The bugs that mattered most were invisible until measured. Payment failures, dead push tokens and runaway model spend all looked like product problems from the outside, and each turned out to be a small, findable defect once production data was paired with the provider's own API.