Building Squared: How I Used Gemini Live API to Create an AI That Coaches You While You Speak

Hey everyone! I just submitted my project Squared to the Gemini Live Agent Challenge, and I wanted to share what I built.

The problem: Every presentation tool gives you feedback after the fact — record, upload, wait, repeat. That's like a GPS that tells you "you missed the turn" five minutes too late.

The solution: Squared is a real-time AI navigator for presentations. A coach evaluates your performance after the fact. A navigator rides with you, knows the road ahead, remembers the turns where you've gotten lost before, and guides you through them as they happen.

Key highlights:

Rehearsal Mode — The AI speaks to you in real time, tracking live metrics (pace, eye contact, posture, filler words) and remembering your weak spots across sessions.

Presentation Mode — On stage, the AI goes completely silent but surfaces a live HUD with micro-prompts and rescue text visible only to you.

Pattern Memory — Built with Gemini Embedding API + pgvector, it retains every rehearsal so each session builds on the last.

Dual-Agent Architecture — Two parallel Gemini Live sessions: one navigating your delivery, one monitoring audience engagement.

Built with Gemini 2.5 Flash Live API, MediaPipe, and Google Cloud. Running in full production — not a localhost demo.

Would love to hear your thoughts and feedback!

Links

Live Demo
https://squared-j2gx3ygtta-uc.a.run.app

Source Code
https://github.com/gaborishka/squared

Mac App
https://github.com/gaborishka/squared/releases/tag/v0.2.0

Technical Deep-Dive
https://ivanhabor.substack.com/p/building-squared-how-i-used-gemini

Devpost Submission
https://devpost.com/software/squared-ai-presentation-navigator