Skip to content
Adarsh Soni
All projects

Personal project · real-time AI

Hotel Concierge Voice AI

A real-time hotel voice concierge: LiveKit WebRTC, Silero VAD, Deepgram Nova-3, a Groq-hosted LLM and Deepgram Aura-2, streaming at every stage.

Built for a sub-650 ms p50 response target

  • Python
  • LiveKit
  • Deepgram
  • Groq
Try it live

Problem

A voice concierge guests talk to from the browser. Voice only works if the reply starts quickly.

Approach

  • Mic audio goes over LiveKit WebRTC to a Python agent.
  • Silero VAD detects speech, and LiveKit's turn detector judges when the visitor has finished, waiting longer only when a sentence sounds unfinished.
  • Deepgram Nova-3 transcribes, Qwen3.8 27B on Groq writes the reply and Deepgram Aura-2 speaks it.
  • Every stage streams, so the reply starts before the model finishes.

Architecture

Voice pipeline of the Hotel Concierge Voice AIEight steps, top to bottom: the browser microphone, LiveKit WebRTC, Silero VAD, LiveKit turn detector, Deepgram Nova-3, an LLM on Groq, Deepgram Aura-2, and the audio played back in the browser. Streaming at every stage.streaming at every stageBrowser micThe visitor speaksLiveKit WebRTCAudio streams to a Python agentSilero VADVoice activity detectionLiveKit turn detectorJudges when the visitor has finishedDeepgram Nova-3Speech to textLLM on GroqQwen3.8 27B writes the replyDeepgram Aura-2Text to speechAudio backThe reply plays in the browser
Every stage streams, so the reply starts before the model finishes.

Result

  • Built for a sub-650 ms p50 response target.
  • The same design powers the Talk to my AI assistant on this site.
  • Phase 2 plans telephony over SIP.

Hiring a backend or founding engineer? Let’s talk.

Email me

↑ ↓ to moveEnter to runCtrl K anywhere