---

name: livekit description: >- Build real-time audio and video applications with LiveKit. Use when a user asks to implement WebRTC rooms, video conferencing, live streaming, screen sharing, or real-time communication using the LiveKit SDK. license: Apache-2.0 compatibility: "No special requirements" metadata: author: terminal-skills version: "1.1.0" category: data-ai repository: https://github.com/livekit/livekit tags: ["webrtc", "realtime", "voice", "video", "streaming"]

LiveKit — Real-Time Voice & Video Infrastructure

Overview

LiveKit is an open-source WebRTC platform for real-time voice and video: a self-hostable SFU server, client SDKs for web/mobile, server SDKs for room and token management, SIP telephony bridging, and the Agents framework for building voice AI. Checked against livekit-agents 1.8.4, livekit-server-sdk (Node) 2.19.1 and livekit-client 2.22.3 (October 2026).

Breaking change since the 1.0 Agents release: livekit.agents.voice_assistant.VoiceAssistant and fnc_ctx are gone. Voice agents are now built from an Agent (instructions + tools) run inside an AgentSession (STT/LLM/TTS/VAD pipeline), started with session.start(...).

Instructions

Voice Agent with LiveKit Agents Framework

# agent.py — AI voice agent using LiveKit Agents 1.x
from livekit.agents import Agent, AgentSession, JobContext, JobProcess, WorkerOptions, cli, function_tool
from livekit.plugins import deepgram, openai, elevenlabs, silero

def prewarm(proc: JobProcess):
    """Pre-load models at worker startup for faster first response."""
    proc.userdata["vad"] = silero.VAD.load()

class ClinicAssistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="You are a friendly scheduling assistant for a dental clinic. "
                         "Confirm the patient's name and desired date before booking.",
        )

    @function_tool
    async def book_appointment(self, patient_name: str, date_iso: str) -> str:
        """Book an appointment for the patient on the given ISO date."""
        return f"Booked {patient_name} for {date_iso}."

async def entrypoint(ctx: JobContext):
    """Called when a participant joins a LiveKit room."""
    await ctx.connect()

    session = AgentSession(
        vad=ctx.proc.userdata["vad"],
        stt=deepgram.STT(model="nova-3"),
        llm=openai.LLM(model="gpt-4o"),
        tts=elevenlabs.TTS(
            voice_id="pNInz6obpgDQGcFmaJgB",
            model="eleven_turbo_v2_5",
        ),
    )

    await session.start(agent=ClinicAssistant(), room=ctx.room)
    await session.generate_reply(instructions="Greet the caller and ask how you can help.")

if __name__ == "__main__":
    cli.run_app(
        WorkerOptions(
            entrypoint_fnc=entrypoint,
            prewarm_fnc=prewarm,
        ),
    )

Run it locally with python agent.py console (talk to it in the terminal, no LiveKit room needed) or python agent.py dev (connects to a real room, hot-reloads on save).

SIP Telephony Integration

# livekit-sip.yaml — Connect phone calls to LiveKit rooms
# Incoming calls from a SIP trunk (Twilio, Telnyx) route to AI agent
apiVersion: livekit.io/v1
kind: SIPTrunk
metadata:
  name: clinic-inbound
spec:
  inbound:
    numbers: ["+15551234567"]
    allowed_addresses: ["*.pstn.twilio.com"]
    auth_username: "livekit"
    auth_password: "${SIP_PASSWORD}"
  rules:
    - rule: ".*"
      room_name: "clinic-agent-room"
      participant_identity: "caller-${phone}"

# Create SIP trunk and dispatch rules via API
from livekit import api

lk = api.LiveKitAPI(
    url=os.environ["LIVEKIT_URL"],
    api_key=os.environ["LIVEKIT_API_KEY"],
    api_secret=os.environ["LIVEKIT_API_SECRET"],
)

# Create SIP trunk for inbound calls
trunk = await lk.sip.create_sip_inbound_trunk(
    api.CreateSIPInboundTrunkRequest(
        trunk=api.SIPInboundTrunkInfo(
            name="clinic",
            numbers=["+15551234567"],
        )
    )
)

# Create dispatch rule — route calls to agent
await lk.sip.create_sip_dispatch_rule(
    api.CreateSIPDispatchRuleRequest(
        rule=api.SIPDispatchRule(
            dispatch_rule_individual=api.SIPDispatchRuleIndividual(
                room_prefix="call-",      # Each call gets its own room
                pin="",                    # No PIN required
            ),
        ),
        trunk_ids=[trunk.sip_trunk_id],
    )
)

Video Room (Web Client)

// React component for video conferencing
import { LiveKitRoom, VideoConference, RoomAudioRenderer } from "@livekit/components-react";
import "@livekit/components-styles";

function MeetingRoom({ token, serverUrl }: { token: string; serverUrl: string }) {
  return (
    <LiveKitRoom
      token={token}
      serverUrl={serverUrl}
      connect={true}
      audio={true}
      video={true}
    >
      <VideoConference />
      <RoomAudioRenderer />
    </LiveKitRoom>
  );
}

// Generate access token on server — AccessToken.toJwt() is async as of livekit-server-sdk v2
import { AccessToken } from "livekit-server-sdk";

async function createToken(roomName: string, participantName: string): Promise<string> {
  const token = new AccessToken(
    process.env.LIVEKIT_API_KEY,
    process.env.LIVEKIT_API_SECRET,
    { identity: participantName }
  );
  token.addGrant({
    roomJoin: true,
    room: roomName,
    canPublish: true,
    canSubscribe: true,
  });
  return await token.toJwt();
}

Installation

# Server
docker run -d --name livekit -p 7880:7880 -p 7881:7881 \
  livekit/livekit-server --dev

# Python agent framework
pip install livekit-agents livekit-plugins-deepgram livekit-plugins-openai livekit-plugins-elevenlabs livekit-plugins-silero

# JavaScript client
npm install livekit-client @livekit/components-react

# LiveKit Cloud: https://cloud.livekit.io (managed hosting)

Examples

Example 1: "Build a voice agent that books dental appointments over the phone"

Write agent.py with the ClinicAssistant from the Instructions section (an Agent with a book_appointment tool), run python agent.py console to talk to it from the terminal first, then wire the SIP trunk and dispatch rule from the SIP Telephony section so inbound calls to the clinic's number land in a fresh room per caller and dispatch to this agent.

Example 2: "Add a video call screen to our React app"

Request a token from the server (the createToken function in the Video Room section — call it from an API route, not the browser, since it needs LIVEKIT_API_SECRET), then render <LiveKitRoom token={token} serverUrl={process.env.NEXT_PUBLIC_LIVEKIT_URL}><VideoConference /><RoomAudioRenderer /></LiveKitRoom> in the call page.

Guidelines

1. Agents framework for voice AI — Build an Agent (instructions + @function_tool methods) and run it in an AgentSession (stt/llm/tts/vad); the session handles interruptions, turn-taking, and audio routing. VoiceAssistant and fnc_ctx were removed in the 1.0 Agents framework — don't use them in new code.
2. SIP for telephony — Connect phone numbers via SIP trunks (Twilio, Telnyx); LiveKit handles the WebRTC↔SIP bridge.
3. Silero VAD — Include voice activity detection (silero.VAD.load()); prevents the agent from responding to background noise.
4. Prewarm models — Load VAD and other models in prewarm_fnc; eliminates cold-start latency on first call.
5. Room-per-call — Create a unique room for each phone call or interaction for clean isolation and easy cleanup.
6. Cloud for production — LiveKit Cloud (cloud.livekit.io) handles scaling, TURN servers and global edge nodes; self-host only if you need that control.
7. toJwt() is async in livekit-server-sdk v2 — await it; a missed await silently returns a Promise object as the "token" and auth will fail.
8. Interruption handling — Agents handle barge-in natively; the caller can interrupt the AI mid-sentence.
9. Never hardcode LIVEKIT_API_SECRET or generate tokens in client-side code — issue them from a server endpoint per participant.