算法艺术创作
algorithmic-art
通过代码和可调参数创作生成艺术与交互式视觉效果。
), or **LLM semantic** check.\n- **Simulation tests** — full or partial multi-turn simulations (a partial one starts mid-conversation to unit-test a sub-flow), with **tool mocking** and custom evaluation, scored against an outcome statement you define.\n- **Probabilistic testing** — run a test N times; badges show green (100%), amber (≥80%), red (\u003c80%) pass rates, grouping failures by reason. Because LLM output is stochastic, single-run \"it worked once\" is not a pass.\n- **Create test from conversation** — turn a real failed call into a regression test in one click.\n\n**[Documented]** Post-conversation **evaluation criteria** (success rubrics) and **data collection** (structured extraction) run automatically on live conversations using the same criteria as simulation, feeding analytics.\n\n**[Heuristic]** Build a suite covering the happy path, each tool, each transfer condition, and the hostile/off-topic/silent edges; run it before every prompt or model change. Prompt and model swaps routinely regress behavior that \"obviously\" worked.\n\n## Analytics\n\n**[Documented]** The platform provides conversation search, transcripts, success-evaluation results, extracted data-collection fields, A/B testing, and performance metrics for monitoring at scale. **[Heuristic]** Watch median *and* tail latency, interruption/barge-in rates, tool error rates, transfer-to-human rate, and success-criteria pass rate — the tail and the transfer rate reveal production pain the averages hide.\n\n## Pricing, concurrency, and burst\n\n**[Documented, volatile — verified 2026-07-10]** ElevenAgents bills by **call minutes**, separate from the shared TTS credit pool; the **LLM and telephony are billed on top** by usage. Representative self-serve tiers (included minutes / concurrent calls):\n\n| Plan | Included minutes | Concurrent calls |\n|---|---|---|\n| Free | 15 | 4 |\n| Starter | 75 | 6 |\n| Creator | 275 | 10 |\n| Pro | 1,238 | 20 |\n| Scale | 3,738 | 30 |\n| Business | 12,375 | 40 |\n\nOverage ≈ **$0.08/min** standard, **$0.16/min** burst; text messages ≈ **$0.003 each**. **Burst pricing**, when enabled, lets you exceed your concurrency cap up to ~3× at the doubled rate. There is no cap on how many agents you can create; the binding constraint is **concurrent calls**.\n\n**[Heuristic]** Concurrency, not minutes, is what breaks at scale — a spike of simultaneous inbound calls hits the ceiling instantly and extra callers get rejected or burst-billed. For real call-center volume (hundreds of simultaneous calls) you need an enterprise concurrency arrangement; do not assume a self-serve plan absorbs a marketing blast. Model *both* levers: total minutes for the monthly bill, concurrent calls for the busy-hour peak.\n\n## Safety, consent, and disclosure — a synthetic voice talking to real people\n\nThis is a legal and ethical surface, not a nicety. **[Documented / regulatory]**\n\n- **AI disclosure**: Disclose that the caller is talking to an AI. Under the EU AI Act (Article 52 transparency for synthetic/deepfake audio), and analogous emerging rules, artificially generated voice interacting with a person must be disclosed. **[Heuristic]** Put the disclosure in the **first message** (\"Hi, I'm an AI assistant from …\"), not buried later.\n- **TCPA (US)**: ElevenLabs' synthetic voices count as an \"artificial or prerecorded voice.\" **Marketing** calls need **Prior Express Written Consent**; **informational** calls need prior express consent. Call only **8am–9pm** in the recipient's local time zone; scrub against the National DNC Registry and your internal DNC list; honor opt-outs promptly (within 10 business days) and offer an automated opt-out. Consent cannot be a condition of purchase. Many US states impose stricter rules — get legal counsel.\n- **GDPR / biometrics**: Voice is generally biometric data under GDPR. Processing it typically needs an Article 9 lawful basis (usually explicit consent) plus a DPIA (Article 35). For EU/biometric processing use the enterprise tier and a signed DPA.\n- **Voice cloning consent**: If the agent uses a **cloned** voice, you need documented, explicit written consent from the voice owner covering AI/synthetic voice-model creation and generated speech — cloning carries its own compliance posture beyond stock TTS.\n- **Prohibited uses**: impersonation, fraud, evading voice verification, unauthorized robocalling, harassment, voter suppression, and similar. Don't build them.\n\n**[Heuristic]** Bake compliance into config: disclosure in the first message, an easy \"talk to a human\" transfer, a recording/consent notice where required, timezone-aware scheduling for outbound, and DNC checks before dialing. Treat \"is the user OK talking to a bot, and did they consent to the call?\" as a design requirement, not an afterthought.\n\n## Common failure modes\n\n**[Heuristic, from documented behavior]**\n- **Reads formatting aloud** — markdown/emojis/URLs in output. Fix: forbid them in the prompt; request plain speech and spelled-out numbers.\n- **Rambling turns** — no brevity constraint. Fix: cap turns at 1–3 sentences in the prompt.\n- **Gets cut off in noise** — VAD false-triggers. Fix: Patient turn-taking, test on real telephony audio, tune the timeout.\n- **Sluggish replies** — frontier LLM + high-quality voice. Fix: Flash-tier LLM + `eleven_flash_v2_5`, tighten the prompt, add spoken fillers for tool waits.\n- **Tools misfire or don't fire** — weak LLM or vague tool descriptions. Fix: stronger LLM for tool-heavy flows, sharpen descriptions/params, add tool-call tests.\n- **Leaked API key / unauthenticated agent** — key in client code. Fix: signed URLs / conversation tokens minted server-side; authenticated agents in production.\n- **Concurrency wall at launch** — self-serve concurrency too low for peak. Fix: size the plan to busy-hour simultaneous calls, enable burst, or go enterprise.\n- **No AI disclosure / no consent** — regulatory exposure. Fix: disclose in the first message; secure TCPA/GDPR consent before deploying.\n\n## Complete example (labeled example, not a required formula)\n\n**Intent**: an authenticated, low-latency support agent for a mid-size e-commerce brand, deployed as a website voice widget with a phone fallback, that can look up order status and escalate to a human.\n\n- **Trust context**: `low` (external callers).\n- **LLM**: a fast Flash/Sonnet-tier model with high enough tool reliability for order lookups; LLM cascading left on.\n- **TTS**: `eleven_flash_v2_5`, moderate stability, brand voice matched to the customer base's accent.\n- **First message** (also the AI disclosure): *\"Hi, you're speaking with an AI assistant from Northwind Support. I can check an order or connect you to a person — how can I help?\"*\n- **System prompt** (essentials): role = order-status support; keep replies to 1–2 spoken sentences; never read URLs, markdown, or emojis; spell out order numbers and tracking codes; if the order can't be found or the request is out of scope (refunds, complaints), apologize briefly and transfer to a human; call `get_order_status` whenever the user references an order.\n- **Tools**: server tool `get_order_status(order_id)` → your backend; system tool `transfer_to_number` for escalation. `order_id` parameter has a clear description and a regex-checkable format.\n- **Personalization**: `{{customer_first_name}}` and `{{account_tier}}` injected as dynamic variables at session start; `system__caller_id` used to pre-lookup the account on phone calls.\n- **Knowledge base**: shipping-policy and returns-policy documents (as focused files, RAG on) for general questions — *not* per-customer data.\n- **Turn-taking**: Normal eagerness, ~7s timeout; test barge-in on recorded phone audio.\n- **Channel**: React SDK widget over WebRTC using a server-minted conversation token; Twilio number for inbound phone with a personalization webhook.\n- **Testing**: scenario test (empathetic handling of a \"where's my order\" with success + failure examples), tool-call test asserting `get_order_status` fires with a regex-valid `order_id`, and a partial simulation for the escalation branch — each run with `repeat_count` ≥ 5.\n- **Expected result**: sub-second-feel replies for cached info, a spoken filler while `get_order_status` runs, clean handoff to a human on out-of-scope asks.\n- **Likely failure modes**: order-lookup tool misfires on ambiguous input (mitigate with a stronger LLM + clear param description); noisy callers cut off the agent (mitigate with turn-taking tuning); launch-day concurrency spike (size the plan / enable burst).\n- **Variations**: swap the widget for pure telephony via SIP trunk; add a billing subagent and route via Agent Workflows; run outbound reminders as a batch campaign — which then triggers full TCPA consent + calling-window obligations.\n\n## Primary sources (verified 2026-07-10)\n\n- ElevenAgents overview — https://elevenlabs.io/docs/eleven-agents/overview\n- Quickstart — https://elevenlabs.io/docs/eleven-agents/quickstart\n- LLM / Models — https://elevenlabs.io/docs/eleven-agents/customization/llm ; LLM cascading — https://elevenlabs.io/docs/agents-platform/customization/llm/llm-cascading\n- TTS models — https://elevenlabs.io/docs/overview/models\n- Tools (client / server / system) — https://elevenlabs.io/docs/eleven-agents/customization/tools and .../tools/client-tools\n- Knowledge base — https://elevenlabs.io/docs/eleven-agents/customization/knowledge-base\n- Personalization / dynamic variables / overrides — https://elevenlabs.io/docs/eleven-agents/customization/personalization\n- Conversation flow & workflows — https://elevenlabs.io/docs/eleven-agents/customization/agent-workflows ; agent transfer — .../tools/system-tools/agent-transfer\n- SDKs (React / WebSocket / JS) & authentication — https://elevenlabs.io/docs/eleven-agents/libraries/react ; https://elevenlabs.io/docs/agents-platform/customization/authentication\n- Telephony: Twilio native — https://elevenlabs.io/docs/eleven-agents/phone-numbers/twilio-integration/native-integration ; SIP trunking — https://elevenlabs.io/docs/eleven-agents/phone-numbers/sip-trunking\n- Testing / simulation — https://elevenlabs.io/docs/eleven-agents/customization/agent-testing ; https://elevenlabs.io/docs/agents-platform/guides/simulate-conversations\n- Latency optimization — https://elevenlabs.io/blog/how-do-you-optimize-latency-for-conversational-ai\n- Pricing — https://elevenlabs.io/pricing/agents ; concurrency FAQ — help.elevenlabs.io (Agents requests/concurrency)\n- Legal: TCPA — https://elevenlabs.io/docs/eleven-agents/legal/tcpa ; privacy policy — https://elevenlabs.io/privacy-policy\n- Secondary/independent (disclosed method, labeled): Deepgram and Cekura production-latency/turn-taking write-ups (2026).\n","licenseDocuments":[]}}
elevenlabs-agents【设计与创意】Build, configure, and ship production voice agents on the ElevenLabs Agents platform (branded "ElevenAgents," formerly "Conversational AI"). Use when an agent must design a spoken-conversation system prompt, pick an LLM and TTS voice/model for a real-time voice bot, wire client/server/system tools and knowledge-base RAG, connect a channel (WebRTC/Web…
独立 GitHub 快照 · 自动分类 · 以原作者文档为准
Production guidance for building voice (and text) agents on the ElevenLabs Agents platform. The product was renamed from "Conversational AI" to "ElevenLabs Agents" / "ElevenAgents" during 2026; documentation URLs use both /docs/eleven-agents/... and legacy /docs/agents-platform/... paths, and the two mostly mirror each other. All model IDs, prices, limits, and endpoints below are volatile and were verified on 2026-07-10 against elevenlabs.io/docs — re-verify before quoting them to a user, because ElevenLabs adds LLMs and revises plans frequently.
Use this skill when the work is a two-way spoken (or typed) conversation driven by the ElevenLabs Agents runtime: phone bots, website voice widgets, in-app voice assistants, WhatsApp/chat agents, outbound call campaigns, or a custom SDK integration. It also applies to configuring the agent's brain (prompt, LLM, tools, knowledge), its voice, its channel, and its evaluation.
Do not use it for one-shot text-to-speech, audiobook narration, dubbing, sound effects, or music — those are separate ElevenLabs products with their own APIs. If the user only wants a .mp3 from text, they want the TTS API, not an agent.
[Documented] An ElevenAgents conversation is an orchestrated loop over four components (elevenlabs.io/docs/eleven-agents/overview, verified 2026-07-10):
Your job is to choose and tune each stage so the whole loop feels like a person. [Heuristic] Latency and quality trade against each other at every stage; optimize the loop as a whole, not one component in isolation.
[Documented] ElevenLabs advertises first-turn latency under ~500ms for the ElevenAgents 2.0 stack (elevenlabs.io blog, 2026). Approximate per-stage contributions from ElevenLabs' own latency write-up (elevenlabs.io/blog/how-do-you-optimize-latency-for-conversational-ai, verified 2026-07-10):
| Stage | Typical contribution |
|---|---|
| ASR (ElevenLabs custom) | <100 ms |
| Turn-taking / VAD silence detection | tunable; adds the "end-of-turn" wait |
| LLM first-token | Gemini Flash <350 ms; GPT-4-class / Claude Sonnet 700–1000 ms |
| TTS (Flash v2.5) | ~75 ms model time, ~135 ms end-to-end |
| Network (same-region) | ~200 ms+ |
| Telephony (global) | ~500 ms+ |
[Independent benchmark, disclosed method] Third-party monitoring (Cekura, Deepgram, 2026) reports ElevenLabs among the fastest orchestrators — median turn ~1.7s — but with a long tail (P95 ~3.2s), driven mostly by LLM variance and telephony. Treat the sub-500ms figure as a best-case for the ElevenLabs-controlled stages, not the end-to-end phone experience.
[Heuristic] Practical latency levers, highest-impact first:
eleven_flash_v2_5 for the voice unless you have a specific quality reason not to.[Heuristic] A voice-agent prompt is not a chatbot prompt. Spoken output is heard once, in sequence, with no scrollback, so:
[Documented] The dashboard splits this into a First message (the opening line, supports dynamic variables) and a System prompt. A recent platform field, trust_context, marks an agent as low (serving untrusted external callers — restrict tool access) or high (serving the owner — full tool access appropriate) (changelog, 2026, verified 2026-07-10).
[Documented, volatile — verified 2026-07-10] ElevenAgents supports a large, frequently-updated roster including:
The exact list changes almost monthly; always read the live Models page rather than trusting a memorized roster.
[Heuristic] Choose by the job:
[Documented] LLM cascading / fallback: ElevenAgents can fall back through a backup sequence when the primary model errors or times out, protecting the conversation from a single provider outage. A documented default cascade (verified via docs search 2026-07-10) runs Gemini 2.5 Flash → Gemini 2.0 Flash → Claude 3.7 Sonnet → Claude 3.5 Sonnet v2. [Heuristic] For production, keep cascading on and make sure every model in the chain is fast enough that a fallback doesn't wreck the call.
[Documented, verified 2026-07-10] TTS model tradeoffs for agents:
| Model ID | Latency | Languages | Best for |
|---|---|---|---|
eleven_flash_v2_5 | ~75 ms | ~32 | Default for real-time agents. Best speed/quality balance, ~50% lower cost. |
eleven_flash_v2 | ~75 ms | English only | English-only, cost-sensitive. |
eleven_multilingual_v2 | Higher | 29 | Highest lifelike quality/emotion; long-form; when quality > latency. |
eleven_v3 | Higher | 70+ | Most expressive/dramatic; character work; generally not the low-latency real-time pick. |
eleven_turbo_v2_5 | Mid | ~32 | Deprecated for agents; superseded by Flash v2.5. |
[Heuristic] Default to eleven_flash_v2_5. Escalate to eleven_multilingual_v2 only when a brand demands maximum warmth/expressiveness and can absorb the added latency. Higher-quality voice + model + LLM combined can visibly slow the turn — the dashboard itself warns of this.
[Documented] Voice settings that matter for agents: stability (lower = more emotional range and variation, higher = flatter/consistent), similarity boost (adherence to the source voice), and speaker boost (similarity at a slight latency cost). TTS output can be mp3, pcm, or ulaw — telephony typically needs ulaw 8kHz; SIP media runs G711 8kHz or G722 16kHz.
[Heuristic] Pick a voice whose native accent/language matches the caller base, keep stability moderate (very low stability can produce jarring swings across turns), and test the voice on your actual script — sample text flatters voices that stumble on domain jargon, phone numbers, and proper nouns.
[Documented] Agents can run in a fixed language or detect/switch languages; TTS covers 70+ languages overall, with Flash v2.5 at ~32 and Multilingual v2 at 29. [Heuristic] For a multilingual agent, confirm all three stages support the target language: ASR recognition quality, the LLM's fluency, and the chosen TTS model's coverage. A voice cloned from one language may carry an accent when speaking another — audition it.
[Documented] ElevenAgents 2.0 uses a hybrid turn-taking model: a VAD plus deep-learning signals (filler words, prosody, rhythm, micro-pauses) to judge end-of-turn and handle barge-in. Configurable controls include turn eagerness (Eager / Normal / Patient) and a turn timeout from 1–30s (verified 2026-07-10).
[Heuristic]
background_sound (ambient audio) option exists and an agent_response_complete client event now fires once when a turn is fully delivered — useful for reliable end-of-turn UI, wake-word gating, or message batching (changelog 2026).[Documented, verified 2026-07-10] Attach documents to an agent as grounded knowledge:
conversation_config.agent.prompt.knowledge_base, each with type, name, id, and an optional usage_mode. RAG can be enabled so the agent retrieves relevant chunks rather than stuffing everything into context.[Heuristic] Split large manuals into focused documents, keep them current, and mine transcripts for questions the KB failed to answer. Put stable reference material in the KB; put per-user facts (account balance, order status) in dynamic variables or a server tool — never bake a customer's data into the KB.
[Documented] Three tool types:
type: "client", a name, description, parameters, and expects_response / "Wait for response" — when enabled, the agent pauses and appends the returned data to the conversation before continuing.[Heuristic] Tool reliability depends heavily on the LLM. Give each tool a crisp description and each parameter a clear description and value_type so the model extracts arguments correctly. For flows where a mis-fired tool is costly (payments, cancellations), use a higher-intelligence LLM and add tool-call tests (below). Design tools to be idempotent where possible and mask their latency with a spoken filler.
[Documented, verified 2026-07-10]
{{user_name}}, {{account_tier}}) into the prompt, first message, and tools — the recommended way to personalize per conversation. Passed via conversation_initiation_client_data.dynamic_variables. System variables like system__caller_id, system__called_number, system__call_duration_secs, system__agent_id are auto-available but cannot be set/overridden by the client payload.[Documented] Agent Workflows are a visual editor for branching conversation graphs that route to specialized subagents, each with its own prompt, tools, and scoped knowledge base, and can hand off to a human. Two handoff mechanisms:
transfer_to_agent call is stripped from the child's visible history so it continues seamlessly. Supports an optional delay and an optional spoken transfer message (silent if blank).[Heuristic] Prefer several small, single-purpose subagents over one giant prompt: each stays short (lower latency, fewer instructions to violate) and easier to test. Reserve human transfer for genuine escalation and always give the agent an explicit fallback path so a stuck caller is never trapped.
[Documented, verified 2026-07-10]
@elevenlabs/react), JavaScript/Web (@elevenlabs/client), Swift (iOS), Kotlin (Android), React Native, plus Python/TypeScript server SDKs. A CLI (@elevenlabs/cli) manages agents as code.startSession takes signedUrl, conversationToken, or agentId and infers transport from mode.[Documented] Never put the ElevenLabs API key in client code. For authenticated agents:
[Heuristic] Always run authenticated agents in production so anyone can't rack up minutes on your agent ID, and gate the token/signed-URL endpoint behind your own auth. Public/unauthenticated agents are fine only for open demos.
[Documented, verified 2026-07-10] ElevenAgents has a built-in test framework, runnable from the dashboard Tests tab, the CLI (elevenlabs agents test <agent_id>), or the SDK (run_tests / runTests, with repeat_count for probabilistic runs):
^ORD-[0-9]{8}$), or LLM semantic check.[Documented] Post-conversation evaluation criteria (success rubrics) and data collection (structured extraction) run automatically on live conversations using the same criteria as simulation, feeding analytics.
[Heuristic] Build a suite covering the happy path, each tool, each transfer condition, and the hostile/off-topic/silent edges; run it before every prompt or model change. Prompt and model swaps routinely regress behavior that "obviously" worked.
[Documented] The platform provides conversation search, transcripts, success-evaluation results, extracted data-collection fields, A/B testing, and performance metrics for monitoring at scale. [Heuristic] Watch median and tail latency, interruption/barge-in rates, tool error rates, transfer-to-human rate, and success-criteria pass rate — the tail and the transfer rate reveal production pain the averages hide.
[Documented, volatile — verified 2026-07-10] ElevenAgents bills by call minutes, separate from the shared TTS credit pool; the LLM and telephony are billed on top by usage. Representative self-serve tiers (included minutes / concurrent calls):
| Plan | Included minutes | Concurrent calls |
|---|---|---|
| Free | 15 | 4 |
| Starter | 75 | 6 |
| Creator | 275 | 10 |
| Pro | 1,238 | 20 |
| Scale | 3,738 | 30 |
| Business | 12,375 | 40 |
Overage ≈ $0.08/min standard, $0.16/min burst; text messages ≈ $0.003 each. Burst pricing, when enabled, lets you exceed your concurrency cap up to ~3× at the doubled rate. There is no cap on how many agents you can create; the binding constraint is concurrent calls.
[Heuristic] Concurrency, not minutes, is what breaks at scale — a spike of simultaneous inbound calls hits the ceiling instantly and extra callers get rejected or burst-billed. For real call-center volume (hundreds of simultaneous calls) you need an enterprise concurrency arrangement; do not assume a self-serve plan absorbs a marketing blast. Model both levers: total minutes for the monthly bill, concurrent calls for the busy-hour peak.
This is a legal and ethical surface, not a nicety. [Documented / regulatory]
[Heuristic] Bake compliance into config: disclosure in the first message, an easy "talk to a human" transfer, a recording/consent notice where required, timezone-aware scheduling for outbound, and DNC checks before dialing. Treat "is the user OK talking to a bot, and did they consent to the call?" as a design requirement, not an afterthought.
[Heuristic, from documented behavior]
eleven_flash_v2_5, tighten the prompt, add spoken fillers for tool waits.Intent: an authenticated, low-latency support agent for a mid-size e-commerce brand, deployed as a website voice widget with a phone fallback, that can look up order status and escalate to a human.
low (external callers).eleven_flash_v2_5, moderate stability, brand voice matched to the customer base's accent.get_order_status whenever the user references an order.get_order_status(order_id) → your backend; system tool transfer_to_number for escalation. order_id parameter has a clear description and a regex-checkable format.{{customer_first_name}} and {{account_tier}} injected as dynamic variables at session start; system__caller_id used to pre-lookup the account on phone calls.get_order_status fires with a regex-valid order_id, and a partial simulation for the escalation branch — each run with repeat_count ≥ 5.get_order_status runs, clean handoff to a human on out-of-scope asks.