skillsmcpSign in
← All packs
skill pack

ElevenLabs

ElevenLabs' skills for speech, voice agents, dubbing, music and sound effects.

12 skills by ElevenLabs · elevenlabs/skills · MIT licence · checked on 18 Nov 2026

  • agentsBuild voice AI agents with ElevenLabs. Use when creating voice assistants, customer service bots, interactive voice characters, or any real-time voice conversation experience, and when configuring an agent's tools, workflows, or procedures, including creating, editing, and publishing free-form and structured procedure drafts on an agent branch over the SDKs or REST API.Read
    ---
    name: agents
    description: Build voice AI agents with ElevenLabs. Use when creating voice assistants, customer service bots, interactive voice characters, or any real-time voice conversation experience, and when configuring an agent's tools, workflows, or procedures, including creating, editing, and publishing free-form and structured procedure drafts on an agent branch over the SDKs or REST API.
    license: MIT
    compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
    metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
    ---
    
    # ElevenLabs Agents Platform
    
    Build voice AI agents with natural conversations, multiple LLM providers, custom tools, and easy web embedding.
    
    > **Setup, before writing any code:** if `ELEVENLABS_API_KEY` isn't set for this project, run `npx -y @elevenlabs/cli@latest onboard init` from the app's folder first and follow what it prints: it signs the user in and stores a key for the project, so nobody copies one by hand. When the user approved in the browser, that also signed the ElevenLabs CLI in, so don't run `elevenlabs auth login` after it. When they added the key by hand (on a remote machine), or a CLI command answers 401, load the env file into the command instead, for example `(set -a && . ./.env.local && set +a && npx -y @elevenlabs/cli@latest agents list)`; the CLI reads a `.env` on its own. Otherwise see the [Installation Guide](references/installation.md) for CLI and SDK setup.
    
    ## Quick Start with CLI
    
    The ElevenLabs CLI is the recommended way to create and manage agents:
    
    ```bash
    # Install CLI and authenticate. After onboarding, skip these two lines and run each
    # command below through npx instead, e.g. `npx -y @elevenlabs/cli@latest agents init`
    npm install -g @elevenlabs/cli
    elevenlabs auth login
    
    # Initialize project and create an agent
    elevenlabs agents init
    elevenlabs agents add "My Assistant" --template default
    
    # Push to ElevenLabs platform
    elevenlabs agents push
    ```
    
    **Available templates:** `default`, `minimal`, `voice-only`, `text-only`, `customer-service`, `assistant`
    
    Once the agent exists, give the user its links: they can talk to it at `https://elevenlabs.io/app/talk-to?agent_id=<agent-id>` and edit it at `https://elevenlabs.io/app/agents/agents/<agent-id>`. On a data residency account, use its region's host instead of `elevenlabs.io`, such as `eu.residency.elevenlabs.io`. The talk-to page only opens public agents, which is the default; for an agent with authentication enabled, give only the dashboard link.
    
    ### Python
    
    ```python
    from elevenlabs import ElevenLabs
    
    client = ElevenLabs()
    
    agent = client.conversational_ai.agents.create(
        name="My Assistant",
        conversation_config={
            "agent": {
                "first_message": "Hello! How can I help?",
                "language": "en",
                "prompt": {
                    "prompt": "You are a helpful assistant. Be concise and friendly.",
                    "llm": "gemini-2.0-flash",
                    "temperature": 0.7
                }
            },
            "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
        }
    )
    ```
    
    ### JavaScript
    
    ```javascript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    const client = new ElevenLabsClient();
    
    const agent = await client.conversationalAi.agents.create({
      name: "My Assistant",
      conversationConfig: {
        agent: {
          firstMessage: "Hello! How can I help?",
          language: "en",
          prompt: {
            prompt: "You are a helpful assistant.",
            llm: "gemini-2.0-flash",
            temperature: 0.7
          }
        },
        tts: { voiceId: "JBFqnCBsd6RMkjVDRZzb" }
      }
    });
    ```
    
    ### CLI
    
    The CLI reads `ELEVENLABS_API_KEY` from the environment automatically:
    
    ```bash
    elevenlabs agents create \
      --json '{"name": "My Assistant", "conversation_config": {"agent": {"first_message": "Hello!", "language": "en", "prompt": {"prompt": "You are helpful.", "llm": "gemini-2.0-flash"}}, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}}}'
    ```
    
    ## Starting Conversations
    
    **Authenticated WebRTC:** Request a session token from your backend. The response includes both
    the token and the conversation ID:
    ```python
    session = client.conversational_ai.conversations.get_webrtc_token(
        agent_id="your-agent-id",
    )
    print(session.token, session.conversation_id)
    ```
    
    **Server-side (Python):** Get signed URL for client connection:
    ```python
    signed_url = client.conversational_ai.conversations.get_signed_url(
        agent_id="your-agent-id",
        environment="staging",
    )
    ```
    
    **Client-side (JavaScript):**
    ```javascript
    import { Conversation } from "@elevenlabs/client";
    
    const conversation = await Conversation.startSession({
      agentId: "your-agent-id",
      environment: "staging",
      overrides: { asr: { keywords: ["ElevenLabs", "TechCorp"] } },
      onMessage: (msg) => console.log("Agent:", msg.message),
      onUserTranscript: (t) => console.log("User:", t.message),
      onPing: (event) => console.log("Estimated latency:", event.ping_ms),
      onContextUsage: ({ model, context_tokens, context_limit_tokens }) =>
        console.log(`${model}: ${context_tokens}/${context_limit_tokens} context tokens`),
      onError: (e) => console.error(e)
    });
    ```
    
    **React Hook:** Wrap hook consumers in `ConversationProvider`. Prefer granular hooks such as
    `useConversationControls` and `useConversationStatus` for session controls and UI state;
    `useConversation` remains available as the convenience all-in-one hook. Pass provider-level
    callbacks such as `onError` when you want React to handle conversation errors in one place.
    ```typescript
    import {
      ConversationProvider,
      useConversationControls,
      useConversationStatus,
    } from "@elevenlabs/react";
    
    function Agent({ signedUrl }: { signedUrl: string }) {
      const { startSession, endSession } = useConversationControls();
      const { status } = useConversationStatus();
    
      if (status === "connected") {
        return <button onClick={endSession}>End conversation</button>;
      }
    
      return (
        <button onClick={() => startSession({ signedUrl })}>
          Start conversation
        </button>
      );
    }
    
    function App({ signedUrl }: { signedUrl: string }) {
      return (
        <ConversationProvider
          onError={(error) => console.error("Conversation error:", error)}
          onPing={(event) => console.log("Estimated latency:", event.ping_ms)}
          onContextUsage={({ model, context_tokens, context_limit_tokens }) =>
            console.log(`${model}: ${context_tokens}/${context_limit_tokens} context tokens`)
          }
        >
          <Agent signedUrl={signedUrl} />
        </ConversationProvider>
      );
    }
    ```
    
    ## Configuration
    
    | Provider | Models |
    |----------|--------|
    | OpenAI | `gpt-6.1-sol`, `gpt-6.1-terra`, `gpt-6-sol`, `gpt-6-luna`, `gpt-6-astra`, `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, `gpt-5.5-2026-04-23`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.4-2026-03-05`, `gpt-5.4-mini-2026-03-17`, `gpt-5.4-nano-2026-03-17`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` |
    | Anthropic | `claude-opus-5-5`, `claude-opus-5`, `claude-sonnet-5-5`, `claude-opus-4-7`, `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-sonnet-4`, `claude-haiku-4-5`, `claude-3-7-sonnet`, `claude-3-5-sonnet`, `claude-3-haiku` |
    | Google | `gemini-4-argon`, `gemini-3.8-flash`, `gemini-3.7-flash`, `gemini-3.6-flash`, `gemini-3.5-flash`, `gemini-3.5-flash-lite`, `gemini-3.1-flash-lite-preview`, `gemini-3.1-pro-preview`, `gemini-3-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.0-flash`, `gemini-2.0-flash-lite` |
    | ElevenLabs | `glm-52`, `deepseek-v41-flash`, `glm-45-air-fp8`, `qwen3-30b-a3b`, `qwen36-35b-a3b`, `qwen35-35b-a3b`, `qwen35-397b-a17b`, `nemotron-35-super-vl-nvfp4`, `gpt-oss-120b` |
    | Custom | `custom-llm` (bring your own endpoint) |
    
    Use `GET /v1/convai/llm/list` to inspect the current model catalog, including deprecation state, token/context limits, capability flags such as image-input support, and model-specific reasoning effort support.
    
    To run an agent on an ElevenLabs Realtime model instead of a separate LLM and TTS model (`conversation_config.realtime_model`), use the `realtime` skill.
    
    **Popular voices:** `JBFqnCBsd6RMkjVDRZzb` (George), `EXAVITQu4vr4xnSDxMaL` (Sarah), `onwK4e9ZLuTAKqWW03F9` (Daniel)
    
    **Turn eagerness:** `patient` (waits longer for user to finish), `normal`, or `eager` (responds quickly)
    
    See [Agent Configuration](references/agent-configuration.md) for all options.
    
    ## System Prompt Structure
    
    Section the prompt with markdown headings — the model prioritizes and interprets instructions more reliably ([prompting guide](https://elevenlabs.io/docs/eleven-agents/best-practices/prompting-guide)):
    
    ```
    # Personality   – named character, 2-3 traits
    # Environment   – where they work, who they talk to
    # Tone          – vocal style as 4-5 bullets
    # Goal          – what success looks like (numbered for multi-step flows)
    ```
    
    Keep instructions short and action-based. Mark critical steps with "This step is important." For critical refusal/safety rules, include concise instructions in the prompt and also configure independent custom Guardrails via `platform_settings.guardrails` (see [Guardrails](#guardrails)).
    
    ## Tools
    
    Extend agents with webhook, client, or built-in system tools. Tools are defined inside `conversation_config.agent.prompt`:
    
    Workspace environment variables can resolve per-environment server tool URLs, headers, and auth connections, and runtime system variables such as `{{system__conversation_history}}` can pass full conversation context into tool calls when needed.
    
    ```python
    "prompt": {
        "prompt": "You are a helpful assistant that can check the weather.",
        "llm": "gemini-2.0-flash",
        "tools": [
            # Webhook: server-side API call
            {"type": "webhook", "name": "get_weather", "description": "Get weather",
             "api_schema": {"url": "https://api.example.com/weather", "method": "POST",
                 "request_body_schema": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}},
            # Client: runs in the browser
            {"type": "client", "name": "show_product", "description": "Display a product",
             "parameters": {"type": "object", "properties": {"productId": {"type": "string"}}, "required": ["productId"]}}
        ],
        "built_in_tools": {
            "end_call": {},
            "transfer_to_number": {"transfers": [{"transfer_destination": {"type": "phone", "phone_number": "+1234567890"}, "condition": "User asks for human support"}]}
        }
    }
    ```
    
    **Client tools** run in browser:
    ```javascript
    clientTools: {
      show_product: async ({ productId }) => {
        document.getElementById("product").src = `/products/${productId}`;
        return { success: true };
      }
    }
    ```
    
    See [Client Tools Reference](references/client-tools.md) for complete documentation.
    
    ### Built-in System Tools
    
    Set under `conversation_config.agent.prompt.built_in_tools`. `{}` enables defaults; provide `description` to customize; omit to disable.
    
    | Tool | Enable for |
    |------|------------|
    | `end_call` | All agents |
    | `language_detection` | Multilingual agents |
    | `transfer_to_number` | Phone-based human escalation |
    | `transfer_to_agent` | Multi-agent workflows |
    | `skip_turn` | Tutoring / coaching (silent listening) |
    | `voicemail_detection` | Outbound calling |
    | `play_keypad_touch_tone` | IVR navigation |
    
    `start_procedure` and `end_procedure` are not configured here. The platform adds them automatically whenever the agent has at least one procedure (see [Procedures](#procedures)).
    
    `run_subagent` is a system tool for delegating a task to another configured agent. Add it to
    `conversation_config.agent.prompt.tools` with `params.system_tool_type: "run_subagent"` and an
    `agents` array. Each entry requires `agent_id` and `description`; `branch_id` and a JSON-schema
    `parameters` object are optional.
    
    `knowledge_base` is a system tool for letting the model choose how to inspect attached knowledge.
    Add it to `conversation_config.agent.prompt.tools` with `type: "system"`, a `name`, and
    `params.system_tool_type: "knowledge_base"`. Use `enabled_strategies` to expose any combination of
    `cat`, `keyword`, `semantic`, and `ls`:
    
    ```json
    {
      "type": "system",
      "name": "knowledge_base",
      "description": "Search the attached knowledge base.",
      "params": {
        "system_tool_type": "knowledge_base",
        "enabled_strategies": ["semantic", "keyword"]
      }
    }
    ```
    
    ### Integration Tools
    
    Pre-built connectors managed by the platform. Create a connection with credentials, then attach via `tool_ids`:
    
    | Integration | Use case |
    |-------------|----------|
    | `calcom` | Scheduling appointments |
    | `salesforce` | CRM lookups, case creation |
    | `hubspot` | CRM, marketing, contacts |
    | `zendesk` | Support ticketing |
    
    Three-step flow: `POST /v1/convai/api-integrations/{id}/connections` → `GET /v1/convai/api-integrations/{id}/tools` → `POST /v1/convai/tools` with `api_integration_id` and `api_integration_connection_id`. Attach to the agent with `"prompt": {"tool_ids": ["tool_xxxx"]}`. Inline `tools` and `tool_ids` can coexist — prefer an integration over a duplicate custom webhook.
    
    ### Public-API Webhook Examples
    
    No-auth APIs useful for prototypes (URLs must be HTTPS):
    
    | Tool | URL | Purpose |
    |------|-----|---------|
    | `get_weather` | `https://wttr.in/{location}?format=j1` | Current weather |
    | `search_wikipedia` | `https://en.wikipedia.org/api/rest_v1/page/summary/{topic}` | Topic summary |
    | `get_exchange_rate` | `https://open.er-api.com/v6/latest/{base_currency}` | FX rates |
    
    ## Workflows
    
    Route conversations through discrete steps with branching logic. Define under the agent's top-level `workflow` field. Reference: [Agent Workflows](https://elevenlabs.io/docs/eleven-agents/customization/agent-workflows).
    
    **Node types:** `start` (ID must be `"start_node"`), `end` (`outcome` records the conversation as `success`, the default, or `failure`), `override_agent` (subagent step with `label` + `additional_prompt`), `dispatch_tool` (executes a tool with success/failure routing), `agent_transfer`, `transfer_to_number`.
    
    **Edge types:** `unconditional`, `llm` (natural-language condition), `expression` (deterministic data check). Tool nodes have separate success/failure edges.
    
    **Scope tools per step** with `additional_tool_ids` on a node — prevents the wrong tool firing at the wrong step. Set `additional_tool_ids: []` on conversational routing nodes such as greeting and `classify_intent` so they only converse:
    
    ```json
    {
      "type": "override_agent",
      "label": "Book Appointment",
      "additional_prompt": "Discuss preferred dates and doctors. Show the booking form once agreed.",
      "entry_behavior": "wait_for_user",
      "additional_tool_ids": ["show_booking_form", "display_appointment_card"],
      "position": {"x": 0, "y": 400}
    }
    ```
    
    Include `position` (`{x, y}`) on every node so the editor renders cleanly. Start at `y=0`, put `end` at the bottom, and space branches horizontally at `x=-150` and `x=150`; suggested spacing is 200px vertical between levels and 300px horizontal between branches. Keep workflows to 4-7 nodes and always have a path to `end`.
    
    Use `entry_behavior` on `override_agent` nodes to choose whether a sub-agent speaks immediately (`generate_immediately`), waits for user input (`wait_for_user`), or lets the platform decide (`auto`).
    
    For nested agent transfers, set `enable_nesting` on a `standalone_agent` node and
    `return_when_nested` on an `end` node that should return control to the parent workflow.
    
    ### Build Workflows in Code
    
    The Python and JavaScript SDKs include a `Workflow` builder that produces the `workflow` value for `agents.create()` and `agents.update()`. Chain `converse`, `tool`, `transfer`, and `end` steps, and route between them with `when` (a string is an LLM condition; an `Expr` is an expression on dynamic variables), `on_success` / `on_failure`, and `go_to`. `build()` assigns `start_node`, edge IDs, `edge_order`, and editor positions, and raises one `WorkflowValidationError` listing every workflow graph problem. Reference: [Workflows as code](https://elevenlabs.io/docs/eleven-agents/customization/agent-workflows-as-code).
    
    ```python
    from elevenlabs import ElevenLabs
    from elevenlabs.agent_workflows import Destination, Workflow
    
    client = ElevenLabs()
    
    wf = Workflow()
    triage = wf.converse("triage", "Greet the caller and find out why they're calling")
    lookup = triage.when("Caller gave an order number").tool("lookup_order", ["tool_get_order"])
    lookup.on_result(
        success=lambda ok: ok.converse("resolve", "Explain the order status").when("Caller is satisfied").end(),
        failure=lambda failed: failed.go_to(triage),
    )
    triage.when("Caller wants a human").transfer("human", Destination.phone("+15551234567"))
    
    client.conversational_ai.agents.update("agent_id", workflow=wf.build())
    ```
    
    ```javascript
    import { Destination, ElevenLabsClient, Workflow } from "@elevenlabs/elevenlabs-js";
    
    const client = new ElevenLabsClient();
    
    const wf = new Workflow();
    const triage = wf.converse("triage", "Greet the caller and find out why they're calling");
    const lookup = triage.when("Caller gave an order number").tool("lookup_order", ["tool_get_order"]);
    lookup.onResult({
      success: (ok) => ok.converse("resolve", "Explain the order status").when("Caller is satisfied").end(),
      failure: (failed) => failed.goTo(triage),
    });
    triage.when("Caller wants a human").transfer("human", Destination.phone("+15551234567"));
    
    await client.conversationalAi.agents.update("agent_id", { workflow: wf.build() });
    ```
    
    To edit an existing workflow, load it with `Workflow.from_agent(agent)` / `Workflow.fromAgent(agent)`, then change nodes with `wf.node(id).update(...)`, `insert_between` / `insertBetween`, or `remove`; IDs and positions are kept. `to_json_string()` / `toJSONString()` and `Workflow.from_json()` / `Workflow.fromJSON()` store workflows as JSON in version control, and `elevenlabs agents workflow diagram <file>` draws a workflow file as a flowchart (`--format mermaid` for Mermaid output).
    
    A Start procedure node starts an attached procedure at a chosen point in the workflow graph. See [Procedures](#procedures) for the `start_procedure` node and its continuation behavior.
    
    ## Procedures
    
    Reusable instruction blocks an agent runs when a trigger matches or a Start procedure workflow node starts it. A procedure is `free_form` (markdown guidance the agent adapts, and the only type that can reference knowledge base documents) or `deterministic` (called "structured" in the dashboard: typed steps that run in a fixed order, for flows that must happen the same way every time). See [Using the Procedure API](references/using-procedure-api.md) for the full CLI and SDK flow, and [Writing Procedures](references/writing-procedures.md) for the step reference, validation rules, and authoring guidance.
    
    Procedures live on an agent branch, and every write stages a per-user draft:
    
    | Operation | Call |
    |-----------|------|
    | List, create, read, update, discard, remove | `/v1/convai/agents/{agent_id}/branches/{branch_id}/procedures...` (`procedures.*` and `procedures.drafts.*` in the SDKs) |
    | Publish | `PATCH /v1/convai/agents/{agent_id}?branch_id=...` (`agents.update`) |
    
    Semantics worth knowing before writing any of these calls:
    
    - Nothing reaches the live agent until you publish. Publishing is not a procedure endpoint; one PATCH on the agent versions every changed procedure draft on the branch.
    - `GET .../procedures/{procedure_id}` reads branch HEAD and returns `404` until that procedure's first publish. Read the `/draft` variant to see a procedure you just created; do not retry the create.
    - Turning a structured procedure's steps into the form the agent executes is called compiling. The platform compiles every structured procedure when the agent draft is saved or the agent is published; authors never compile anything themselves. Both draft save and publish validate every structured procedure on the branch. A failed operation writes nothing. To validate without publishing, save an agent draft with `POST /v1/convai/agents/{agent_id}/drafts?branch_id=...`, sending the agent's current `name`, `conversation_config`, `platform_settings`, and `workflow` unchanged, then discard that agent draft with `DELETE` on the same path; see [Using the Procedure API](references/using-procedure-api.md#validate-without-publishing).
    - If a structured procedure is invalid, the publish or draft save returns `400` with `status` `procedure_validation_failed` and `errors` keyed by procedure ID, each entry carrying the `path` of the offending field and a message. Nothing is written; repair the procedure draft and publish again.
    - A draft update replaces the whole body. Read the draft first, then resend `name`, `type`, and `trigger` alongside the new `content`. `type` cannot change after creation.
    - `content` is markdown for a `free_form` procedure, and a JSON-encoded object with a `steps` array for a `deterministic` one. The trigger is the top-level `trigger` field in both cases, not part of `content`. Serialize it; do not hand-escape quotes.
    - Use a Start procedure workflow node (`type: "start_procedure"`, `procedure_id: "<attached procedure ID>"`) to start either procedure type at a chosen point in the workflow graph without a trigger match. Free-form content loads and traversal continues in the same turn; a structured procedure runs before the node's outgoing edges are evaluated. Connect workflow edges to the Start procedure node. Use a `sub_procedure` step to chain structured procedures.
    - A third `type`, `folder`, groups procedures in the dashboard. Folders carry no `content` or `trigger`; move a procedure into one with `POST .../procedures/{procedure_id}/move`.
    - The agent keeps the five most recently started procedures in context. When more have been started, the content, inline tools, and knowledge base documents of the oldest free-form ones drop out of the prompt, although the procedures remain active. Keep procedures focused and use sub-procedures so that few are active at once.
    - Routing is driven by the `trigger` text, not the procedure name. Write concrete, non-overlapping triggers that cover the phrasings a user would actually say. The model sees only a numbered menu of triggers; it never sees procedure names or IDs.
    - To restrict the starting agent to selected procedures for one conversation, enable `platform_settings.overrides.enable_procedure_ids_from_client`, then pass their IDs as `procedure_ids` in conversation initiation data. An empty list disables all procedures for that starting agent. The same setting allows `start_procedure_ids`: up to five free-form procedures that start with the conversation, for when the user's task is already known (for example from an IVR choice or the page hosting the widget).
    - Procedure APIs require `elevenlabs` (Python) or `@elevenlabs/elevenlabs-js` at `2.60.0` or newer.
    
    ## Guardrails
    
    Layered safety enforcement that runs independently of the LLM — configured under `platform_settings.guardrails`, not in the system prompt. Reference: [Guardrails](https://elevenlabs.io/docs/eleven-agents/best-practices/guardrails).
    
    ```json
    "platform_settings": {
      "guardrails": {
        "version": "1",
        "focus": {"is_enabled": true},
        "prompt_injection": {"is_enabled": true},
        "content": {"config": {"harassment": {"is_enabled": true, "threshold": 0.5}}},
        "custom": {
          "config": {
            "configs": [{
              "is_enabled": true,
              "name": "No medical diagnoses",
              "prompt": "Block the agent from providing medical diagnoses or treatment advice.",
              "execution_mode": "blocking",
              "model": "gemini-3.1-flash-lite",
              "history_message_count": 1,
              "trigger_action": {"type": "retry", "feedback": "Reason: {{trigger_reason}}"}
            }]
          }
        }
      }
    }
    ```
    
    **Types:** `focus` (on-topic), `prompt_injection` (manipulation defense), `content` (category filters), `custom` (LLM-evaluated domain rules). Content categories include `harassment`, `profanity`, `sexual`, `violence`, `self_harm`, and `medical_and_legal_information` — threshold range `0.0`–`1.0` (default `0.3`). Custom rules use `execution_mode: "blocking"` with a `model`, `history_message_count`, and `trigger_action` (e.g., `retry` with feedback). Custom guardrails evaluate in parallel and fail-open.
    
    **Per vertical:** healthcare/finance/legal → enable `medical_and_legal_information`; education/youth → `sexual`/`violence`/`self_harm`/`profanity`; support/sales → `harassment`/`profanity`. All agents benefit from `focus` + `prompt_injection` + 2-4 custom rules.
    
    ## Testing Agents
    
    Three test types via `POST /v1/convai/agent-testing/create`, then attached with PATCH on the agent. Reference: [Agent Testing](https://elevenlabs.io/docs/eleven-agents/customization/agent-testing).
    
    | Type | Purpose |
    |------|---------|
    | `llm` | Scenario test — does the agent respond appropriately to a message? |
    | `tool` | Tool-call test — right tool, right parameters? |
    | `simulation` | Multi-turn flow with a simulated user persona |
    
    ```json
    // Tool-call test (snake_case throughout; chat_history role is "user" or "agent")
    {
      "name": "Books with correct doctor and date",
      "type": "tool",
      "chat_history": [
        {"role": "user", "message": "Dr. Smith on March 5 at 2pm", "time_in_call_secs": 10}
      ],
      "tool_call_parameters": {
        "referenced_tool": {"id": "show_booking_form", "type": "client"},
        "parameters": [
          {"path": "doctor_name", "eval": {"type": "llm", "description": "Should reference Dr. Smith"}},
          {"path": "date", "eval": {"type": "regex", "pattern": "2025-03-05|March 5"}}
        ]
      }
    }
    ```
    
    Eval strategies: `exact`, `regex`, `llm`. Prompt evaluation criteria can use binary scoring or
    numeric scoring with `scoring_mode: "numeric_uniform"`, `max_score`, and `score_instructions`;
    numeric scores are normalized into the aggregate conversation success percentage. Attach via an agent update:
    
    ```bash
    elevenlabs agents update --agent-id "your-agent-id" \
      --json '{"platform_settings": {"testing": {"attached_tests": [{"test_id": "test_xxxx"}]}}}'
    ```
    
    Run selected tests with `POST /v1/convai/agents/{agent_id}/run-tests`. The request
    body requires `tests` and accepts `repeat_count` from `1` to `50` for repeated runs.
    Simulation tests can define up to 30 `success_conditions` prompts; all criteria are
    evaluated and merged into the final result.
    Simulation tests can also define `tool_mock_overrides`, keyed by tool ID, to replace shared response
    mocks for one test. Each override is an array of mocks with a required `mock_result`; set
    `is_error: true` to exercise a tool-failure path. Overrides only apply to tools enabled for mocking
    through `tool_mock_config`.
    For completed conversations, rerun one evaluation criterion with `POST /v1/convai/conversations/{conversation_id}/analysis/evaluations/run` and a request body containing `evaluation_id`.
    
    ## Widget Embedding
    
    ```html
    <elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
    <script src="https://unpkg.com/@elevenlabs/convai-widget-embed" async type="text/javascript"></script>
    ```
    
    Customize with attributes: `avatar-image-url`, `action-text`, `start-call-text`, `end-call-text`.
    
    See [Widget Embedding Reference](references/widget-embedding.md) for all options.
    
    ## Outbound Calls
    
    Make outbound phone calls using your agent via Twilio or Exotel integration:
    
    The examples below use Twilio. See the reference for Exotel usage.
    
    ### Python
    
    ```python
    response = client.conversational_ai.twilio.outbound_call(
        agent_id="your-agent-id",
        agent_phone_number_id="your-phone-number-id",
        to_number="+1234567890",
        call_recording_enabled=True
    )
    print(f"Call initiated: {response.conversation_id}")
    ```
    
    ### JavaScript
    
    ```javascript
    const response = await client.conversationalAi.twilio.outboundCall({
      agentId: "your-agent-id",
      agentPhoneNumberId: "your-phone-number-id",
      toNumber: "+1234567890",
      callRecordingEnabled: true,
    });
    ```
    
    ### CLI
    
    ```bash
    elevenlabs agents twilio outbound_call \
      --agent-id "your-agent-id" \
      --agent-phone-number-id "your-phone-number-id" \
      --to-number "+1234567890" \
      --call-recording-enabled true
    ```
    
    See [Outbound Calls Reference](references/outbound-calls.md) for provider-specific endpoints, configuration overrides, and dynamic variables.
    
    ## Managing Agents
    
    ### Using CLI (Recommended)
    
    ```bash
    # List agents and check status
    elevenlabs agents list
    elevenlabs agents status
    
    # Import agents from platform to local config
    elevenlabs agents pull                      # Import all agents
    elevenlabs agents pull --agent <agent-id>   # Import specific agent
    
    # Push local changes to platform
    elevenlabs agents push              # Upload configurations
    elevenlabs agents push --dry-run    # Preview changes first
    
    # Add tools (names use letters, numbers, - and _)
    elevenlabs tools add weather-api --type webhook
    elevenlabs tools add ui-tool --type client
    ```
    
    ### Project Structure
    
    The CLI creates a project structure for managing agents:
    
    ```
    your_project/
    ├── agents.json       # Agent definitions
    ├── tools.json        # Tool configurations
    ├── tests.json        # Test configurations
    ├── agent_configs/    # Individual agent configs
    ├── tool_configs/     # Individual tool configs
    └── test_configs/     # Individual test configs
    ```
    
    ### SDK Examples
    
    ```python
    # List
    agents = client.conversational_ai.agents.list()
    
    # Get
    agent = client.conversational_ai.agents.get(agent_id="your-agent-id")
    
    # Update (partial - only include fields to change)
    client.conversational_ai.agents.update(agent_id="your-agent-id", name="New Name")
    client.conversational_ai.agents.update(agent_id="your-agent-id",
        conversation_config={
            "agent": {"prompt": {"prompt": "New instructions", "llm": "claude-sonnet-4"}}
        })
    
    # Delete
    client.conversational_ai.agents.delete(agent_id="your-agent-id")
    ```
    
    See [Agent Configuration](references/agent-configuration.md) for all configuration options and SDK examples.
    
    ## Error Handling
    
    ```python
    try:
        agent = client.conversational_ai.agents.create(...)
    except Exception as e:
        print(f"API error: {e}")
    ```
    
    Common errors: **401** (invalid key), **404** (not found), **422** (invalid config), **429** (rate limit)
    
    ## References
    
    - [Installation Guide](references/installation.md) - SDK setup and migration
    - [Agent Configuration](references/agent-configuration.md) - All config options and CRUD examples
    - [Client Tools](references/client-tools.md) - Webhook, client, and system tools
    - [Using the Procedure API](references/using-procedure-api.md) - Procedure CLI and SDK flow, drafts and publish
    - [Writing Procedures](references/writing-procedures.md) - Trigger and content authoring, structured step reference and rules
    - [Widget Embedding](references/widget-embedding.md) - Website integration
    - [Outbound Calls](references/outbound-calls.md) - Phone call integrations
    
    Original on GitHub
  • dubbingDub audio and video into other languages using the ElevenLabs Dubbing API (dubbing_v2), preserving the original speakers' voices. Use when translating videos, podcasts, or recordings into other languages, localizing media content, reviewing or correcting dubbing transcripts and translations, or regenerating a dub after edits.Read
    ---
    name: dubbing
    description: Dub audio and video into other languages using the ElevenLabs Dubbing API (dubbing_v2), preserving the original speakers' voices. Use when translating videos, podcasts, or recordings into other languages, localizing media content, reviewing or correcting dubbing transcripts and translations, or regenerating a dub after edits.
    license: MIT
    compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
    metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
    ---
    
    # ElevenLabs Dubbing
    
    Dub audio or video into other languages while preserving the original speakers' voices. Create a project from a file or URL, review and edit the source transcript, add one or more target languages, refine translations per segment, and regenerate outputs.
    
    > **Important:** Use the Dubbing Projects API — `elevenlabs.dubbing.project.*` in the SDKs, or the `/v1/dubbing/project` REST endpoints. Do **not** use the legacy v1 dubbing surface (`client.dubbing.create()`, `client.dubbing.get()`, `client.dubbing.audio.get()`, or bare `/v1/dubbing` routes) — that is the older dubbing API, now under Legacy in the API reference.
    
    > **Setup, before writing any code:** if `ELEVENLABS_API_KEY` isn't set for this project, run `npx -y @elevenlabs/cli@latest onboard init` from the app's folder first and follow what it prints: it signs the user in and stores a key for the project, so nobody copies one by hand. Otherwise see the [Installation Guide](references/installation.md). The `elevenlabs` CLI and the SDKs read `ELEVENLABS_API_KEY` automatically; REST base URL is `https://api.elevenlabs.io` with your API key in the `xi-api-key` header.
    
    ## Concepts
    
    | Concept | Meaning |
    |---------|---------|
    | **Project** | One source of media (file or URL) plus its source transcript. Prepared (transcribed) once, then rests in `ready` while you add languages. |
    | **Source transcript** | Editable segments (text, speaker, timing) transcribed from the source. The single source of truth every language is translated from. |
    | **Language (target)** | One dubbed output language. Each has its own transcript (source segments + a translation per segment) and its own dubbed audio output. |
    | **Revisions** | Independent monotonic counters. The project's `revision` bumps on source-transcript edits; a language's `revision` bumps on translation edits or source edits that affect it. A language's `output_revision` is the revision its current audio was generated from — when it's behind `revision`, the output is out of date. |
    
    **Recommended order of operations:** finalize the source transcript **before** adding any languages. Translations are produced from the source, so correcting the source first means every language starts from the right text — editing the source after a language completes marks it `stale` and requires a (charged) regeneration.
    
    > **Enterprise:** Transcript editing and regeneration are available to enterprise workspaces only. Creating projects, adding languages, and downloading dubs work on all plans.
    
    ## Workflow
    
    1. **Create** the project from a file or URL → `queued`
    2. **Poll** the project until `ready`
    3. **Review and finalize the source transcript** (edit/add/delete segments)
    4. **Add** one language per target → `queued` → `processing` → `completed`
    5. **Download** each language's `outputs.lossless_audio` when `completed`
    6. **Refine** translations per segment if needed → the language goes `stale`
    7. **Regenerate** the language → `completed` again with fresh output
    
    ## Quick Start (Python)
    
    ```python
    import os
    import time
    import requests
    from elevenlabs.client import ElevenLabs
    
    elevenlabs = ElevenLabs(api_key=os.getenv("ELEVENLABS_API_KEY"))
    
    # 1. Create a project from a local file (or pass source_url=... instead of file)
    with open("promo.mp4", "rb") as f:
        project = elevenlabs.dubbing.project.create(
            file=f,
            source_language="en",
            reference="Q3 marketing video",
        )
    
    # 2. Wait for the source media to be transcribed
    while True:
        project = elevenlabs.dubbing.project.get(project.project_id)
        if project.status == "ready":
            break
        if project.status == "failed":
            raise RuntimeError("Project preparation failed")
        time.sleep(5)
    
    # 3. Add a Spanish language target
    language = elevenlabs.dubbing.project.language.create(
        project.project_id,
        target_language="es",
    )
    
    # 4. Wait for the dub to finish generating
    while True:
        language = elevenlabs.dubbing.project.language.get(
            project.project_id, language.language_id
        )
        if language.status == "completed":
            break
        if language.status == "failed":
            raise RuntimeError("Dub generation failed")
        time.sleep(5)
    
    # 5. Download the dubbed audio (signed URL, valid ~1 hour — re-fetch the language for a fresh one)
    audio = requests.get(language.outputs.lossless_audio)
    with open("promo_es.wav", "wb") as f:
        f.write(audio.content)
    ```
    
    ## Quick Start (JavaScript)
    
    ```typescript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    import { writeFile } from "fs/promises";
    
    const elevenlabs = new ElevenLabsClient();
    
    // 1. Create a project (sourceUrl shown; file upload is also supported)
    let project = await elevenlabs.dubbing.project.create({
      sourceUrl: "https://example.com/promo.mp4",
      sourceLanguage: "en",
      reference: "Q3 marketing video",
    });
    
    // 2. Wait for the source media to be transcribed
    while (true) {
      project = await elevenlabs.dubbing.project.get(project.projectId);
      if (project.status === "ready") break;
      if (project.status === "failed") throw new Error("Project preparation failed");
      await new Promise((resolve) => setTimeout(resolve, 5000));
    }
    
    // 3. Add a Spanish language target
    let language = await elevenlabs.dubbing.project.language.create(project.projectId, {
      targetLanguage: "es",
    });
    
    // 4. Wait for the dub to finish generating
    while (true) {
      language = await elevenlabs.dubbing.project.language.get(project.projectId, language.languageId);
      if (language.status === "completed") break;
      if (language.status === "failed") throw new Error("Dub generation failed");
      await new Promise((resolve) => setTimeout(resolve, 5000));
    }
    
    // 5. Download the dubbed audio from the signed URL
    const response = await fetch(language.outputs!.losslessAudio!);
    await writeFile("promo_es.wav", Buffer.from(await response.arrayBuffer()));
    ```
    
    ## Quick Start (CLI)
    
    The `elevenlabs` CLI reads `ELEVENLABS_API_KEY` from the environment automatically.
    
    ```bash
    # 1. Create a project (use --source-url "https://..." instead of --file to dub from a URL)
    elevenlabs dubbing project create --file promo.mp4 --source-language en
    # → {"project_id": "proj_...", "status": "queued", ...}
    
    # 2. Poll until status is "ready"
    elevenlabs dubbing project get --project-id proj_...
    
    # 3. Add a target language
    elevenlabs dubbing project language create --project-id proj_... --target-language es
    
    # 4. Poll the language until "completed", then download outputs.lossless_audio
    elevenlabs dubbing project language get --project-id proj_... --language-id lang_...
    ```
    
    ## Create Options
    
    `elevenlabs dubbing project create` (REST: `POST /v1/dubbing/project`, `multipart/form-data`) takes **either** `file` **or** `source_url` (not both):
    
    | Field | Required | Notes |
    |-------|----------|-------|
    | `file` | one of file/source_url | Source media to dub (audio or video), up to 3 GiB |
    | `source_url` | one of file/source_url | Public URL to fetch the source media from |
    | `source_language` | no | ISO 639 code (e.g. `en`). Omit to auto-detect — the detected language is reported on the source transcript's `language` field |
    | `reference` | no | Free-form label to identify the project on your end (max 500 chars) |
    | `model_id` | no | `dubbing_v2` (default) |
    | `target_language` | no | Optionally queue the first language target at creation; add more with `language.create` |
    | `keyterms` | no | Terms to bias transcription/translation toward (product/brand names). Up to 1000 terms; each at most 50 chars and 5 words; `<>{}[]\` not allowed. Repeat the field once per term in multipart |
    
    ## Editing the Source Transcript
    
    Once the project is `ready`, read the transcript, then correct it before adding languages. Every edit bumps the project's `revision`. Each segment has a stable `id` used to edit or delete it. (Enterprise workspaces only.)
    
    ```python
    # Read the source transcript
    transcript = elevenlabs.dubbing.project.transcript.get(project_id)
    
    # Correct a segment's text — send only the fields to change (text, speaker_id, start_s, end_s)
    elevenlabs.dubbing.project.transcript.update_segment(
        project_id,
        segment_id=transcript.segments[0].id,
        text="Welcome to our latest product demo.",
    )
    
    # Add a segment (reuse an existing speaker_id so it's dubbed with that speaker's voice)
    added = elevenlabs.dubbing.project.transcript.create_segment(
        project_id,
        text="Thanks for watching.",
        speaker_id=transcript.segments[0].speaker_id,
        start_s=40.0,
        end_s=42.0,
    )
    
    # Delete a segment
    elevenlabs.dubbing.project.transcript.delete_segment(project_id, segment_id=added.segment.id)
    ```
    
    Via the CLI: `elevenlabs dubbing project transcript get --project-id proj_...`, then update a segment with only the changed fields (`--text`, `--speaker-id`, `--start-s`, `--end-s`):
    
    ```bash
    elevenlabs dubbing project transcript update_segment \
      --project-id proj_... --segment-id seg_... \
      --text "Welcome to our latest product demo."
    ```
    
    ## Refining Translations and Regenerating
    
    A language's transcript pairs each source segment with its `translation` (`null` = not yet translated; segment ids match the source). Edit a single translation, then regenerate. (Enterprise workspaces only.)
    
    ```python
    # Read the language's translations
    target = elevenlabs.dubbing.project.language.transcript.get(project_id, language_id)
    
    # Refine a single translation (pass translation=None to clear it and mark for re-translation)
    elevenlabs.dubbing.project.language.transcript.update_segment(
        project_id,
        language_id,
        segment_id=target.segments[0].id,
        translation="Bienvenido a nuestra última demostración de producto.",
    )
    
    # Regenerate the dub from the current transcript (charged like a generation)
    elevenlabs.dubbing.project.language.transcript.regenerate(project_id, language_id)
    ```
    
    Via the CLI: `elevenlabs dubbing project language transcript update_segment --project-id proj_... --language-id lang_... --segment-id seg_... --translation "..."`, then `elevenlabs dubbing project language transcript regenerate --project-id proj_... --language-id lang_...` (returns `202 Accepted`).
    
    A translation edit affects only that language. After the edit, a `completed` language becomes `stale` — it keeps serving its previous output until you regenerate. Poll until `completed`; `output_revision` then equals `revision` and `outputs.lossless_audio` reflects the current transcript.
    
    ## Dubbing into Multiple Languages
    
    Add one language target per language — each generates independently. Track them all with `language.list` instead of polling one by one:
    
    ```python
    for lang in ["es", "fr", "de", "ja"]:
        elevenlabs.dubbing.project.language.create(project_id, target_language=lang)
    
    while True:
        result = elevenlabs.dubbing.project.language.list(project_id)
        if not any(l.status in ("queued", "processing") for l in result.languages):
            break
        time.sleep(5)
    ```
    
    ## States
    
    **Project:**
    
    | Status | Meaning |
    |--------|---------|
    | `queued` | Created; source fetch + preparation enqueued |
    | `preparing` | Preparation (transcription) running |
    | `ready` | Source transcript available; add/generate languages. Projects **stay** `ready` — per-language progress lives on the languages |
    | `failed` | Preparation failed (e.g. source couldn't be fetched or decoded) |
    
    **Language:**
    
    | Status | Meaning |
    |--------|---------|
    | `queued` | Waiting on the project becoming `ready`, or on a generation worker |
    | `processing` | The dub is being generated |
    | `completed` | Finished; `outputs` populated with a signed download URL (valid ~1 hour — re-fetch for a fresh one) |
    | `stale` | Previously completed, but the transcript changed; keeps the last output until regenerated |
    | `failed` | Generation failed |
    
    You can add a language before the project is `ready` — it stays `queued` and starts automatically once the project becomes `ready`. Adding a language accepts optional `model_id` (defaults to the project's) and `voice_settings` (e.g. `{"cloning_strength": 7}`, range 0–10, default 7 — controls how strongly dubbed speakers clone the source voices).
    
    ## Error Handling
    
    - **401**: Invalid API key
    - **409 Conflict** on regenerate: The project isn't `ready` or the language isn't settled (e.g. already generating) — wait and retry
    - **Expired download URL**: `outputs.lossless_audio` is signed and valid ~1 hour; re-fetch the language for a fresh URL
    - **Transcript editing / regeneration unavailable**: These endpoints are enterprise-only — on other plans, create the project with a finalized source and add languages directly
    
    ## References
    
    - [Installation Guide](references/installation.md)
    - [API Reference](references/api-reference.md) — every endpoint with full request/response schemas and SDK method names
    
    Original on GitHub
  • musicGenerate music using ElevenLabs Music API. Use when creating instrumental tracks, songs with lyrics, background music, jingles, or any AI-generated music composition. Supports prompt-based generation, composition plans for granular control, and detailed output with metadata.Read
    ---
    name: music
    description: Generate music using ElevenLabs Music API. Use when creating instrumental tracks, songs with lyrics, background music, jingles, or any AI-generated music composition. Supports prompt-based generation, composition plans for granular control, and detailed output with metadata.
    license: MIT
    compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
    metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
    ---
    
    # ElevenLabs Music Generation
    
    Generate music from text prompts - supports instrumental tracks, songs with lyrics, and fine-grained control via composition plans.
    
    > **Setup, before writing any code:** if `ELEVENLABS_API_KEY` isn't set for this project, run `npx -y @elevenlabs/cli@latest onboard init` from the app's folder first and follow what it prints: it signs the user in and stores a key for the project, so nobody copies one by hand. Otherwise see the [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only.
    
    All examples below use `music_v2_5`, the most advanced generation model. Pass `music_v2` or
    `music_v1` only when an older model is explicitly requested.
    
    ## Quick Start
    
    ### Python
    
    ```python
    from elevenlabs import ElevenLabs
    
    client = ElevenLabs()
    
    audio = client.music.compose(
        prompt="A chill lo-fi hip hop beat with jazzy piano chords",
        music_length_ms=30000,
        model_id="music_v2_5",
    )
    
    with open("output.mp3", "wb") as f:
        for chunk in audio:
            f.write(chunk)
    ```
    
    ### TypeScript
    
    ```typescript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    import { createWriteStream } from "fs";
    
    const client = new ElevenLabsClient();
    const audio = await client.music.compose({
      prompt: "A chill lo-fi hip hop beat with jazzy piano chords",
      musicLengthMs: 30000,
      modelId: "music_v2_5",
    });
    audio.pipe(createWriteStream("output.mp3"));
    ```
    
    ### CLI
    
    ```bash
    elevenlabs music compose \
      --prompt "A chill lo-fi beat" \
      --music-length-ms 30000 \
      --model-id music_v2_5 \
      --output output.mp3
    ```
    
    ## Methods
    
    | Method | Description |
    |--------|-------------|
    | `music.compose` | Generate audio from a prompt or composition plan |
    | `music.stream` | Stream audio chunks as they are generated (paid plans) |
    | `music.composition_plan.create` | Generate a structured plan for fine-grained control |
    | `music.compose_detailed` | Generate audio + composition plan + metadata; pass `store_for_inpainting=True` to enable inpainting |
    | `music.compose_detailed_stream` | Stream audio plus composition plan, metadata, and optional word timestamps as Server-Sent Events |
    | `music.video_to_music` | Generate background music from one or more uploaded video files |
    | `music.upload` | Upload an audio file for later inpainting workflows, optionally extracting its composition plan or word-level timestamps |
    | `music.finetunes.list` | List accessible music finetunes |
    | `music.finetunes.create` | Train a music finetune from uploaded audio |
    | `music.finetunes.get` | Retrieve finetune status and metadata |
    | `music.finetunes.update` | Update finetune metadata or visibility |
    | `music.finetunes.delete` | Delete a music finetune |
    
    See [API Reference](references/api_reference.md) for full parameter details.
    
    `music.upload` is available to enterprise clients with access to the inpainting feature.
    
    ## Music Finetunes
    
    Create a finetune from training audio with
    [`POST /v1/music/finetunes`](https://elevenlabs.io/docs/api-reference/music/finetunes/create),
    then poll the [get endpoint](https://elevenlabs.io/docs/api-reference/music/finetunes/get) until
    its status is `completed`. Pass the returned `id` as `finetune_id` when composing music.
    
    Use the [list](https://elevenlabs.io/docs/api-reference/music/finetunes/list),
    [update](https://elevenlabs.io/docs/api-reference/music/finetunes/update), and
    [delete](https://elevenlabs.io/docs/api-reference/music/finetunes/delete) endpoints to manage
    accessible finetunes.
    
    ## Video to Music
    
    Generate background music from uploaded video clips via
    [`POST /v1/music/video-to-music`](https://elevenlabs.io/docs/api-reference/music/video-to-music)
    (`client.music.video_to_music`). This is separate from prompt-based
    [`music.compose`](https://elevenlabs.io/docs/api-reference/music/compose) (`POST /v1/music`).
    
    The API combines videos in order, accepts an optional natural-language description, and lets you
    steer style with up to 10 tags such as `upbeat` or `cinematic`. This endpoint still defaults to
    `music_v1`; pass `model_id="music_v2_5"` to use the most advanced model.
    
    Three optional flags shape the result: `use_audio_as_reference` takes genre, BPM, instrumentation,
    and mood from the video's own audio, `add_sfx` adds sound effects that match the video, and
    `separate_tracks` returns a ZIP with the music and sound effects as separate files for mixing.
    
    ### Python
    
    ```python
    from elevenlabs import ElevenLabs
    
    client = ElevenLabs()
    
    audio = client.music.video_to_music(
        videos=["trailer.mp4"],
        description="Build suspense, then resolve with a warm cinematic finish.",
        tags=["cinematic", "suspenseful", "uplifting"],
        model_id="music_v2_5",
    )
    
    with open("video-score.mp3", "wb") as f:
        for chunk in audio:
            f.write(chunk)
    ```
    
    ### TypeScript
    
    ```typescript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    import { createReadStream, createWriteStream } from "fs";
    
    const client = new ElevenLabsClient();
    
    const audio = await client.music.videoToMusic({
      videos: [createReadStream("trailer.mp4")],
      description: "Build suspense, then resolve with a warm cinematic finish.",
      tags: ["cinematic", "suspenseful", "uplifting"],
      modelId: "music_v2_5",
    });
    
    audio.pipe(createWriteStream("video-score.mp3"));
    ```
    
    ### CLI
    
    ```bash
    elevenlabs music video_to_music \
      --videos trailer.mp4 \
      --description "Build suspense, then resolve with a warm cinematic finish." \
      --tags cinematic \
      --model-id music_v2_5 \
      --output video-score.mp3
    ```
    
    The CLI currently accepts one `--videos` file and one `--tags` value per request; use the Python
    or TypeScript SDK to send multiple videos or tags.
    
    Constraints from the current API schema:
    
    - Upload 1-10 video files per request
    - Keep total combined upload size at or below 200 MB
    - Keep total combined video duration at or below 600 seconds
    - Use `description` for high-level musical direction and `tags` for concise style cues
    
    ## Composition Plans
    
    `music_v2_5` composition plans are an ordered list of `chunks`. Each chunk specifies its own
    `text` (section label, lyrics, inline cues), `duration_ms`, `positive_styles`, `negative_styles`,
    and `context_adherence` (`low`, `medium`, or `high`, default `high`). Up to 30 chunks per plan,
    each 3,000–120,000 ms, total length 3 s to 10 minutes. Each chunk's `text` supports up to 6,132
    characters, with up to 30 lines of 200 characters each.
    
    Generate a plan first, edit it, then compose:
    
    ```python
    plan = client.music.composition_plan.create(
        prompt="An epic orchestral piece building to a climax",
        music_length_ms=60000,
        model_id="music_v2_5",
    )
    
    # Edit chunks in place
    plan["chunks"][0]["text"] = "[Intro]\nQuiet strings rising"
    
    audio = client.music.compose(
        composition_plan=plan,
        model_id="music_v2_5",
    )
    ```
    
    ```typescript
    const plan = await client.music.compositionPlan.create({
      prompt: "An epic orchestral piece building to a climax",
      musicLengthMs: 60000,
      modelId: "music_v2_5",
    });
    
    plan.chunks[0].text = "[Intro]\nQuiet strings rising";
    
    const audio = await client.music.compose({
      compositionPlan: plan,
      modelId: "music_v2_5",
    });
    ```
    
    Or hand-build a plan to control lyrics and style per section:
    
    ```python
    composition_plan = {
        "chunks": [
            {
                "text": "[Verse]\nWalking down an empty street",
                "duration_ms": 15000,
                "positive_styles": ["pop", "upbeat", "female vocals", "acoustic guitar"],
                "negative_styles": ["dark", "slow"],
                "context_adherence": "high",
            },
            {
                "text": "[Chorus]\nThis is my moment",
                "duration_ms": 15000,
                "positive_styles": ["powerful vocals", "full band"],
                "negative_styles": [],
                "context_adherence": "high",
            },
        ]
    }
    
    audio = client.music.compose(composition_plan=composition_plan, model_id="music_v2_5")
    ```
    
    ```typescript
    const compositionPlan = {
      chunks: [
        {
          text: "[Verse]\nWalking down an empty street",
          durationMs: 15000,
          positiveStyles: ["pop", "upbeat", "female vocals", "acoustic guitar"],
          negativeStyles: ["dark", "slow"],
          contextAdherence: "high",
        },
        {
          text: "[Chorus]\nThis is my moment",
          durationMs: 15000,
          positiveStyles: ["powerful vocals", "full band"],
          negativeStyles: [],
          contextAdherence: "high",
        },
      ],
    };
    
    const audio = await client.music.compose({
      compositionPlan,
      modelId: "music_v2_5",
    });
    ```
    
    Put broader characteristics (genre, instrumentation, vocal style) in `positive_styles`, not in
    `text`. The first chunk's styles set the overall tone — include 6–7 styles there.
    
    ## Output Formats
    
    Use the `output_format` query parameter on compose, detailed compose, or stream requests to select
    the generated audio format. `auto` chooses a model-appropriate MP3 format; for `music_v2` and
    `music_v2_5`, it selects `mp3_48000_192`. Higher-bitrate MP3 options include
    `mp3_48000_240` and `mp3_48000_320`.
    
    ## Streaming
    
    For paid plans, stream audio chunks as they are generated instead of waiting for the full file:
    
    ```python
    from io import BytesIO
    
    stream = client.music.stream(
        prompt="A driving synthwave track with arpeggiated leads",
        music_length_ms=30000,
        model_id="music_v2_5",
    )
    
    buffer = BytesIO()
    for chunk in stream:
        if chunk:
            buffer.write(chunk)
    ```
    
    ```typescript
    const stream = await client.music.stream({
      prompt: "A driving synthwave track with arpeggiated leads",
      musicLengthMs: 30000,
      modelId: "music_v2_5",
    });
    
    const chunks: Buffer[] = [];
    for await (const chunk of stream) {
      chunks.push(chunk);
    }
    ```
    
    ### Detailed streaming
    
    Use detailed streaming when the application needs generated music metadata while audio is still
    arriving. `POST /v1/music/detailed/stream` accepts the same prompt or composition-plan body as
    detailed compose, streams `text/event-stream`, and can include word timestamps with
    `with_timestamps`.
    
    ```bash
    elevenlabs music compose_detailed_stream \
      --prompt "A bright indie pop hook with warm guitars" \
      --music-length-ms 30000 \
      --model-id music_v2_5 \
      --with-timestamps true \
      --output-format auto
    ```
    
    ## Inpainting
    
    Inpainting edits or extends a stored song by mixing **audio reference chunks** (unchanged slices
    of a stored song) with new **generation chunks** in a single composition plan. 
    
    Step 1 — get a `song_id`, either by storing a fresh generation or uploading existing audio:
    
    ```python
    # Option A: keep a generation for later editing
    result = client.music.compose_detailed(
        prompt="An upbeat pop song with verse and chorus",
        music_length_ms=60000,
        model_id="music_v2_5",
        store_for_inpainting=True,
    )
    song_id = result.song_id
    
    # Option B: upload an existing track and extract its plan
    uploaded = client.music.upload(
        file=open("my-song.mp3", "rb"),
        extract_composition_plan="music_v2_5",
    )
    song_id = uploaded.song_id
    composition_plan = uploaded.composition_plan
    ```
    
    ```typescript
    import { createReadStream } from "fs";
    
    // Option A: keep a generation for later editing
    const result = await client.music.composeDetailed({
      prompt: "An upbeat pop song with verse and chorus",
      musicLengthMs: 60000,
      modelId: "music_v2_5",
      storeForInpainting: true,
    });
    let songId = result.songId;
    
    // Option B: upload an existing track and extract its plan
    const uploaded = await client.music.upload({
      file: createReadStream("my-song.mp3"),
      extractCompositionPlan: "music_v2_5",
    });
    songId = uploaded.songId;
    const compositionPlan = uploaded.compositionPlan;
    ```
    
    Step 2 — compose a plan that references the stored audio and regenerates the part you want to
    change:
    
    ```python
    plan = {
        "chunks": [
            {"song_id": song_id, "range": {"start_ms": 0, "end_ms": 30000}},
            {
                "text": "[Chorus]\nWe're rising up tonight",
                "duration_ms": 30000,
                "positive_styles": ["bigger drums", "layered vocals", "anthemic"],
                "negative_styles": ["sparse"],
                "context_adherence": "high",
            },
        ]
    }
    
    audio = client.music.compose(composition_plan=plan, model_id="music_v2_5")
    ```
    
    ```typescript
    const plan = {
      chunks: [
        { songId, range: { startMs: 0, endMs: 30000 } },
        {
          text: "[Chorus]\nWe're rising up tonight",
          durationMs: 30000,
          positiveStyles: ["bigger drums", "layered vocals", "anthemic"],
          negativeStyles: ["sparse"],
          contextAdherence: "high",
        },
      ],
    };
    
    const audio = await client.music.compose({
      compositionPlan: plan,
      modelId: "music_v2_5",
    });
    ```
    
    To match the feel of a stored slice without copying it, attach a `conditioning_ref` (up to
    30,000 ms) plus a `condition_strength` of `low`, `medium`, `high`, or `xhigh` to a generation
    chunk. Conditioning placed on the first chunk influences every later chunk.
    
    See [API Reference](references/api_reference.md) for the full inpainting parameter list.
    
    ## Content Restrictions
    
    - Cannot reference specific artists, bands, or copyrighted lyrics
    - `bad_prompt` errors include a `prompt_suggestion` with alternative phrasing
    - `bad_composition_plan` errors include a `composition_plan_suggestion`
    
    ## Error Handling
    
    ```python
    try:
        audio = client.music.compose(prompt="...", music_length_ms=30000)
    except Exception as e:
        print(f"API error: {e}")
    ```
    
    ```typescript
    try {
      const audio = await client.music.compose({
        prompt: "...",
        musicLengthMs: 30000,
      });
    } catch (err) {
      console.error("API error:", err);
    }
    ```
    
    Common errors: 401 (invalid key), 422 (invalid params), 429 (rate limit).
    
    ## References
    
    - [Installation Guide](references/installation.md)
    - [API Reference](references/api_reference.md)
    
    Original on GitHub
  • onboardingAdd ElevenLabs to a project from scratch. Use when the user wants to start using ElevenLabs, add text to speech, speech to text, voice agents, real-time voice for their own agent runtime (Speech Engine), sound effects, music, voice changing, voice isolation, or dubbing to their app, or get set up with an ElevenLabs account or API key for the first time. Handles sign-in, API key creation, SDK install and a test request end to end; the user leaves the conversation once, to approve a browser page. Not for fixing a key that stopped working or placing one an administrator issued (use setup-api-key).Read
    ---
    name: onboarding
    description: Add ElevenLabs to a project from scratch. Use when the user wants to start using ElevenLabs, add text to speech, speech to text, voice agents, real-time voice for their own agent runtime (Speech Engine), sound effects, music, voice changing, voice isolation, or dubbing to their app, or get set up with an ElevenLabs account or API key for the first time. Handles sign-in, API key creation, SDK install and a test request end to end; the user leaves the conversation once, to approve a browser page. Not for fixing a key that stopped working or placing one an administrator issued (use setup-api-key).
    license: MIT
    compatibility: Requires internet access to elevenlabs.io and api.elevenlabs.io, and Node.js for npx (or the ElevenLabs CLI installed another way). Where no browser can reach the machine (SSH, containers, cloud agents), the user adds the key by hand and the flow continues.
    ---
    
    # ElevenLabs Onboarding
    
    The ElevenLabs CLI runs the whole setup and tells you each step. From the
    folder of the app that will use the key (in a monorepo, that app's folder),
    run:
    
    ```bash
    npx -y @elevenlabs/cli@latest onboard init
    ```
    
    Every `onboard` command prints JSON. Act on it, and keep going until `next`
    is `done`:
    
    - `next`: what to do now, spelled out in `instruction`.
    - `run`: the exact next command. Run it as given.
    - `say`: what to tell the user. Tell them as written.
    - `details`: facts for the next step, such as how the app loads the key and
      the SDK install command.
    - `rules`: rules for the whole setup. Follow them.
    
    What each `next` means:
    
    - `run`: run `run`.
    - `wait`: the approval page is open. Show `say` (it has the link), then run
      `run` again right away, without ending your turn. Whenever you give the
      user the link, copy it whole, exactly as printed: a shortened link doesn't
      work.
    - `ask_what_to_build`: tell the user `say`, which asks what they want to
      build or whether to just add the SDK, and wait for their answer before
      writing any code. Ask it even if they said earlier, and don't pick a product
      for them. Once they answer, run `run`.
    - `write_code`: write what they asked for, as `instruction` says: the SDK
      alone, or the integration, with the skill that fits from
      `details.fetch_skills` when there is one. If it needs something made in
      ElevenLabs first, like an agent, make it yourself with the key (load the
      env file in the command, never print it), not a script for the user to run.
      Load the key the way `details.load_key` says, then run `run`.
    - `load_key`: the code already uses the SDK, so there is nothing to build.
      Make sure the app loads the key the way `details.load_key` says, change
      nothing else, then run `run`.
    - `fix`: do what `say` describes, then run `run`. If it needs the user's
      shell, settings or secrets, ask them instead, as for `ask_user`.
    - `ask_user`: ask the user what `say` says and wait for their answer, then
      run `run`, unless they said no.
    - `offer_test`: say what you set up and anything the user still has to do
      themselves, like installing the packages you added. Then end your reply
      with the question in `say`. Run `run` only if they say yes.
    - `done`: tell the user `say`, then anything they still have to do
      themselves, in one or two concrete steps.
    
    ## Rules
    
    - Never open, print or `cat` the env file; `onboard check` verifies it.
    - Never ask for an API key in chat, and never accept one pasted there. If one
      appears, tell the user to rotate it on the API keys page.
    - Never put the key in code; the SDK reads `ELEVENLABS_API_KEY`.
    - Never run `git clean` or delete untracked files: the key is in one.
    - Don't write or change any code until a reply's `next` is `write_code` or
      `load_key`.
    - Put anything you download only to read, like a package or an archive, in a
      temp folder, never in the project.
    - Don't run the app (or its start script) or make speech to try it until the
      user says yes to the test request. API calls that what they asked for
      needs, like creating an agent, are fine. Once the code is written, run
      `onboard check` before you finish: it verifies the key and the code.
    
    ## No npx
    
    If `npx` isn't available (Node.js isn't installed), install the CLI (or
    update it if it's already installed: `brew upgrade elevenlabs`, or run the
    installer again), then run `elevenlabs onboard init` instead. Everything after
    that is the same.
    
    - macOS or Linux: `brew install elevenlabs/tap/elevenlabs`, or
      `curl --proto '=https' --tlsv1.2 -LsSf https://github.com/elevenlabs/cli/releases/latest/download/elevenlabs-cli-installer.sh | sh`
    - Windows: `powershell -ExecutionPolicy ByPass -c "irm https://github.com/elevenlabs/cli/releases/latest/download/elevenlabs-cli-installer.ps1 | iex"`
    
    If `elevenlabs` isn't found after the installer ran, use
    `~/.cargo/bin/elevenlabs` in its place, here and in every `run` the CLI gives.
    
    Original on GitHub
  • realtimeUse ElevenLabs Realtime models (`eleven_realtime_v1_mini`, `eleven_realtime_v1_max`), the low-latency models built for voice conversation. Use when the user wants to create an ElevenAgents agent on a Realtime model or switch an existing agent's Voice mode to Realtime, set `conversation_config.realtime_model`, choose between Realtime Mini and Max, or debug a Realtime agent that won't publish or transfer. Also use when migrating a voice app built on OpenAI's Realtime API and SDK (`@openai/agents-realtime`, `gpt-realtime`) to the OpenAI-compatible ElevenLabs Realtime API, including the `client_secrets` mint, the WebSocket transport, and audio after leaving WebRTC (closes with 4001/4003, no audio). Not for choosing an agent's LLM or TTS model in Custom mode, Scribe realtime speech-to-text, or migrating Chat Completions or Responses API text calls.Read
    ---
    name: realtime
    description: Use ElevenLabs Realtime models (`eleven_realtime_v1_mini`, `eleven_realtime_v1_max`), the low-latency models built for voice conversation. Use when the user wants to create an ElevenAgents agent on a Realtime model or switch an existing agent's Voice mode to Realtime, set `conversation_config.realtime_model`, choose between Realtime Mini and Max, or debug a Realtime agent that won't publish or transfer. Also use when migrating a voice app built on OpenAI's Realtime API and SDK (`@openai/agents-realtime`, `gpt-realtime`) to the OpenAI-compatible ElevenLabs Realtime API, including the `client_secrets` mint, the WebSocket transport, and audio after leaving WebRTC (closes with 4001/4003, no audio). Not for choosing an agent's LLM or TTS model in Custom mode, Scribe realtime speech-to-text, or migrating Chat Completions or Responses API text calls.
    license: MIT
    ---
    
    # ElevenLabs Realtime
    
    Realtime models are ElevenLabs models built for low-latency voice conversation. There are two ways to use them:
    
    - **An ElevenAgents agent** (an agent ID, the dashboard, or an `elevenlabs agents` project with `agent_configs/`): set the agent's Voice mode to Realtime with `conversation_config.realtime_model`, and ElevenLabs manages its LLM, TTS model, and ASR settings. See [Run an agent on a Realtime model](#run-an-agent-on-a-realtime-model).
    - **A browser app built on OpenAI's Realtime SDK**: keep the SDK and point it at the OpenAI-compatible ElevenLabs Realtime API, `wss://api.elevenlabs.io/v1/convai/realtime`. See [Migrate an app from OpenAI Realtime](#migrate-an-app-from-openai-realtime).
    
    When the user is starting from scratch and it's unclear which they want, ask. An agent brings a managed system prompt, tools, knowledge base, widget, and telephony; the Realtime API is the quicker path for code that already speaks OpenAI's realtime events. For agent features beyond the voice mode (prompt, tools, workflows, widget), use the `agents` skill.
    
    ## Models
    
    | Model                   | Model ID                  | Optimized for                                                                             |
    | ----------------------- | ------------------------- | ----------------------------------------------------------------------------------------- |
    | Eleven Realtime v1 Mini | `eleven_realtime_v1_mini` | Lowest latency and cost. The default.                                                     |
    | Eleven Realtime v1 Max  | `eleven_realtime_v1_max`  | Reasons before it responds, for complex instructions and multi-step tool use. Costs more. |
    
    Start with Mini. Choose Max when the work involves long or complex instructions or multi-step tool calls and Mini's responses aren't good enough, or when an app being migrated was on a larger OpenAI model for reasoning-heavy work. Agents add compliance and data residency constraints on the choice; see [Choose the model](#step-2-choose-the-model).
    
    ## Run an agent on a Realtime model
    
    Every voice agent has a Voice mode. In **Custom** mode the agent's LLM, TTS model, and ASR settings are chosen individually. In **Realtime** mode the agent runs on a Realtime model and ElevenLabs manages those settings. The mode is one field: `conversation_config.realtime_model` set to a model ID means Realtime, `null` means Custom.
    
    The system prompt, first message, voice, languages, temperature, tools, knowledge base, and conversation flow settings (turn eagerness, timeouts, interruptions) work the same in both modes.
    
    ### Before you start
    
    - These steps call the ElevenLabs API. The CLI and SDKs read `ELEVENLABS_API_KEY` from the environment; if it isn't set, use the `setup-api-key` skill. Never ask the user to paste the key into the conversation.
    - Realtime models are rolling out gradually. If the API rejects `realtime_model` with `Realtime models are not available for this workspace.`, the workspace doesn't have access yet. Stop and tell the user; no configuration change fixes it.
    
    ### Step 1: Check the agent can use Realtime
    
    For an existing agent, fetch its configuration first (`elevenlabs agents pull --agent <agent-id>`, or `agents.get` in the SDK). Keep the agent in Custom mode, and tell the user why, if any of these apply:
    
    | Blocker                                                                                     | Where to look                                                                                                              |
    | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
    | Custom LLM or Amazon Bedrock                                                                | `conversation_config.agent.prompt.llm` is `custom-llm` or a Bedrock model                                                  |
    | Created through another ElevenLabs product, such as Speech Engine                           | Ask the user if unsure; only ElevenAgents agents can use a Realtime model                                                  |
    | Needs a specific LLM or TTS model                                                           | Ask the user whether the current `agent.prompt.llm` or `tts.model_id` is a requirement                                     |
    | Workflow nodes override a [managed setting](#managed-settings) or `agent.prompt.max_tokens` | Overrides on the nodes under the agent's `workflow`                                                                        |
    | Transfers to or from agents in Custom mode                                                  | `transfer_to_agent` tools and `agent_transfer` workflow nodes on this agent, and on any agent that transfers into this one |
    | Text-only agent                                                                             | `conversation_config.conversation.text_only` is `true`; Realtime models are built for voice                                |
    
    A workflow override can be removed and a transfer partner can be switched to Realtime too, but only with the user's agreement: both change how other parts of their setup behave.
    
    ### Step 2: Choose the model
    
    Start with `eleven_realtime_v1_mini`. Choose `eleven_realtime_v1_max` when the agent follows long or complex instructions or relies on multi-step tool calls and Mini isn't good enough, or when the agent must comply with Zero Retention Mode or HIPAA. Then check availability:
    
    | Environment                   | `eleven_realtime_v1_mini` | `eleven_realtime_v1_max`   |
    | ----------------------------- | ------------------------- | -------------------------- |
    | US                            | Available                 | Available                  |
    | EU data residency             | Available                 | Available                  |
    | India data residency          | Available                 | Not available              |
    | Singapore data residency      | Available                 | Not available              |
    | Zero Retention Mode and HIPAA | Not available             | Available in the US and EU |
    
    Zero Retention Mode applies whether it's enforced on the workspace or enabled on the agent. Ask the user when the workspace's data residency or compliance settings aren't clear from the project. If neither model is available for the agent's environment, keep it in Custom mode.
    
    Max also needs `agent.prompt.max_tokens` of at least 1024, or `-1` for no limit.
    
    ### Step 3: Set the model
    
    On an existing agent, change only `realtime_model`; leave the rest of the configuration alone.
    
    With the CLI, pull the agent, set the field in its file under `agent_configs/`, and push:
    
    ```bash
    elevenlabs agents pull --agent <agent-id>
    ```
    
    ```json
    {
      "conversation_config": {
        "realtime_model": "eleven_realtime_v1_mini"
      }
    }
    ```
    
    ```bash
    elevenlabs agents push --agent <agent-id>
    ```
    
    With the SDKs:
    
    ```python
    from elevenlabs import ElevenLabs
    
    elevenlabs = ElevenLabs()
    
    elevenlabs.conversational_ai.agents.update(
        agent_id="<agent-id>",
        conversation_config={
            "realtime_model": "eleven_realtime_v1_mini",
        },
    )
    ```
    
    ```typescript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    
    const elevenlabs = new ElevenLabsClient();
    
    await elevenlabs.conversationalAi.agents.update("<agent-id>", {
      conversationConfig: {
        realtimeModel: "eleven_realtime_v1_mini",
      },
    });
    ```
    
    For a new agent, put `realtime_model` (`realtimeModel` in TypeScript) in the `conversation_config` passed to `agents.create`, alongside the prompt, first message, and voice. Don't pick an LLM or TTS model for it; a Realtime agent ignores them.
    
    To turn Realtime off, set `realtime_model` to `null` (`None` in Python). The agent goes back to the LLM, TTS model, and ASR settings stored on it.
    
    ### Managed settings
    
    While a Realtime model is set, the agent ignores the stored values of these fields in `conversation_config`. They come back into use when the agent switches to Custom, so don't clear them, and don't edit them to tune a Realtime agent:
    
    | Setting             | Field                                     |
    | ------------------- | ----------------------------------------- |
    | LLM                 | `agent.prompt.llm`                        |
    | Reasoning effort    | `agent.prompt.reasoning_effort`           |
    | Thinking budget     | `agent.prompt.thinking_budget`            |
    | Parallel tool calls | `agent.prompt.enable_parallel_tool_calls` |
    | Cascade timeout     | `agent.prompt.cascade_timeout_seconds`    |
    | TTS model           | `tts.model_id`                            |
    | Expressive mode     | `tts.expressive_mode`                     |
    | ASR provider        | `asr.provider`                            |
    
    `agent.prompt.max_tokens` keeps applying. Temperature does too, but the dashboard hides it in Realtime mode, so change it through the CLI or API. If the Realtime model errors or doesn't respond in time, the agent falls back to its backup LLMs (LLM cascading); the Realtime model sets how long it waits.
    
    ### Step 4: Publish and verify
    
    The Step 1 rules are checked when the agent is published, not when a draft is saved: a draft that breaks one saves, then publishing fails with an error naming the setting to change.
    
    1. Fetch the agent and confirm `conversation_config.realtime_model` holds the chosen model. If the push, update, or the user's publish in the dashboard fails, match the error below.
    2. Find every agent that transfers into this one and confirm it's in Realtime mode too. A Custom-mode agent's transfer into a Realtime agent is only caught when that agent is next published; until then the transfer is rejected mid-conversation and the caller stays with the current agent.
    3. Have the user hold a test conversation.
    
    ### Errors
    
    | Error                                                                   | Cause and fix                                                                                         |
    | ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
    | `Realtime models are not available for this workspace.`                 | The workspace doesn't have access yet. Nothing to fix in the configuration.                           |
    | `Transfers between default and Realtime model modes are not supported.` | A transfer to or from a Custom-mode agent ("default" means Custom). Put both agents in the same mode. |
    | An error naming the Realtime model                                      | The model isn't available under the agent's compliance or data residency settings. See Step 2.        |
    | Publishing fails naming a setting                                       | One of the Step 1 blockers or Max's token limit. Change that setting, or keep the agent in Custom.    |
    
    ### Pricing
    
    A Realtime agent is billed the standard ElevenAgents per-minute rate plus its input, cached input, and output tokens at the Realtime model's rates; Max costs more per token than Mini. Point the user to https://elevenlabs.io/pricing rather than quoting rates.
    
    ## Migrate an app from OpenAI Realtime
    
    ElevenLabs Realtime speaks the same wire protocol as OpenAI's GA Realtime API: the same client and server events over WebSocket. An app built on OpenAI's Realtime SDK keeps that SDK; the migration changes where credentials are minted, which URL the transport connects to, and the model, voice, and session configuration.
    
    The one structural change: ElevenLabs Realtime is WebSocket-only. In a browser, `RealtimeSession` defaults to WebRTC, which wires the microphone and speaker for you. After switching to the WebSocket transport, the app has to capture and play audio itself.
    
    This covers apps where the browser opens the realtime connection. It doesn't cover apps where a server opens the connection and streams audio to its clients over its own protocol, such as a bespoke WebSocket relay or a telephony bridge. Connecting to ElevenLabs Realtime from a server should work, but this skill has no steps for it. Tell the user so, and suggest they open an issue or a pull request at https://github.com/elevenlabs/skills describing their setup, rather than guessing at the server-side migration.
    
    ### Step 0: Inventory the app
    
    Find every place the app touches OpenAI Realtime before editing anything:
    
    ```bash
    grep -rnE "api\.openai\.com/v1/realtime|client_secrets|realtime/sessions|@openai/agents-realtime|@openai/agents/realtime|gpt-realtime|gpt-4o(-mini)?-realtime|OpenAIRealtimeWebRTC|OpenAIRealtimeWebSocket|RealtimeSession|OPENAI_API_KEY" --include='*.{ts,tsx,js,jsx,mjs,py}' .
    ```
    
    Classify what you find, because each part migrates differently:
    
    - **Credential minting**: a server route that calls `/v1/realtime/client_secrets` (GA) or `/v1/realtime/sessions` (beta). Migrate in Step 1.
    - **Browser session**: `RealtimeSession`, `OpenAIRealtimeWebRTC`, `OpenAIRealtimeWebSocket`, or a hand-rolled `WebSocket`/`RTCPeerConnection`. Migrate in Steps 2 to 4.
    - **Server-side realtime connections**: a Node or Python process that opens the realtime socket itself, for example a relay that streams audio to clients over its own WebSocket protocol, a telephony bridge, `openai` Python's `client.realtime.connect()`, or a `RealtimeSession` running in Node with a full API key. Not covered; see the scope above. If the app also has a browser session, migrate only that part, and only with the user's agreement.
    - **Features beyond a voice conversation**: function tools, handoffs, guardrails, MCP servers, image input, SIP. ElevenLabs documents the voice conversation flow only. Leave this code in place, list it for the user as unverified against ElevenLabs, and have them test it.
    
    `OPENAI_API_KEY` often serves other OpenAI calls too. Remove it only if nothing else uses it.
    
    ### Step 1: Mint the client secret from ElevenLabs
    
    The browser never sees the ElevenLabs API key. A server route mints a short-lived client secret and returns only its `value`.
    
    | What            | OpenAI                                                   | ElevenLabs                                                         |
    | --------------- | -------------------------------------------------------- | ------------------------------------------------------------------ |
    | URL             | `POST https://api.openai.com/v1/realtime/client_secrets` | `POST https://api.elevenlabs.io/v1/convai/realtime/client_secrets` |
    | Auth header     | `Authorization: Bearer $OPENAI_API_KEY`                  | `xi-api-key: $ELEVENLABS_API_KEY`                                  |
    | `session.model` | `gpt-realtime`, …                                        | `eleven_realtime_v1_mini` or `eleven_realtime_v1_max`              |
    | Response        | `{ value, expires_at, session }`                         | Same shape                                                         |
    
    The request body keeps the GA shape, `{ session, expires_after }`. The session config sent here is the authoritative initial session for that credential. The `model` string selects the agent behavior directly; there is no `agent_id`.
    
    Mint with the ElevenLabs SDK (`npm install @elevenlabs/elevenlabs-js` or `pip install elevenlabs`), which sends the `xi-api-key` header for you:
    
    ```typescript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    
    const elevenlabs = new ElevenLabsClient();
    
    const clientSecret =
      await elevenlabs.conversationalAi.realtime.createClientSecret({
        session: {
          type: "realtime",
          model: "eleven_realtime_v1_mini",
          instructions:
            "You are a helpful, friendly voice assistant. Keep replies concise.",
          outputModalities: ["audio"],
          audio: {
            input: {
              format: { type: "audio/pcm", rate: 24000 },
              turnDetection: { type: "semantic_vad", eagerness: "medium" },
            },
            output: {
              format: { type: "audio/pcm" },
              voice: "brian",
            },
          },
        },
        expiresAfter: { anchor: "created_at", seconds: 600 },
      });
    ```
    
    ```python
    import os
    
    from elevenlabs import (
        CreateClientSecretRequest,
        ExpiresAfter,
        OpenAiAudioConfigInput,
        OpenAiAudioInputConfigInput,
        OpenAiAudioInputFormat,
        OpenAiAudioOutputConfigInput,
        OpenAiAudioOutputFormat,
        OpenAiSessionConfigInput,
        OpenAiTurnDetection,
    )
    from elevenlabs.client import ElevenLabs
    
    elevenlabs = ElevenLabs(api_key=os.environ["ELEVENLABS_API_KEY"])
    
    client_secret = elevenlabs.conversational_ai.realtime.create_client_secret(
        request=CreateClientSecretRequest(
            session=OpenAiSessionConfigInput(
                type="realtime",
                model="eleven_realtime_v1_mini",
                instructions="You are a helpful, friendly voice assistant. Keep replies concise.",
                output_modalities=["audio"],
                audio=OpenAiAudioConfigInput(
                    input=OpenAiAudioInputConfigInput(
                        format=OpenAiAudioInputFormat(type="audio/pcm", rate=24000),
                        turn_detection=OpenAiTurnDetection(type="semantic_vad", eagerness="medium"),
                    ),
                    output=OpenAiAudioOutputConfigInput(
                        format=OpenAiAudioOutputFormat(type="audio/pcm"),
                        voice="brian",
                    ),
                ),
            ),
            expires_after=ExpiresAfter(anchor="created_at", seconds=600),
        ),
    )
    ```
    
    Return only `clientSecret.value` (`client_secret.value` in Python) to the browser.
    
    Keep the route's existing framework, path, and response contract; only the upstream call changes. If the app minted with the beta `/v1/realtime/sessions` endpoint, its frontend reads `client_secret.value`. Either update the frontend to read the top-level `value`, or have the route return `{ client_secret: { value } }` so the frontend stays untouched.
    
    ### Step 2: Convert the session config to the GA shape
    
    Send the GA session shape. Apps written against OpenAI's beta API use flat fields (`modalities`, `voice`, `input_audio_format`, `turn_detection`, …) that need converting, and ElevenLabs rejects the deprecated `modalities` field outright. This matters most for the mint request body and any hand-rolled `session.update`: the SDK's `config` option converts deprecated camelCase fields to the GA wire shape for you, though moving it to the GA form too removes the deprecation warnings.
    
    Read [Session config mapping](references/session-config.md) for the full beta-to-GA field table, voice handling, and the camelCase form the SDK's `config` option uses.
    
    ### Step 3: Point the browser session at ElevenLabs
    
    Replace the transport with `OpenAIRealtimeWebSocket` aimed at the ElevenLabs URL, set the model on the session, and pass the minted `value` as the API key:
    
    ```typescript
    import {
      OpenAIRealtimeWebSocket,
      RealtimeAgent,
      RealtimeSession,
    } from "@openai/agents-realtime";
    
    const { value: clientSecret } = await fetch("/api/token").then((res) =>
      res.json(),
    );
    
    const agent = new RealtimeAgent({
      name: "Assistant",
      instructions:
        "You are a helpful, friendly voice assistant. Keep replies concise.",
    });
    
    const transport = new OpenAIRealtimeWebSocket({
      url: "wss://api.elevenlabs.io/v1/convai/realtime",
    });
    
    const session = new RealtimeSession(agent, {
      transport,
      model: "eleven_realtime_v1_mini",
      config: {
        outputModalities: ["audio"],
        audio: { output: { voice: "brian" } },
      },
    });
    
    await session.connect({ apiKey: clientSecret });
    ```
    
    Keep the app's existing `RealtimeAgent` name, instructions, and event handlers. Drop WebRTC-only options (`audioElement`, `mediaStream`, `changePeerConnection`) and any `transport: "webrtc"` setting; they have no WebSocket equivalent.
    
    ### Step 4: Wire up audio (when the app was on WebRTC)
    
    If the app previously used WebRTC (an explicit `OpenAIRealtimeWebRTC`, `transport: "webrtc"`, or no transport at all in a browser), it now has to:
    
    1. Capture microphone audio as PCM16 at 24 kHz and stream it with `session.sendAudio(...)`.
    2. Play PCM16 chunks from the session's `audio` event.
    3. Stop playback on `audio_interrupted`. Without this, the assistant keeps talking over the user after a barge-in, because already-buffered audio keeps playing.
    4. Release the microphone and close the session on teardown.
    
    Read [Browser audio over WebSocket](references/browser-audio.md) for a working implementation with `wavtools`, the helper OpenAI uses in its own WebSocket examples, and its pitfalls.
    
    Apps already on `OpenAIRealtimeWebSocket` have this wiring; check the sample rate is 24 kHz and keep it.
    
    ### Step 5: Verify
    
    1. Find where the server loads `OPENAI_API_KEY` from (`.env`, `.env.local`, deployment secrets) and ask the user to store `ELEVENLABS_API_KEY` there. Don't ask for the key in the conversation or write it into a file yourself. Add an empty `ELEVENLABS_API_KEY=` to `.env.example` if the project has one.
    2. Once the user confirms the key is in place, run the app, start a conversation, and speak.
    3. Confirm the `session.created` transport event reports the `eleven_realtime_*` model and the chosen voice:
    
       ```typescript
       session.on("transport_event", (event) => {
         if (event.type === "session.created") console.log(event);
       });
       ```
    
    4. Interrupt the assistant mid-sentence and confirm playback stops.
    5. Tell the user which features from Step 0 remain unverified (tools, handoffs, guardrails, …).
    
    ### Troubleshooting
    
    | Symptom                                             | Cause and fix                                                                                         |
    | --------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
    | WebSocket closes with code `4003`                   | Realtime isn't enabled for this ElevenLabs workspace. Contact ElevenLabs; the code can't fix this.    |
    | WebSocket closes with code `4001`                   | Client secret rejected: expired, or not the minted `value` passed as `apiKey` to `session.connect()`. |
    | Mint returns 401                                    | Still sending `Authorization: Bearer`; ElevenLabs uses the `xi-api-key` header.                       |
    | Mint or `session.update` is rejected                | Beta-shaped fields such as `modalities`. Convert with the [mapping](references/session-config.md).    |
    | Connects, but no audio either way                   | The app was on WebRTC and audio isn't wired yet. See Step 4.                                          |
    | Audio is noise, chipmunk-fast, or slowed down       | Sample-rate or buffer mismatch. Capture and play at 24 kHz; see the pitfalls in the audio reference.  |
    | Assistant talks over the user after an interruption | Playback isn't stopped on `audio_interrupted`.                                                        |
    
    ## References
    
    - [Session config mapping](references/session-config.md)
    - [Browser audio over WebSocket](references/browser-audio.md)
    
    Original on GitHub
  • setup-api-keyFix or hand-configure an existing ElevenLabs API key. Use when a key stopped working (401 or invalid_api_key), when an administrator issued the key, when the user's role cannot create keys, or when the user already has a key and needs it in the right place. Checks whether ELEVENLABS_API_KEY is configured and valid, without reading it into the conversation. Not for first-time setup or adding ElevenLabs to a project (use onboarding).Read
    ---
    name: setup-api-key
    description: Fix or hand-configure an existing ElevenLabs API key. Use when a key stopped working (401 or invalid_api_key), when an administrator issued the key, when the user's role cannot create keys, or when the user already has a key and needs it in the right place. Checks whether ELEVENLABS_API_KEY is configured and valid, without reading it into the conversation. Not for first-time setup or adding ElevenLabs to a project (use onboarding).
    license: MIT
    compatibility: Requires internet access to elevenlabs.io and api.elevenlabs.io, and Node.js for npx (or the ElevenLabs CLI installed another way).
    ---
    
    # ElevenLabs API Key Setup
    
    For a key the user already has, or one that stopped working. For adding
    ElevenLabs to a project from scratch, use the `onboarding` skill.
    
    **A key the user already has (for example from an administrator):** don't
    sign in for a new one. Tell them to paste it after `ELEVENLABS_API_KEY=` in
    the env file of the app that uses it (`.env.local`, or `.env` in a Python
    project), never into this chat, and to tell you when it's done. Then check it
    (if `check` finds no key, or ElevenLabs rejects it, and offers to sign in,
    ask the user to check the paste instead of signing in):
    
    ```bash
    npx -y @elevenlabs/cli@latest onboard check
    ```
    
    **A key that stopped working:** run the same command from the app's folder.
    It finds the key the app would use (the shell, the env file, or a `.env` up
    the tree), checks it with ElevenLabs without printing it, and prints JSON:
    `next` is the step (spelled out in `instruction`; where the list below differs,
    follow the list), `run` the exact command, and `say` what to tell the user (tell
    them as written). Follow it until the key works; for a rejected key it
    signs the user in for a new one. What each `next` means:
    
    - `run`: run `run`. `wait`: show `say`, then run `run` again right away.
    - `ask_user`, or a `fix` that needs the user's shell or secrets: ask them
      what `say` says, wait for their answer, then run `run`, unless they said no.
    - `write_code`: if the app already reads `ELEVENLABS_API_KEY` (through the
      SDK or its own requests), don't change its code: after `connect`, run
      `run`; after `check`, tell the user the key works (not `say`) and stop.
      Otherwise make sure the code uses the SDK, then run `run`; if nothing
      shows what the app should do with ElevenLabs, ask the user first, and don't
      pick a product for them.
    - `offer_test`: ask the question in `say`; run `run` only if they say yes.
    - `done`: tell the user `say`.
    
    ## Rules
    
    - Never open, print or `cat` the env file, and never echo
      `ELEVENLABS_API_KEY`; `onboard check` verifies the key for you.
    - Never ask for an API key in chat, and never accept one pasted there. If one
      appears, tell the user to rotate it on the API keys page.
    - Never put the key in code; the SDK reads `ELEVENLABS_API_KEY`.
    - Don't run the app (or its start script) or make speech to try it until the
      user says yes. API calls that what they asked for needs, like creating an
      agent, are fine. Once the code is written, run `onboard check` before you
      finish: it verifies the key and the code.
    - For browser or client-side apps, keep the key on the server and issue
      short-lived tokens where the product supports them.
    
    ## No npx
    
    If `npx` isn't available (Node.js isn't installed), install the CLI (or
    update it if it's already installed: `brew upgrade elevenlabs`, or run the
    installer again), then run `elevenlabs onboard check` instead:
    
    - macOS or Linux: `brew install elevenlabs/tap/elevenlabs`, or
      `curl --proto '=https' --tlsv1.2 -LsSf https://github.com/elevenlabs/cli/releases/latest/download/elevenlabs-cli-installer.sh | sh`
    - Windows: `powershell -ExecutionPolicy ByPass -c "irm https://github.com/elevenlabs/cli/releases/latest/download/elevenlabs-cli-installer.ps1 | iex"`
    
    If `elevenlabs` isn't found after the installer ran, use
    `~/.cargo/bin/elevenlabs` in its place, here and in every `run` the CLI gives.
    
    Original on GitHub
  • sound-effectsGenerate sound effects from text descriptions using ElevenLabs. Use when creating sound effects, generating audio textures, producing ambient sounds, cinematic impacts, UI sounds, or any audio that isn't speech. Supports looping, duration control, and prompt influence tuning.Read
    ---
    name: sound-effects
    description: Generate sound effects from text descriptions using ElevenLabs. Use when creating sound effects, generating audio textures, producing ambient sounds, cinematic impacts, UI sounds, or any audio that isn't speech. Supports looping, duration control, and prompt influence tuning.
    license: MIT
    compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
    metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
    ---
    
    # ElevenLabs Sound Effects
    
    Generate sound effects from text descriptions — supports looping, custom duration, and prompt adherence control.
    
    > **Setup, before writing any code:** if `ELEVENLABS_API_KEY` isn't set for this project, run `npx -y @elevenlabs/cli@latest onboard init` from the app's folder first and follow what it prints: it signs the user in and stores a key for the project, so nobody copies one by hand. Otherwise see the [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only.
    
    ## Quick Start
    
    ### Python
    
    ```python
    from elevenlabs import ElevenLabs
    
    client = ElevenLabs()
    
    audio = client.text_to_sound_effects.convert(
        text="Thunder rumbling in the distance with light rain",
        model_id="eleven_text_to_sound_v3",
    )
    
    with open("thunder.mp3", "wb") as f:
        for chunk in audio:
            f.write(chunk)
    ```
    
    ### JavaScript
    
    ```javascript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    import { createWriteStream } from "fs";
    
    const client = new ElevenLabsClient();
    const audio = await client.textToSoundEffects.convert({
      text: "Thunder rumbling in the distance with light rain",
      modelId: "eleven_text_to_sound_v3",
    });
    audio.pipe(createWriteStream("thunder.mp3"));
    ```
    
    ### CLI
    
    ```bash
    elevenlabs text-to-sound-effects convert \
      --text "Thunder rumbling in the distance with light rain" \
      --model-id eleven_text_to_sound_v3 \
      --output thunder.mp3
    ```
    
    ## Parameters
    
    | Parameter | Type | Default | Description |
    |-----------|------|---------|-------------|
    | `text` | string (required) | — | Description of the desired sound effect |
    | `model_id` | string | `eleven_text_to_sound_v2` | Model to use. Set `eleven_text_to_sound_v3` (SFX v3) explicitly: it follows multi-part prompts more closely, handles music and vocals better, and sounds more natural |
    | `duration_seconds` | number \| null | null (auto) | Duration 0.5–30s; auto-calculated if null |
    | `prompt_influence` | number \| null | 0.3 | How closely to follow the prompt (0–1) |
    | `loop` | boolean | false | Generate a seamlessly looping sound |
    
    ## Examples with Parameters
    
    ```python
    # Looping ambient sound, 10 seconds
    audio = client.text_to_sound_effects.convert(
        text="Gentle forest ambiance with birds chirping",
        model_id="eleven_text_to_sound_v3",
        duration_seconds=10.0,
        prompt_influence=0.5,
        loop=True,
    )
    
    # Short UI sound, high prompt adherence
    audio = client.text_to_sound_effects.convert(
        text="Soft notification chime",
        model_id="eleven_text_to_sound_v3",
        duration_seconds=1.0,
        prompt_influence=0.8,
    )
    ```
    
    ## Output Formats
    
    Pass `--output-format` (CLI) or `output_format` as an SDK parameter:
    
    | Format | Description |
    |--------|-------------|
    | `mp3_44100_128` | MP3 44.1kHz 128kbps (default) |
    | `pcm_44100` | Raw uncompressed CD quality |
    | `opus_48000_128` | Opus 48kHz 128kbps — efficient compressed |
    | `ulaw_8000` | μ-law 8kHz — telephony |
    
    Full list: `mp3_22050_32`, `mp3_24000_48`, `mp3_44100_32`, `mp3_44100_64`, `mp3_44100_96`, `mp3_44100_128`, `mp3_44100_192`, `pcm_8000`, `pcm_16000`, `pcm_22050`, `pcm_24000`, `pcm_32000`, `pcm_44100`, `pcm_48000`, `wav_8000`, `wav_16000`, `wav_22050`, `wav_24000`, `wav_32000`, `wav_44100`, `wav_48000`, `ulaw_8000`, `alaw_8000`, `opus_48000_32`, `opus_48000_64`, `opus_48000_96`, `opus_48000_128`, `opus_48000_192`.
    
    ## Prompt Tips
    
    - Be specific: "Heavy rain on a tin roof" > "Rain"
    - Combine elements: "Footsteps on gravel with distant traffic"
    - Specify style: "Cinematic braam, horror" or "8-bit retro jump sound"
    - Mention mood/context: "Eerie wind howling through an abandoned building"
    
    ## Error Handling
    
    ```python
    try:
        audio = client.text_to_sound_effects.convert(text="Explosion")
    except Exception as e:
        print(f"API error: {e}")
    ```
    
    Common errors:
    - **401**: Invalid API key
    - **422**: Invalid parameters (check duration range, prompt_influence range)
    - **429**: Rate limit exceeded
    
    ## References
    
    - [Installation Guide](references/installation.md)
    
    Original on GitHub
  • speech-engineAdd real-time voice conversations to a custom agent runtime with ElevenLabs Speech Engine. Use when building Speech Engine servers, WebSocket handlers, WebRTC browser clients, conversation token endpoints, interruption-aware streaming responses, or voice-enabled chat agents that connect developer-owned server logic to ElevenLabs speech-to-text and text-to-speech.Read
    ---
    name: speech-engine
    description: Add real-time voice conversations to a custom agent runtime with ElevenLabs Speech Engine. Use when building Speech Engine servers, WebSocket handlers, WebRTC browser clients, conversation token endpoints, interruption-aware streaming responses, or voice-enabled chat agents that connect developer-owned server logic to ElevenLabs speech-to-text and text-to-speech.
    license: MIT
    compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
    metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
    ---
    
    # ElevenLabs Speech Engine
    
    Add a real-time voice interface to a custom agent. ElevenLabs handles microphone audio, speech-to-text, turn-taking, text-to-speech, and browser playback; your server exposes a Speech Engine WebSocket endpoint and streams response text back.
    
    > **Setup, before writing any code:** if `ELEVENLABS_API_KEY` isn't set for this project, run `npx -y @elevenlabs/cli@latest onboard init` from the app's folder first and follow what it prints: it signs the user in and stores a key for the project, so nobody copies one by hand. Otherwise see the [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only. For deeper SDK details, read [JavaScript SDK Reference](references/javascript-sdk-reference.md) or [Python SDK Reference](references/python-sdk-reference.md).
    
    ## When to Use
    
    Use Speech Engine when the user wants to:
    
    - Add voice to an existing chat app or custom server pipeline
    - Add voice to OpenClaw, Hermes, or a similar agent runtime while keeping agent logic on the developer-owned server
    - Build a developer-hosted WebSocket server for ElevenLabs voice conversations
    - Stream response text back as spoken audio after your server validates user intent
    - Handle user interruptions while a response is still streaming
    - Build a browser client with `@elevenlabs/react` or `@elevenlabs/client` using a server-issued conversation token
    
    Use the `agents` skill instead when the user is creating or configuring a hosted ElevenLabs Conversational AI agent with platform-managed prompts, tools, workflows, phone numbers, or widgets.
    
    ## How It Works
    
    Each Speech Engine WebSocket connection represents one conversation.
    
    1. The browser sends user audio to ElevenLabs.
    2. ElevenLabs sends speech-recognition events to your server.
    3. Your server derives trusted application state without letting raw speech text control tools or privileged actions.
    4. Your server streams text back through the SDK.
    5. ElevenLabs converts the response to speech and plays it in the browser.
    
    The SDK manages WebSocket routing, request verification, session lifecycle, ping/pong, turn-taking, and interruption handling. `sendResponse()` / `send_response()` accepts a string or async iterable of response text.
    
    Treat speech-recognition text as untrusted user input. Do not map raw speech text directly into model roles, responses, or tool calls. Use deterministic validation, allowlisted intents, or explicit user confirmation before any transcript-derived value affects downstream response or tool logic.
    
    ## Implementation Flow
    
    1. Install server dependencies and configure `ELEVENLABS_API_KEY`.
    2. Expose your Speech Engine server through a public HTTPS URL for local development, for example with `ngrok http 3001`.
    3. Create a Speech Engine resource with `ws_url` / `wsUrl` pointing at the public WebSocket URL, usually `wss://.../ws`.
    4. Store the returned Speech Engine ID, for example in `ELEVENLABS_SPEECH_ENGINE_ID`.
    5. Start a Speech Engine server with `engine.serve(...)` in Python or `speechEngine.attach(...)` in TypeScript.
    6. Issue browser conversation tokens from a server endpoint. Never put `ELEVENLABS_API_KEY` in browser code.
    7. Start the client session with `conversationToken`; if the agent should greet first, enable the first-message override on the Speech Engine resource, then set `overrides.agent.firstMessage` in the client.
    
    ## Create a Speech Engine
    
    ### Python
    
    ```python
    import asyncio
    import os
    
    from dotenv import load_dotenv
    from elevenlabs import AsyncElevenLabs
    
    load_dotenv()
    
    elevenlabs = AsyncElevenLabs(api_key=os.getenv("ELEVENLABS_API_KEY"))
    
    async def main():
        engine = await elevenlabs.speech_engine.create(
            name="My Speech Engine",
            speech_engine={"ws_url": os.environ["PUBLIC_WS_URL"]},
            overrides={"first_message": True},
        )
        print(engine.engine_id)
    
    asyncio.run(main())
    ```
    
    ### TypeScript
    
    ```typescript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    import "dotenv/config";
    
    const elevenlabs = new ElevenLabsClient({
      apiKey: process.env.ELEVENLABS_API_KEY,
    });
    
    const engine = await elevenlabs.speechEngine.create({
      name: "My Speech Engine",
      speechEngine: { wsUrl: process.env.PUBLIC_WS_URL! },
      overrides: { firstMessage: true },
    });
    
    console.log(engine.engineId);
    ```
    
    `PUBLIC_WS_URL` should look like `wss://example.ngrok.app/ws` locally or your production WebSocket route in deployment.
    
    The create request can also configure `tts`, `asr`, `turn`, `speech_engine.request_headers` / `speechEngine.requestHeaders`, `overrides`, and `privacy` for custom voices, transcription keywords, turn-taking, server auth headers, client-provided first messages, and recording behavior. See the SDK reference files for expanded examples.
    
    Set `auth={"enable_auth": True}` (Python) or `auth: { enableAuth: true }` (TypeScript) on create or update so clients can only start a conversation with a signed URL or conversation token issued by your server (see [Browser Client](#browser-client)), not with the Speech Engine ID alone. It defaults to `false`.
    
    ## Server Pattern
    
    Run the Speech Engine server at the `ws_url` / `wsUrl` configured on the resource. Keep response generation behind your own validation boundary: raw speech-recognition text should not directly control responses, tools, secrets, or other privileged actions.
    
    ### Python
    
    ```python
    engine = await elevenlabs.speech_engine.get(os.environ["ELEVENLABS_SPEECH_ENGINE_ID"])
    await engine.serve(port=3001, path="/ws", debug=True, callbacks=validated_callbacks)
    ```
    
    ### TypeScript
    
    ```typescript
    const engine = await elevenlabs.speechEngine.get(process.env.ELEVENLABS_SPEECH_ENGINE_ID!);
    engine.attach(httpServer, "/ws", { debug: true, ...validatedCallbacks });
    ```
    
    In TypeScript, pass interruption signals to downstream async work when it supports cancellation so interrupted responses stop quickly. In Python, the SDK cancels the previous turn handler when a newer turn arrives.
    
    Server callbacks can distinguish clean closes from dropped connections: use `onClose` / `on_close` for clean disconnects and `onDisconnect` / `on_disconnect` for unexpected WebSocket drops.
    
    Security note: speech-recognition text can contain prompt-injection attempts from user speech or played audio. Treat it as untrusted input. Convert it into trusted application state before invoking response generation, tools, or privileged workflows.
    
    ### Disabling authentication (advanced, dangerous)
    
    Both `engine.attach()` (TypeScript) and `engine.serve()` / `SpeechEngineServer` (Python) verify a JWT on every incoming WebSocket by default. This is what proves the connection is really coming from ElevenLabs and not from an attacker who guessed the URL. **Do not turn this off.**
    
    An escape hatch exists — `disableAuth: true` in the callback options (TypeScript) or `disable_auth=True` on `serve()` / `SpeechEngineServer(...)` (Python) — for the narrow case where a compensating network-level control is already in place. Without such a control, disabling auth means **any client on the internet that finds your URL can open sessions**. Concretely, an attacker can:
    
    - open unlimited conversations to drain your ElevenLabs quota and downstream LLM budget
    - feed crafted transcripts to your response pipeline, effectively impersonating a user
    - use your server as an oracle to probe backend state, tools, or prompts
    
    Only recommend `disableAuth` / `disable_auth` when the user has already implemented **at least one** of:
    
    - **IP allowlist** — the server (or an upstream firewall / load balancer / API gateway) only accepts inbound traffic from [ElevenLabs' documented egress ranges](https://elevenlabs.io/docs/overview/capabilities/speech-engine#ip-allowlisting).
    - **Custom shared-secret header** — a secret header configured on the Speech Engine resource via `speech_engine.request_headers` / `speechEngine.requestHeaders` at create time, validated by an upstream proxy (or by the developer's own middleware in front of `attach()` / `serve()`) before requests reach the SDK.
    
    If the user cannot confirm one of the above is in place, leave the default authentication on. Skipping JWT verification without a mitigation is not an optimization or a convenience — it is unauthenticated public compute.
    
    ## Browser Client
    
    Create a server-side token endpoint and have the browser request a token before starting the microphone session. Keep the Speech Engine ID and API key on the server. If the client passes `overrides.agent.firstMessage`, the Speech Engine resource must have the first-message override enabled.
    
    ```typescript
    import express from "express";
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    import "dotenv/config";
    
    const app = express();
    const elevenlabs = new ElevenLabsClient();
    
    app.get("/api/token", async (_req, res) => {
      const response = await elevenlabs.conversationalAi.conversations.getWebrtcToken({
        agentId: process.env.ELEVENLABS_SPEECH_ENGINE_ID!,
      });
      res.json({ token: response.token });
    });
    ```
    
    React clients can use `@elevenlabs/react`:
    
    ```tsx
    import { useConversation } from "@elevenlabs/react";
    
    export function VoiceControls() {
      const conversation = useConversation({
        onConnect: () => console.log("connected"),
        onDisconnect: () => console.log("disconnected"),
        onError: (error) => console.error(error),
      });
    
      async function startConversation() {
        await navigator.mediaDevices.getUserMedia({ audio: true });
        const { token } = await fetch("/api/token").then((res) => res.json());
    
        await conversation.startSession({
          conversationToken: token,
          overrides: {
            agent: { firstMessage: "Hello! How can I help you today?" },
          },
        });
      }
    
      return <button onClick={startConversation}>Start conversation</button>;
    }
    ```
    
    ## References
    
    - [Installation Guide](references/installation.md)
    - [JavaScript SDK Reference](references/javascript-sdk-reference.md)
    - [Python SDK Reference](references/python-sdk-reference.md)
    
    Original on GitHub
  • speech-to-textTranscribe audio to text using ElevenLabs Scribe v3. Use when converting audio/video to text, generating subtitles, transcribing meetings, or processing spoken content.Read
    ---
    name: speech-to-text
    description: Transcribe audio to text using ElevenLabs Scribe v3. Use when converting audio/video to text, generating subtitles, transcribing meetings, or processing spoken content.
    license: MIT
    compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
    metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
    ---
    
    # ElevenLabs Speech-to-Text
    
    Transcribe audio to text with Scribe v3 - supports 90+ languages, speaker diarization, and word-level timestamps.
    
    > **Setup, before writing any code:** if `ELEVENLABS_API_KEY` isn't set for this project, run `npx -y @elevenlabs/cli@latest onboard init` from the app's folder first and follow what it prints: it signs the user in and stores a key for the project, so nobody copies one by hand. Otherwise see the [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only.
    
    ## Quick Start
    
    ### Python
    
    ```python
    from elevenlabs import ElevenLabs
    
    client = ElevenLabs()
    
    with open("audio.mp3", "rb") as audio_file:
        result = client.speech_to_text.convert(file=audio_file, model_id="scribe_v3")
    
    print(result.text)
    ```
    
    ### JavaScript
    
    ```javascript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    import { createReadStream } from "fs";
    
    const client = new ElevenLabsClient();
    const result = await client.speechToText.convert({
      file: createReadStream("audio.mp3"),
      modelId: "scribe_v3",
    });
    console.log(result.text);
    ```
    
    ### CLI
    
    ```bash
    elevenlabs speech-to-text convert --file audio.mp3 --model-id scribe_v3
    ```
    
    ## Models
    
    | Model ID | Description | Best For |
    |----------|-------------|----------|
    | `scribe_v3` | Most accurate batch model, 90+ languages; adds `numbers_format` and `secondary_language_codes` | Batch transcription, subtitles, long-form and mixed-language audio |
    | `scribe_v3_live` | Most accurate realtime model, with turn prediction and multilingual mode | Live transcription, voice agents |
    | `scribe_v2` | Previous-generation batch model, 90+ languages | Existing batch integrations |
    | `scribe_v2_medical` | Specialized recognition for medical and clinical audio, 90+ languages | Clinical documentation, medical dictation, patient calls |
    | `scribe_v2_realtime` | Previous-generation realtime model, low latency (~150ms) | Existing realtime integrations |
    | `scribe_v2_realtime_turbo` | Realtime transcription variant | Live transcription |
    | `scribe_v2_realtime_lite` | Realtime transcription variant | Live transcription |
    
    ## Transcription with Timestamps
    
    Word-level timestamps include type classification and speaker identification:
    
    ```python
    result = client.speech_to_text.convert(
        file=audio_file, model_id="scribe_v3", timestamps_granularity="word"
    )
    
    for word in result.words:
        print(f"{word.text}: {word.start}s - {word.end}s (type: {word.type})")
    
    ```
    
    ## Speaker Diarization
    
    Identify WHO said WHAT - the model labels each word with a speaker ID, useful for meetings, interviews, or any multi-speaker audio:
    
    ```python
    result = client.speech_to_text.convert(
        file=audio_file,
        model_id="scribe_v3",
        diarize=True
    )
    
    for word in result.words:
        print(f"[{word.speaker_id}] {word.text}")
    ```
    
    For call recordings, the batch API can label diarized speakers as `agent` and `customer` by setting `detect_speaker_roles=true` alongside `diarize=true`. This option is not compatible with `use_multi_channel=true`.
    
    If your workspace has registered speaker profiles, set `use_speaker_library=true` with `diarize=true` to match detected speakers against the speaker library.
    
    ```bash
    elevenlabs speech-to-text convert \
      --file call.mp3 \
      --model-id scribe_v3 \
      --diarize true \
      --detect-speaker-roles true \
      --use-speaker-library true
    ```
    
    ## Multichannel Audio
    
    Use `use_multi_channel=true` when each speaker is isolated on a separate audio channel. By default, the API returns one transcript per channel under `transcripts`; set `multichannel_output_style="combined"` to receive one transcript merged by timestamp, with `channel_index` on each word.
    
    ```python
    result = client.speech_to_text.convert(
        file=audio_file,
        model_id="scribe_v3",
        use_multi_channel=True,
        multichannel_output_style="combined",
    )
    ```
    
    ## Keyterm Prompting
    
    Help the model recognize specific words it might otherwise mishear - product names, technical jargon, or unusual spellings (up to 100 terms):
    
    ```python
    result = client.speech_to_text.convert(
        file=audio_file,
        model_id="scribe_v3",
        keyterms=["ElevenLabs", "Scribe", "API"]
    )
    ```
    
    ## Language Detection
    
    Automatic detection with optional language hint:
    
    ```python
    result = client.speech_to_text.convert(
        file=audio_file,
        model_id="scribe_v3",
        language_code="eng"  # ISO 639-1 or ISO 639-3 code
    )
    
    print(f"Detected: {result.language_code} ({result.language_probability:.0%})")
    ```
    
    `scribe_v3` also accepts `secondary_language_codes` for mixed-language audio (other languages expected alongside `language_code`) and `numbers_format` (`digits` or `words`) to control how numbers are written. The SDKs don't have typed arguments for these yet, so pass them through `request_options`:
    
    ```python
    result = client.speech_to_text.convert(
        file=audio_file,
        model_id="scribe_v3",
        language_code="en",
        request_options={
            "additional_body_parameters": {
                "secondary_language_codes": ["es", "fr"],
                "numbers_format": "digits",
            },
        },
    )
    ```
    
    ## Supported Formats
    
    **Audio:** MP3, WAV, M4A, FLAC, OGG, WebM, AAC, AIFF, Opus
    **Video:** MP4, AVI, MKV, MOV, WMV, FLV, WebM, MPEG, 3GPP
    
    **Limits:** Up to 5.0GB file size, 10 hours duration
    
    ## Response Format
    
    ```json
    {
      "text": "The full transcription text",
      "language_code": "eng",
      "language_probability": 0.98,
      "words": [
        {"text": "The", "start": 0.0, "end": 0.15, "type": "word", "speaker_id": "speaker_0"},
        {"text": " ", "start": 0.15, "end": 0.16, "type": "spacing", "speaker_id": "speaker_0"}
      ]
    }
    ```
    
    **Word types:**
    - `word` - An actual spoken word
    - `spacing` - Whitespace between words (useful for precise timing)
    - `audio_event` - Non-speech sounds the model detected (laughter, applause, music, etc.)
    
    ## Error Handling
    
    ```python
    try:
        result = client.speech_to_text.convert(file=audio_file, model_id="scribe_v3")
    except Exception as e:
        print(f"Transcription failed: {e}")
    ```
    
    Common errors:
    - **401**: Invalid API key
    - **422**: Invalid parameters
    - **429**: Rate limit exceeded
    
    ## Tracking Costs
    
    Monitor usage via `request-id` response header:
    
    ```python
    response = client.speech_to_text.with_raw_response.convert(file=audio_file, model_id="scribe_v3")
    result = response.data
    print(f"Request ID: {response.headers.get('request-id')}")
    ```
    
    ## Real-Time Streaming
    
    For live transcription with ultra-low latency, use the real-time API. The real-time API produces two types of transcripts:
    
    - **Partial transcripts**: Interim results that update frequently as audio is processed - use these for live feedback (e.g., showing text as the user speaks)
    - **Committed transcripts**: Final, stable results after you "commit" - use these as the source of truth for your application
    
    A "commit" tells the model to finalize the current segment. You can commit manually (e.g., when the user pauses) or use Voice Activity Detection (VAD) to auto-commit on silence.
    
    ### Python (Server-Side)
    
    ```python
    import asyncio
    from elevenlabs import CommitStrategy, ElevenLabs, RealtimeEvents, RealtimeUrlOptions
    
    client = ElevenLabs()
    
    async def transcribe_realtime():
        done = asyncio.Event()
        connection = await client.speech_to_text.realtime.connect(RealtimeUrlOptions(
            model_id="scribe_v3_live",
            url="https://npr-ice.streamguys1.com/live.mp3",
            commit_strategy=CommitStrategy.VAD,  # URL streams never commit on their own
            include_timestamps=True,
            keyterms=["ElevenLabs", "Scribe"],
            no_verbatim=True,
        ))
    
        def on_error(error):
            print(f"Error: {error}")
            done.set()
    
        connection.on(RealtimeEvents.PARTIAL_TRANSCRIPT, lambda data: print(f"Partial: {data.get('text', '')}"))
        connection.on(RealtimeEvents.COMMITTED_TRANSCRIPT, lambda data: print(f"Final: {data.get('text', '')}"))
        connection.on(RealtimeEvents.ERROR, on_error)
        connection.on(RealtimeEvents.CLOSE, done.set)
    
        try:
            await done.wait()
        finally:
            await connection.close()
    
    asyncio.run(transcribe_realtime())
    ```
    
    ### JavaScript (Client-Side with React)
    
    ```typescript
    import { useScribe, CommitStrategy } from "@elevenlabs/react";
    
    function TranscriptionComponent() {
      const [transcript, setTranscript] = useState("");
    
      const scribe = useScribe({
        modelId: "scribe_v3_live",
        commitStrategy: CommitStrategy.VAD, // Auto-commit on silence for mic input
        keyterms: ["ElevenLabs", "Scribe"],
        noVerbatim: true,
        includeLanguageDetection: true,
        onPartialTranscript: (data) => console.log("Partial:", data.text),
        onCommittedTranscript: (data) => setTranscript((prev) => prev + data.text),
      });
    
      const start = async () => {
        // Get token from your backend (never expose API key to client)
        const { token } = await fetch("/scribe-token").then((r) => r.json());
    
        await scribe.connect({
          token,
          microphone: { echoCancellation: true, noiseSuppression: true },
        });
      };
    
      return <button onClick={start}>Start Recording</button>;
    }
    ```
    
    ### Commit Strategies
    
    | Strategy | Description |
    |----------|-------------|
    | **Manual** | You call `commit()` when ready - use for file processing or when you control the audio segments |
    | **VAD** | Voice Activity Detection auto-commits when silence is detected - use for live microphone input |
    | **Turn prediction** | `scribe_v3_live` only. Commits when the model predicts the speaker has finished their turn, with VAD as a fallback - set `commit_strategy=turn_prediction` on the WebSocket URL (not yet in the SDK `CommitStrategy` enums) |
    
    Set `includeLanguageDetection: true` to receive the detected language code in delayed final
    transcript events.
    
    ```typescript
    // React: set commitStrategy on the hook (recommended for mic input)
    import { useScribe, CommitStrategy } from "@elevenlabs/react";
    
    const scribe = useScribe({
      modelId: "scribe_v3_live",
      commitStrategy: CommitStrategy.VAD,
      keyterms: ["ElevenLabs", "Scribe"],
      noVerbatim: true,
      // Optional VAD tuning:
      vadSilenceThresholdSecs: 1.5,
      vadThreshold: 0.4,
    });
    ```
    
    ```javascript
    // JavaScript client: pass vad config on connect
    const connection = await client.speechToText.realtime.connect({
      modelId: "scribe_v3_live",
      keyterms: ["ElevenLabs", "Scribe"],
      noVerbatim: true,
      vad: {
        silenceThresholdSecs: 1.5,
        threshold: 0.4,
      },
    });
    ```
    
    ### Event Types
    
    | Event | Description |
    |-------|-------------|
    | `partial_transcript` | Live interim results |
    | `final_transcript` | Stable segment result sent before the segment is committed |
    | `final_transcript_with_timestamps` | Delayed final result with timestamps and/or detected language |
    | `committed_transcript` | Final results after commit |
    | `committed_transcript_with_timestamps` | Final with word timing |
    | `committed_transcript_entities` | Entities detected in a committed segment |
    | `edited_transcript` | Original and edited text for a committed segment when transcript editing is enabled |
    | `invalid_request` | Connection parameters were rejected and the session closes |
    | `error` | Error occurred |
    
    See real-time references for complete documentation.
    
    ## References
    
    - [Installation Guide](references/installation.md)
    - [Transcription Options](references/transcription-options.md)
    - [Real-Time Client-Side Streaming](references/realtime-client-side.md)
    - [Real-Time Server-Side Streaming](references/realtime-server-side.md)
    - [Commit Strategies](references/realtime-commit-strategies.md)
    - [Real-Time Event Reference](references/realtime-events.md)
    
    Original on GitHub
  • text-to-speechConvert text to speech using ElevenLabs voice AI. Use when generating audio from text, creating voiceovers, building voice apps, or synthesizing speech in 90+ languages.Read
    ---
    name: text-to-speech
    description: Convert text to speech using ElevenLabs voice AI. Use when generating audio from text, creating voiceovers, building voice apps, or synthesizing speech in 90+ languages.
    license: MIT
    compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
    metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
    ---
    
    # ElevenLabs Text-to-Speech
    
    Generate natural speech from text - supports 90+ languages, multiple models for quality vs latency tradeoffs. Examples default to `eleven_v4`, the latest and highest-quality model.
    
    > **Setup, before writing any code:** if `ELEVENLABS_API_KEY` isn't set for this project, run `npx -y @elevenlabs/cli@latest onboard init` from the app's folder first and follow what it prints: it signs the user in and stores a key for the project, so nobody copies one by hand. Otherwise see the [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only.
    
    ## Quick Start
    
    ### Python
    
    ```python
    from elevenlabs import ElevenLabs
    
    client = ElevenLabs()
    
    audio = client.text_to_speech.convert(
        text="Hello, welcome to ElevenLabs!",
        voice_id="JBFqnCBsd6RMkjVDRZzb",  # George
        model_id="eleven_v4"
    )
    
    with open("output.mp3", "wb") as f:
        for chunk in audio:
            f.write(chunk)
    ```
    
    ### JavaScript
    
    ```javascript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    import { createWriteStream } from "fs";
    import { Readable } from "stream";
    
    const client = new ElevenLabsClient();
    const audio = await client.textToSpeech.convert("JBFqnCBsd6RMkjVDRZzb", {
      text: "Hello, welcome to ElevenLabs!",
      modelId: "eleven_v4",
    });
    // convert() returns a web ReadableStream — bridge it to a Node stream to write to disk
    Readable.fromWeb(audio).pipe(createWriteStream("output.mp3"));
    ```
    
    ### CLI
    
    Use `say` to play text immediately with the default voice and `eleven_v3` model:
    
    ```bash
    elevenlabs say "Hello!"
    ```
    
    Pipe text into `say` when another command produces the input:
    
    ```bash
    echo "The build finished successfully." | elevenlabs say
    ```
    
    Use the API command when you need to set request parameters directly:
    
    ```bash
    elevenlabs text-to-speech convert --voice-id JBFqnCBsd6RMkjVDRZzb \
      --text "Hello!" --model-id eleven_v4 --output output.mp3
    ```
    
    The CLI reads `ELEVENLABS_API_KEY` from the environment automatically.
    
    ## Models
    
    | Model ID | Languages | Latency | Best For |
    |----------|-----------|---------|----------|
    | `eleven_v4` | 90+ | Standard | **Default.** Highest quality, emotional range, audio tags, voice cloning accuracy |
    | `eleven_v4_turbo` | 90+ | ~100ms | Real-time v4 quality for agents and interactive apps (via [Text to Dialogue WebSocket](references/streaming.md#eleven-v4-turbo-websocket)) |
    | `eleven_flash_v2_5` | 32 | ~75ms | Lowest latency, lowest cost, `stream-input` WebSocket |
    | `eleven_flash_v2` | English | ~75ms | English-only, lowest latency |
    | `eleven_multilingual_v2` | 29 | Standard | Previous generation; most stable on very long-form generations |
    | `eleven_v3` | 70+ | Standard | Previous generation expressive model |
    
    `eleven_turbo_v2_5` and `eleven_turbo_v2` are superseded by the Flash models (same output, lower latency) — use `eleven_flash_v2_5` / `eleven_flash_v2` instead.
    
    **Eleven v4 notes:** supports only `stability` and `similarity_boost` voice settings (no `style` or `speed`), does not support SSML, and is not available on the `stream-input` TTS WebSocket. Direct delivery with audio tags such as `[whispers]`, `[laughs]`, `[sarcastic]`.
    
    ## Voice IDs
    
    Use pre-made voices or create custom voices in the dashboard.
    
    **Popular voices:**
    - `JBFqnCBsd6RMkjVDRZzb` - George (male, narrative)
    - `EXAVITQu4vr4xnSDxMaL` - Sarah (female, soft)
    - `onwK4e9ZLuTAKqWW03F9` - Daniel (male, authoritative)
    
    **Pick a voice the account's plan can use.** Free plans can use pre-made voices through the API, but not Voice Library voices: those fail with a `402` (`paid_plan_required`). Use George unless the user asks for something else. Do not use a voice ID from memory. Tutorials still use voices that have since been retired (Rachel, Charlotte, Josh and others), and those fail the same way on a free plan. To choose another voice, list the pre-made ones:
    
    ```python
    voices = client.voices.get_all()
    for voice in voices.voices:
        if voice.category == "premade":
            print(f"{voice.voice_id}: {voice.name}")
    ```
    
    ## Voice Settings
    
    Fine-tune how the voice sounds:
    
    - **Stability**: How consistent the voice stays. Lower values = more emotional range and variation, but can sound unstable. Higher = steady, predictable delivery.
    - **Similarity boost**: How closely to match the original voice sample. Higher values sound more like the original but may amplify audio artifacts.
    - **Style**: Exaggerates the voice's unique style characteristics (v2/v3 models; not supported on Eleven v4).
    - **Speed**: Speech speed multiplier (v2/v3 models; not supported on Eleven v4).
    - **Speaker boost**: Post-processing that enhances clarity and voice similarity.
    
    ```python
    from elevenlabs import VoiceSettings
    
    audio = client.text_to_speech.convert(
        text="Customize my voice settings.",
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        model_id="eleven_v4",
        voice_settings=VoiceSettings(
            stability=0.5,          # Lower = more expressive, higher = more consistent
            similarity_boost=0.75,  # Higher = closer to the reference voice
        )
    )
    ```
    
    ## Language Selection
    
    Use `language_code` with models that support language enforcement to guide pronunciation and text normalization. Unsupported language codes are ignored, and `language_code` is not supported on `eleven_multilingual_v2`.
    
    ```python
    audio = client.text_to_speech.convert(
        text="Bonjour, comment allez-vous?",
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        model_id="eleven_v4",
        language_code="fr"  # ISO 639-1 code
    )
    ```
    
    ## Text Normalization
    
    Controls how numbers, dates, and abbreviations are converted to spoken words. For example, "01/15/2026" becomes "January fifteenth, twenty twenty-six":
    
    - `"auto"` (default): Model decides based on context
    - `"on"`: Always normalize (use when you want natural speech)
    - `"off"`: Speak literally (use when you want "zero one slash one five...")
    
    ```python
    audio = client.text_to_speech.convert(
        text="Call 1-800-555-0123 on 01/15/2026",
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        model_id="eleven_v4",
        apply_text_normalization="on"
    )
    ```
    
    ## Audio Effects
    
    Use `audio_effects` to process the generated audio with a filter preset, perceived distance, a reverb environment, a looped background noise bed, and stereo panning. Effects do not change the cost of a request.
    
    | Field | Range | Description |
    |-------|-------|-------------|
    | `filter_preset_id` | Catalog ID | Filter that colors the voice, such as `phone` or `old_radio` |
    | `distance` | 0-1 (default 0) | How far the speaker sounds from the microphone |
    | `environment_id` | Catalog ID | Reverb environment, such as `small_room` or `hall` |
    | `send_level` | 0-1 (default 1) | How much of the voice feeds the reverb; 0 turns reverb off |
    | `background_noise_id` | Catalog ID | Background noise looped under the voice, such as `cafe` or `keyboard` |
    | `pan` | -1 to 1 (default 0) | Stereo position of the voice, from hard left to hard right |
    
    List valid filter preset, environment, and background noise IDs with the catalog endpoint (`GET /v1/audio-effects/catalog`):
    
    ```python
    from elevenlabs import EffectsSpecInput
    
    catalog = client.audio_effects.get_catalog()
    for environment in catalog.environments:
        print(f"{environment.environment_id}: {environment.name}")
    
    audio = client.text_to_speech.convert(
        text="Welcome to the concert hall.",
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        model_id="eleven_v4",
        audio_effects=EffectsSpecInput(
            environment_id="hall",
            distance=0.3,
            pan=-0.5,
        )
    )
    ```
    
    Reverb and non-zero `pan` produce stereo audio and are rejected for `ulaw_*`, `alaw_*`, and `pcm_*` output formats. Unknown effect IDs return a 404 error.
    
    ## Request Stitching
    
    When generating long audio in multiple requests, the audio can have pops, unnatural pauses, or tone shifts at the boundaries. Request stitching solves this by letting each request know what comes before/after it:
    
    ```python
    # First request
    audio1 = client.text_to_speech.convert(
        text="This is the first part.",
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        model_id="eleven_v4",
        next_text="And this continues the story."
    )
    
    # Second request using previous context
    audio2 = client.text_to_speech.convert(
        text="And this continues the story.",
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        model_id="eleven_v4",
        previous_text="This is the first part."
    )
    ```
    
    ## Output Formats
    
    | Format | Description |
    |--------|-------------|
    | `mp3_44100_128` | MP3 44.1kHz 128kbps (default) - compressed, good for web/apps |
    | `mp3_44100_192` | MP3 44.1kHz 192kbps (Creator+) - higher quality compressed |
    | `mp3_44100_64` | MP3 44.1kHz 64kbps - lower quality, smaller files |
    | `mp3_22050_32` | MP3 22.05kHz 32kbps - smallest MP3 files |
    | `pcm_16000` | Raw PCM 16kHz - use for real-time processing |
    | `pcm_22050` | Raw PCM 22.05kHz |
    | `pcm_24000` | Raw PCM 24kHz - good balance for streaming |
    | `pcm_44100` | Raw PCM 44.1kHz (Pro+) - CD quality |
    | `pcm_48000` | Raw PCM 48kHz (Pro+) - highest quality |
    | `ulaw_8000` | μ-law 8kHz - standard for phone systems (Twilio, telephony) |
    | `alaw_8000` | A-law 8kHz - telephony (alternative to μ-law) |
    | `opus_48000_64` | Opus 48kHz 64kbps - efficient streaming codec |
    | `wav_44100` | WAV 44.1kHz - uncompressed with headers |
    
    ## Streaming
    
    Use the `stream` method to receive audio chunks as they're generated:
    
    ```python
    audio_stream = client.text_to_speech.stream(
        text="This text will be streamed as audio.",
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        model_id="eleven_v4"
    )
    
    for chunk in audio_stream:
        play_audio(chunk)
    ```
    
    For real-time apps where latency matters most, use `eleven_v4_turbo` (~100ms) over the Text to Dialogue WebSocket, or `eleven_flash_v2_5` (~75ms) over HTTP streaming or the `stream-input` WebSocket. See [references/streaming.md](references/streaming.md).
    
    ## Error Handling
    
    ```python
    try:
        audio = client.text_to_speech.convert(
            text="Generate speech",
            voice_id="invalid-voice-id",
            model_id="eleven_v4"
        )
    except Exception as e:
        print(f"API error: {e}")
    ```
    
    Common errors:
    - **401**: Invalid API key
    - **422**: Invalid parameters (check voice_id, model_id)
    - **429**: Rate limit exceeded
    
    ## Tracking Costs
    
    Monitor character usage via response headers (`x-character-count`, `request-id`):
    
    ```python
    response = client.text_to_speech.convert.with_raw_response(
        text="Hello!", voice_id="JBFqnCBsd6RMkjVDRZzb", model_id="eleven_v4"
    )
    audio = response.parse()
    print(f"Characters used: {response.headers.get('x-character-count')}")
    ```
    
    ## References
    
    - [Installation Guide](references/installation.md)
    - [Streaming Audio](references/streaming.md)
    - [Voice Settings](references/voice-settings.md)
    
    Original on GitHub
  • voice-changerTransform the voice in an audio recording into a different target voice while preserving emotion, timing, and delivery using the ElevenLabs Voice Changer (speech-to-speech) API. Use when converting one voice to another, changing the speaker/narrator of an existing recording, dubbing a voice-over in a different voice, creating character voices from a scratch performance, anonymizing a speaker, or any "voice conversion / voice transfer / speech-to-speech" task. Make sure to use this skill whenever the user mentions voice changing, voice conversion, speech-to-speech, swapping a voice in audio, re-voicing a clip, or applying a different voice to an existing recording — even if they don't explicitly say "voice changer". Do not use this skill to create or clone a new voice from a voice sample — that is voice cloning (IVC/PVC), a separate feature; this skill only converts an existing recording into an existing voice_id.Read
    ---
    name: voice-changer
    description: Transform the voice in an audio recording into a different target voice while preserving emotion, timing, and delivery using the ElevenLabs Voice Changer (speech-to-speech) API. Use when converting one voice to another, changing the speaker/narrator of an existing recording, dubbing a voice-over in a different voice, creating character voices from a scratch performance, anonymizing a speaker, or any "voice conversion / voice transfer / speech-to-speech" task. Make sure to use this skill whenever the user mentions voice changing, voice conversion, speech-to-speech, swapping a voice in audio, re-voicing a clip, or applying a different voice to an existing recording — even if they don't explicitly say "voice changer". Do not use this skill to create or clone a new voice from a voice sample — that is voice cloning (IVC/PVC), a separate feature; this skill only converts an existing recording into an existing voice_id.
    license: MIT
    compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
    metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
    ---
    
    # ElevenLabs Voice Changer
    
    Transform the voice in an audio recording into a different target voice. Voice Changer (previously called Speech-to-Speech — the REST endpoints are `POST /v1/voice-changer/{voice_id}` and `/v1/voice-changer/{voice_id}/stream`, while the SDK methods and CLI command still use the `speech_to_speech` / `speechToSpeech` / `speech-to-speech` name) keeps the original performance — emotion, pacing, intonation, breaths, whispers, laughs, cries — and only swaps who is speaking.
    
    > **Setup, before writing any code:** if `ELEVENLABS_API_KEY` isn't set for this project, run `npx -y @elevenlabs/cli@latest onboard init` from the app's folder first and follow what it prints: it signs the user in and stores a key for the project, so nobody copies one by hand. Otherwise see the [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only.
    
    ## Key Facts
    
    - **Maximum input length:** 5 minutes per request — split longer recordings into chunks and stitch the outputs.
    - **Maximum file size:** 50 MB per request — compress to MP3 if your source is larger.
    - **Pricing:** 1,000 characters per minute of audio processed (duration-based, not text-based).
    - **Recommended model:** `eleven_multilingual_sts_v2` — often outperforms `eleven_english_sts_v2` even for English-only content.
    
    ## Quick Start
    
    ### Python
    
    ```python
    from elevenlabs import ElevenLabs
    
    client = ElevenLabs()
    
    with open("source.mp3", "rb") as audio_file:
        audio_stream = client.speech_to_speech.convert(
            voice_id="JBFqnCBsd6RMkjVDRZzb",  # George
            audio=audio_file,
            model_id="eleven_multilingual_sts_v2",
            output_format="mp3_44100_128",
        )
    
    with open("converted.mp3", "wb") as f:
        for chunk in audio_stream:
            f.write(chunk)
    ```
    
    ### JavaScript
    
    ```javascript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    import { createReadStream, createWriteStream } from "fs";
    
    const client = new ElevenLabsClient();
    
    const audioStream = await client.speechToSpeech.convert("JBFqnCBsd6RMkjVDRZzb", {
      audio: createReadStream("source.mp3"),
      modelId: "eleven_multilingual_sts_v2",
      outputFormat: "mp3_44100_128",
    });
    
    audioStream.pipe(createWriteStream("converted.mp3"));
    ```
    
    ### CLI
    
    ```bash
    elevenlabs speech-to-speech convert \
      --voice-id JBFqnCBsd6RMkjVDRZzb \
      --audio source.mp3 \
      --model-id eleven_multilingual_sts_v2 \
      --output-format mp3_44100_128 \
      --output converted.mp3
    ```
    
    ## Parameters
    
    | Parameter | Type | Default | Description |
    |-----------|------|---------|-------------|
    | `voice_id` | string (required) | — | Target voice to speak in. Use a pre-made voice ID, a cloned voice, or a voice from the library |
    | `audio` | file (required) | — | Source audio whose performance (emotion, timing, delivery) will be preserved |
    | `model_id` | string | `eleven_english_sts_v2` | `eleven_multilingual_sts_v2` for 29 languages, `eleven_english_sts_v2` for English-only |
    | `output_format` | string | `mp3_44100_128` | See output formats table below |
    | `voice_settings` | JSON string | — | Override stored voice settings for this request only |
    | `seed` | integer | — | Best-effort deterministic sampling (0 – 4294967295) |
    | `remove_background_noise` | boolean | `false` | Run the isolation model on the input before conversion |
    | `file_format` | string | `other` | `other` for any encoded audio, or `pcm_s16le_16` for 16-bit PCM mono @ 16kHz little-endian (lower latency) |
    | `optimize_streaming_latency` | int (query) | — | 0–4. Trade quality for latency. `4` is fastest but disables the text normalizer |
    | `enable_logging` | boolean (query) | `true` | Set to `false` for zero-retention mode (enterprise only — disables history/stitching) |
    
    ## Models
    
    | Model ID | Languages | Best For |
    |----------|-----------|----------|
    | `eleven_multilingual_sts_v2` | 29 | Recommended for everything — often outperforms the English model even on English audio |
    | `eleven_english_sts_v2` | English | API default — English-only fallback |
    
    Only models whose `can_do_voice_conversion` property is true can be used here. Voice Changer does not currently have a low-latency "flash/turbo" tier — if you need one, keep `pcm_s16le_16` input, an `opus_*` / low-bitrate `mp3_*` output, and raise `optimize_streaming_latency`.
    
    ### Languages (`eleven_multilingual_sts_v2`)
    
    English (US, UK, AU, CA), Japanese, Chinese, German, Hindi, French (FR, CA), Korean, Portuguese (BR, PT), Italian, Spanish (ES, MX), Indonesian, Dutch, Turkish, Filipino, Polish, Swedish, Bulgarian, Romanian, Arabic (SA, AE), Czech, Greek, Finnish, Croatian, Malay, Slovak, Danish, Tamil, Ukrainian, Russian.
    
    ## Target Voices
    
    Use any voice ID from pre-made voices, your cloned voices, or the voice library. Free plans can use pre-made voices through the API, but not Voice Library voices: those fail with a `402` (`paid_plan_required`). Do not use a voice ID from memory, since tutorials still use voices that have since been retired.
    
    **Popular voices:**
    - `JBFqnCBsd6RMkjVDRZzb` — George (male, narrative)
    - `EXAVITQu4vr4xnSDxMaL` — Sarah (female, soft)
    - `onwK4e9ZLuTAKqWW03F9` — Daniel (male, authoritative)
    
    ```python
    voices = client.voices.get_all()
    for voice in voices.voices:
        print(f"{voice.voice_id}: {voice.name}")
    ```
    
    ## Converting from a URL
    
    ```python
    import requests
    from io import BytesIO
    from elevenlabs import ElevenLabs
    
    client = ElevenLabs()
    
    audio_url = "https://storage.googleapis.com/eleven-public-cdn/audio/marketing/nicole.mp3"
    response = requests.get(audio_url)
    audio_data = BytesIO(response.content)
    
    audio_stream = client.speech_to_speech.convert(
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        audio=audio_data,
        model_id="eleven_multilingual_sts_v2",
        output_format="mp3_44100_128",
    )
    
    with open("converted.mp3", "wb") as f:
        for chunk in audio_stream:
            f.write(chunk)
    ```
    
    ## Voice Settings Override
    
    Fine-tune the target voice for a single request without changing its stored defaults:
    
    ```python
    from elevenlabs import VoiceSettings
    
    audio_stream = client.speech_to_speech.convert(
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        audio=audio_file,
        model_id="eleven_multilingual_sts_v2",
        voice_settings=VoiceSettings(
            stability=0.5,
            similarity_boost=0.75,
            style=0.0,
            use_speaker_boost=True,
        ),
    )
    ```
    
    - **Stability**: lower = more emotional range (follows the source more freely), higher = steadier delivery.
    - **Similarity boost**: higher = closer to the target voice's timbre, may amplify source artifacts.
    - **Style**: exaggerates the target voice's unique characteristics (v2+ models).
    - **Speaker boost**: post-processing to sharpen clarity of the target voice.
    
    ## Cleaning Up Noisy Source Audio
    
    If the input recording is noisy, either pre-process with the voice-isolator skill or pass `remove_background_noise=True` to do it in a single call:
    
    ```python
    audio_stream = client.speech_to_speech.convert(
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        audio=audio_file,
        model_id="eleven_multilingual_sts_v2",
        remove_background_noise=True,
    )
    ```
    
    Cleaner input almost always produces better conversion — the model is trying to match phonemes and prosody, and background noise gets in the way.
    
    ## Low-Latency PCM Input
    
    If you already have raw 16-bit PCM mono @ 16kHz, passing `file_format="pcm_s16le_16"` skips decoding and reduces latency:
    
    ```python
    audio_stream = client.speech_to_speech.convert(
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        audio=pcm_bytes,
        model_id="eleven_multilingual_sts_v2",
        file_format="pcm_s16le_16",
    )
    ```
    
    Pair this with `optimize_streaming_latency` (0–4) as a query param for further latency reductions at some quality cost.
    
    ## Output Formats
    
    | Format | Description |
    |--------|-------------|
    | `mp3_44100_128` | MP3 44.1kHz 128kbps (default) — good for web/apps |
    | `mp3_44100_192` | MP3 44.1kHz 192kbps (Creator+) — higher quality |
    | `mp3_44100_64` | MP3 44.1kHz 64kbps — smaller files |
    | `mp3_22050_32` | MP3 22.05kHz 32kbps — smallest MP3 |
    | `pcm_16000` | Raw PCM 16kHz — real-time pipelines |
    | `pcm_24000` | Raw PCM 24kHz — good streaming balance |
    | `pcm_44100` | Raw PCM 44.1kHz (Pro+) — CD quality |
    | `pcm_48000` | Raw PCM 48kHz (Pro+) — highest quality |
    | `ulaw_8000` | μ-law 8kHz — Twilio / telephony |
    | `alaw_8000` | A-law 8kHz — telephony |
    | `opus_48000_64` | Opus 48kHz 64kbps — efficient streaming |
    
    ## Deterministic Output
    
    Pass a `seed` to make repeated conversions of the same input return (best-effort) identical audio — useful for testing and A/B comparisons.
    
    ```python
    audio_stream = client.speech_to_speech.convert(
        voice_id="JBFqnCBsd6RMkjVDRZzb",
        audio=audio_file,
        model_id="eleven_multilingual_sts_v2",
        seed=12345,
    )
    ```
    
    ## Input Audio Best Practices
    
    The conversion quality is bounded by the input recording — the model can only swap the timbre, not rescue a bad source. A few practical rules:
    
    - **Be expressive.** Whisper, shout, laugh, cry — the model preserves all of it. Flat input gives you flat output.
    - **Watch microphone gain.** Too quiet and the model under-detects phonemes; too loud and clipping bleeds into the conversion. Aim for healthy peaks, no clipping.
    - **Accent and cadence transfer from the source, not the target.** If you read in an American accent and target the British "George" voice, you get George's timbre with an American accent. To dub *into* a different accent or language, record someone speaking in that target accent/language and convert into a cloned/library voice.
    - **Clean up noise first.** Either pass `remove_background_noise=True` or run the source through the voice-isolator skill before conversion. Noise hurts more here than in TTS.
    - **Split long recordings.** Anything over 5 minutes must be chunked. Cut at natural pauses, convert each piece, and concatenate the resulting audio.
    
    ## Common Workflows
    
    - **Re-voice a narration** — keep the performance of a scratch recording, swap in a different narrator voice.
    - **Localize / dub** — convert a voice-over into the same speaker's cloned voice in another language (using `eleven_multilingual_sts_v2`).
    - **Create character voices** — act out a line yourself, convert into a distinctive character voice for games or animation.
    - **Anonymize a speaker** — replace a recognizable voice with a neutral pre-made voice while preserving what was said and how.
    - **Pair with voice-isolator** — isolate the source voice first (or set `remove_background_noise=True`) for noisy field recordings before conversion.
    - **Pair with voice cloning** — clone a target voice from a short sample, then use its `voice_id` here as the conversion target.
    
    ## Error Handling
    
    ```python
    try:
        audio_stream = client.speech_to_speech.convert(
            voice_id="JBFqnCBsd6RMkjVDRZzb",
            audio=audio_file,
            model_id="eleven_multilingual_sts_v2",
        )
    except Exception as e:
        print(f"Voice changer failed: {e}")
    ```
    
    Common errors:
    - **401**: Invalid API key
    - **422**: Invalid parameters (check `voice_id`, `model_id`, or `file_format` vs the supplied audio)
    - **429**: Rate limit exceeded
    
    ## References
    
    - [Installation Guide](references/installation.md)
    
    Original on GitHub
  • voice-isolatorRemove background noise and isolate vocals/speech from audio using ElevenLabs Voice Isolator (audio isolation) API. Use when cleaning up noisy recordings, removing music or background ambience from dialogue, isolating speech from field recordings, preparing audio for transcription, extracting vocals, or any "denoise / clean up / isolate voice" task.Read
    ---
    name: voice-isolator
    description: Remove background noise and isolate vocals/speech from audio using ElevenLabs Voice Isolator (audio isolation) API. Use when cleaning up noisy recordings, removing music or background ambience from dialogue, isolating speech from field recordings, preparing audio for transcription, extracting vocals, or any "denoise / clean up / isolate voice" task.
    license: MIT
    compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
    metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
    ---
    
    # ElevenLabs Voice Isolator
    
    Removes background noise from audio and isolates vocals/speech — useful for cleaning up noisy recordings, prepping audio for transcription, or pulling dialogue out of a mixed track.
    
    > **Setup, before writing any code:** if `ELEVENLABS_API_KEY` isn't set for this project, run `npx -y @elevenlabs/cli@latest onboard init` from the app's folder first and follow what it prints: it signs the user in and stores a key for the project, so nobody copies one by hand. Otherwise see the [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only.
    
    ## Quick Start
    
    ### Python
    
    ```python
    from elevenlabs import ElevenLabs
    
    client = ElevenLabs()
    
    with open("noisy.mp3", "rb") as audio_file:
        audio_stream = client.audio_isolation.convert(audio=audio_file)
    
    with open("clean.mp3", "wb") as f:
        for chunk in audio_stream:
            f.write(chunk)
    ```
    
    ### JavaScript
    
    ```javascript
    import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
    import { createReadStream, createWriteStream } from "fs";
    
    const client = new ElevenLabsClient();
    
    const audioStream = await client.audioIsolation.convert({
      audio: createReadStream("noisy.mp3"),
    });
    
    audioStream.pipe(createWriteStream("clean.mp3"));
    ```
    
    ### CLI
    
    ```bash
    elevenlabs audio-isolation convert --audio noisy.mp3 --output clean.mp3
    ```
    
    ## Parameters
    
    | Parameter | Type | Default | Description |
    |-----------|------|---------|-------------|
    | `audio` | file (required) | — | Audio file with vocals/speech to isolate |
    | `file_format` | string | `other` | `other` for any encoded audio, or `pcm_s16le_16` for 16-bit PCM mono @ 16kHz little-endian (lower latency) |
    
    ## Isolating from a URL
    
    ```python
    import requests
    from io import BytesIO
    from elevenlabs import ElevenLabs
    
    client = ElevenLabs()
    
    audio_url = "https://example.com/noisy.mp3"
    response = requests.get(audio_url)
    audio_data = BytesIO(response.content)
    
    audio_stream = client.audio_isolation.convert(audio=audio_data)
    
    with open("clean.mp3", "wb") as f:
        for chunk in audio_stream:
            f.write(chunk)
    ```
    
    ## Low-Latency PCM Input
    
    If you already have raw 16-bit PCM mono @ 16kHz, passing `file_format="pcm_s16le_16"` skips decoding and reduces latency:
    
    ```python
    audio_stream = client.audio_isolation.convert(
        audio=pcm_bytes,
        file_format="pcm_s16le_16",
    )
    ```
    
    ## Supported Formats
    
    Any common encoded audio/video container works as input (MP3, WAV, M4A, FLAC, OGG, WebM, MP4, etc.). Response is a streamed MP3 by default.
    
    ## Common Workflows
    
    - **Clean up interview/podcast recordings** — strip room tone, HVAC, traffic before editing.
    - **Prep noisy audio for Speech-to-Text** — isolate voice first, then pass through `speech_to_text.convert()` for better transcription accuracy.
    - **Extract dialogue from mixed tracks** — pull vocals out of a track with music/SFX.
    - **Pre-processing for Voice Changer** — isolate the source voice before applying voice transformation.
    
    ## Error Handling
    
    ```python
    try:
        audio_stream = client.audio_isolation.convert(audio=audio_file)
    except Exception as e:
        print(f"Voice isolation failed: {e}")
    ```
    
    Common errors:
    - **401**: Invalid API key
    - **422**: Invalid parameters (e.g. wrong `file_format` for the supplied audio)
    - **429**: Rate limit exceeded
    
    ## References
    
    - [Installation Guide](references/installation.md)
    
    Original on GitHub

Author and licence

These skills were written and published by ElevenLabs at github.com/elevenlabs/skills under the MIT licence. skillsmcp copies them unchanged, checks the repository once a day, and is not affiliated with ElevenLabs. A skill is a set of instructions an AI agent may follow, so read a skill before you let an agent use it.

Show the licence
MIT License

Copyright (c) 2024 ElevenLabs

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.