Speech to Text API integration: Developer guide

A Speech to Text API allows developers to integrate speech recognition and transcription capabilities directly into their applications. But before you connect the STT API, you first need to make a few architectural decisions.

Which transcription mode will you use? How will you handle long audio files? What about making sure product names or proper nouns are spelled correctly? A little foresight and planning goes a long way here to creating a scalable Speech to Text API integration in your developer apps.

This guide covers everything you need for an ElevenLabs Speech to Text API integration, with clear code examples you can paste directly into production. As references, be sure to have the ElevenLabs Speech to Text quickstart and the API guide up in a different window.

Summary

• There are two ElevenLabs Speech to Text modes: batch (for pre-recorded audio) and realtime (for live audio streams via WebSocket).
• Scribe v2 handles batch transcription across 90+ languages with speaker diarization, keyterm prompting, entity detection, word-level timestamps, and multichannel support.
• Scribe v2 Realtime handles live streaming with ~150ms latency, delivering partial transcripts as you speak and committed transcripts when a speech segment completes.
• For files over 8 minutes, Scribe v2 automatically chunks and transcribes in parallel. For long-running jobs, use webhooks instead of waiting on the synchronous response.
• Keyterm prompting biases the model toward specific terms using context - more reliable than keyword lists for product names, technical jargon, or unusual proper nouns.

Speech to text API integration: Batch vs. realtime

Every ElevenLabs STT API integration starts with an architectural choice: batch or realtime. While both of these will deliver a powerful STT model to your developer environment, your choice here impacts everything from the capabilities you can access to the end-to-end latency.

Here's how the two modes compare:

• Batch (Scribe v2): Takes a complete audio or video file, transcribes it, and returns the full transcript in a single response. It supports the widest set of features: up to 1,000 keyterms, up to 32 speakers for diarization, entity detection, multichannel mode, and webhooks for async delivery. Files up to 3 GB and 10 hours are supported in standard mode.
• Realtime (Scribe v2 Realtime): Accepts a live audio stream over a WebSocket and returns partial and committed transcripts as the audio comes in, at around 150ms latency. It supports up to 50 keyterms and word-level timestamps, and is the right choice for voice assistants, live captioning, or any application where a user is speaking and expecting a response.

Let's break the differences down further.

As a quick rule: use batch when the audio is complete before transcription starts and realtime when you are transcribing audio as it is being produced.

When to choose Scribe v2 vs. Scribe v2 Realtime

Batch and realtime cover different jobs, meaning picking the wrong one early tends to surface as a rework later.

For more insight, here's how you can match the model to your use case:

• Scribe v2 is the right choice for anything where the audio is complete before processing starts, including meeting recordings, podcast transcription, media files, or anything you process offline. It supports the full feature set including diarization, entity detection, and up to 1,000 keyterms.
• Scribe v2 Realtime is built for live audio. It accepts a WebSocket stream and returns transcripts as the audio arrives, making it the right choice for voice assistants or any scenario where a user is speaking and your application needs to respond before they finish.

One additional factor to take into consideration is that accuracy also varies by language. It's worth checking the transcription word error rate before you commit to a language mix. Scribe v2 publishes Word Error Rate (WER) tiers across all 90+ supported languages.

Excellent accuracy (≤5% WER) covers the major European languages, Japanese, Indonesian, and Vietnamese, among others. High accuracy (5-10% WER) covers Hindi, Bengali, Mandarin, Korean, and Georgian, among others. Check the category breakdown in the language support documentation for full details here.

Setting up the ElevenLabs Speech to Text API

Getting a working client only requires two simple steps. First, you'll install the SDK. Second, you'll store your credentials securely.

Here's how to do both before you write your first transcription request.

Install the SDK and store your API key as a managed secret using a .env file or your platform's secrets manager. Never hardcode your API key into your application.

Python

pip install elevenlabs
pip install python-dotenv

TypeScript

npm install @elevenlabs/elevenlabs-js

Create a .env file:

ELEVENLABS_API_KEY=<your_api_key_here>

Initialize the client:

Python

import os
from dotenv import load_dotenv
from elevenlabs.client import ElevenLabs

load_dotenv()

elevenlabs = ElevenLabs(
    api_key=os.getenv("ELEVENLABS_API_KEY"),
)

TypeScript

import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";

const elevenlabs = new ElevenLabsClient({
  apiKey: process.env.ELEVENLABS_API_KEY,
});

Your first batch transcription with the ElevenLabs STT API

Now that you've initialized your client, you're ready to send your first file.

The batch API takes a file, transcribes it, and returns the full result synchronously. The example below transcribes a remote audio file with speaker diarization and audio event tagging enabled.

Python

# example.py
import os
from dotenv import load_dotenv
from io import BytesIO
import requests
from elevenlabs.client import ElevenLabs

load_dotenv()

elevenlabs = ElevenLabs(
    api_key=os.getenv("ELEVENLABS_API_KEY"),
)

audio_url = (
    "https://storage.googleapis.com/eleven-public-cdn/audio/marketing/nicole.mp3"
)
response = requests.get(audio_url)
audio_data = BytesIO(response.content)

transcription = elevenlabs.speech_to_text.convert(
    file=audio_data,
    model_id="scribe_v2",          # Model to use
    tag_audio_events=True,          # Tag audio events like laughter, applause, etc.
    language_code="eng",            # Language of the audio file. If set to None, the model will detect the language automatically.
    diarize=True,                   # Whether to annotate who is speaking
)

print(transcription)

Run it:

python example.py

The response object includes the full transcript text, word-level entries with timestamps and speaker IDs, and detected audio events.

Each word entry carries a type field with one of three values:

• word: For transcribed words in the audio.
• spacing: Any spaces between words in languages that use spaces. This type isn't applicable for several languages, like Japanese, Cantonese, and Burmese.
• audio_event: A tag for any non-speech sound, like a laugh or coughing.

Here is what the response structure looks like:

{
  "language_code": "en",
  "language_probability": 1,
  "text": "With a soft and whispery American accent, I'm the ideal choice for creating ASMR content, meditative guides, or adding an intimate feel to your narrative projects.",
  "words": [
    {
      "text": "With",
      "start": 0.119,
      "end": 0.259,
      "type": "word",
      "speaker_id": "speaker_0"
    },
    {
      "text": " ",
      "start": 0.239,
      "end": 0.299,
      "type": "spacing",
      "speaker_id": "speaker_0"
    },
    {
      "text": "a",
      "start": 0.279,
      "end": 0.359,
      "type": "word",
      "speaker_id": "speaker_0"
    }
  ]
}

The language_probability field tells you how confident the model is in its language detection, scored on a scale of 0.00 to 1.00.

STT API Realtime integration

Realtime STT API integration follows a slightly different pattern. Instead of one request and one response, you'll open a WebSocket connection and read transcripts as they arrive.

The realtime API uses a WebSocket to accept a live audio stream and return transcripts as the audio arrives. It delivers two transcript types:

• Partial transcripts: Interim results that update as the model processes incoming audio. These may change.
• Committed transcripts: Finalized results for a completed speech segment. These will not change. These may also contain word-level timestamps if you set the "include timestamps" option to true.

The client-side implementation uses a single-use token rather than your API key directly. This is a temporary credential that expires after 15 minutes, generated server-side so your API key is never exposed to the browser.

Step 1: Generate a single-use token (server side)

// Node.js server
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";

const elevenlabs = new ElevenLabsClient({
  apiKey: process.env.ELEVENLABS_API_KEY,
});

app.get("/scribe-token", yourAuthMiddleware, async (req, res) => {
  const token = await elevenlabs.tokens.singleUse.create("realtime_scribe");

  res.json(token);
});

Step 2: Connect and transcribe (client side, React)

import { useScribe } from "@elevenlabs/react";

function MyComponent() {
  const scribe = useScribe({
    modelId: "scribe_v2_realtime",
    onPartialTranscript: (data) => {
      console.log("Partial:", data.text);
    },
    onCommittedTranscript: (data) => {
      console.log("Committed:", data.text);
    },
    onCommittedTranscriptWithTimestamps: (data) => {
      console.log("Committed with timestamps:", data.text);
      console.log("Timestamps:", data.words);
    },
  });

  const handleStart = async () => {
    // Fetch a single use token from the server
    const token = await fetchTokenFromServer();

    await scribe.connect({
      token,
      microphone: {
        echoCancellation: true,
        noiseSuppression: true,
      },
    });
  };

  return (
    <div>
      <button onClick={handleStart} disabled={scribe.isConnected}>
        Start Recording
      </button>
      <button onClick={scribe.disconnect} disabled={!scribe.isConnected}>
        Stop
      </button>

      {scribe.partialTranscript && <p>Live: {scribe.partialTranscript}</p>}

      <div>
        {scribe.committedTranscripts.map((t) => (
          <p key={t.id}>{t.text}</p>
        ))}
      </div>
    </div>
  );
}

The useScribe hook manages the WebSocket connection lifecycle, microphone access, and transcript state. partialTranscript holds the current in-progress text; committedTranscripts is the growing array of finalized segments.

For server-side streaming (transcribing audio from a URL or file stream rather than a microphone), see the server-side streaming guide.

Concurrency and scaling long files

Long files need a different scaling model than most APIs use. It's worth understanding before you build around it.

Concurrency for batch transcription works differently from most APIs. Rather than limiting how many requests you can send simultaneously, Scribe v2 handles long files by automatically parallelizing internally.

Files over 8 minutes are split into segments and transcribed concurrently. The number of concurrent segments used is calculated as:

Concurrency = min(4, round_up(audio_duration_secs / 480))

Concretely:

• A 15-minute file uses a concurrency of 2
• A 120-minute file uses a concurrency of 4 (the maximum)

Files up to 10 hours and 3 GB are supported in standard mode. Note: multichannel mode has a lower duration limit - see the Multichannel transcription section below.

In terms of supported formats, the STT API accepts the most common audio and video formats:

• Audio: AAC, AIFF, OGG, MP3, OPUS, WAV, FLAC, M4A, WebM.
• Video: MP4, AVI, MKV, MOV, WMV, FLV, WebM, MPEG, 3GPP.

You can pass a video file directly and receive a transcript of its audio track without any preprocessing.

Keyterm prompting

Generic models tend to mis-transcribe brand names and technical jargon. Keyterm prompting sets out to fix that.

Keyterm prompting biases the model toward specific words or phrases when transcribing. It is the right tool when your audio contains product names, technical terms, or proper nouns with unusual spellings.

One thing that enhances keyterm prompting is that it uses context, making it more reliable than a simple list of keywords. If you supply "ElevenLabs" as a keyterm, the model will transcribe it correctly when the speaker says the company name. Without this, you might find the model still transcribing "I've worked at eleven labs for a year" incorrectly.

Without keyterm prompting:

I work at eleven labs.

With keyterms=["ElevenLabs"]:

I work at ElevenLabs.

Batch supports up to 1,000 keyterms (50 characters each). Realtime supports up to 50 keyterms (20 characters each).

Batch transcription with keyterms

Python

import os
from dotenv import load_dotenv
from io import BytesIO
import requests
from elevenlabs.client import ElevenLabs

load_dotenv()

elevenlabs = ElevenLabs(
    api_key=os.getenv("ELEVENLABS_API_KEY"),
)

audio_url = (
    "https://storage.googleapis.com/eleven-public-cdn/documentation_assets/audio/stt-keyterm-prompting.mp3"
)
response = requests.get(audio_url)
audio_data = BytesIO(response.content)

transcription = elevenlabs.speech_to_text.convert(
    file=audio_data,
    model_id="scribe_v2",
    # Keyterms to prompt the model with.
    # Up to 1,000 keyterms can be provided, with a maximum length of 50 characters each
    keyterms=["ElevenLabs"],
)

print(transcription)

Realtime streaming with keyterms

Pass keyterms when connecting to the realtime WebSocket:

Python

connection = await elevenlabs.speech_to_text.realtime.connect(RealtimeUrlOptions(
    model_id="scribe_v2_realtime",
    keyterms=["ElevenLabs"],
))

Or pass them as query parameters directly to the WebSocket URL:

wss://api.elevenlabs.io/v1/speech-to-text/realtime?model_id=scribe_v2_realtime&keyterms=ElevenLabs&keyterms=AnotherTerm

Keyterm prompting carries an additional cost. See the API pricing page for details.

Advanced features

While the steps above take you through the core STT API transcription flow, there are several features that enrich the final outputs. Here are some advanced features you can use in your speech to text API:

• Speaker diarization to identify who is speaking
• No-verbatim mode to clean up transcriptions
• Entity detection to flag sensitive data
• Multichannel transcription to separate audio channels

Let's break these down further.

Speaker diarization

Enable diarize=True in your batch request to annotate who is speaking. Scribe v2 supports up to 32 speakers. Each word in the response includes a speaker_id field (e.g. speaker_0, speaker_1) that you can use to separate transcript segments by speaker.

Diarization is useful for meeting transcription, interview processing, or any multi-speaker recording where attributing words to the correct speaker matters downstream.

No-verbatim mode

When no_verbatim=True, the model removes filler words, false starts, and disfluencies from the transcript. "Erm, M-maybe we should, uh, go with option A" becomes "Maybe we should go with option A."

This produces cleaner output for subtitles, summaries, or any use case where readability matters more than capturing every spoken sound. It is available for both batch (scribe_v2) and realtime (scribe_v2_realtime).

Entity detection and redaction

Scribe v2 can detect and label entities in the transcript, with exact timestamps for each detected entity. Categories include PII (names, credit card numbers, SSNs), PHI (medical conditions), and PCI (payment card information), among others. For the full list of supported entity types, see the entity detection documentation.

This is particularly useful for compliance use cases, allowing you to identify and automatically redact sensitive information from a transcript before storing or displaying it.

Entity detection carries an additional cost of $0.070 (per hour) on the base transcription cost. See the API pricing page for details.

Multichannel transcription

When use_multi_channel=True, each audio channel is transcribed independently and assigned a speaker ID based on its channel number. Up to 5 channels are supported. The maximum file duration in multichannel mode is 1 hour.

Multichannel mode is useful when you have separate audio tracks per speaker - for example, a phone call recording where each participant is on their own channel. This produces more accurate speaker attribution than diarization on a mixed-down mono file.

Async delivery with webhooks

Polling for a result is fine at low volume but will fail to scale once you start working with long files or with a high-throughput pipeline. Webhooks solve that issue by pushing the result to you instead.

For long files or high-volume transcription pipelines, waiting on the synchronous response is not always practical. Webhooks let you submit a transcription request and receive the result at your endpoint when processing completes, without polling.

Setting up a webhook

In the ElevenLabs dashboard, go to Developers > Webhooks, click Create webhook, and configure:

• Name: Write in a descriptive and memorable name for your webhook.
• Callback URL: Add a publicly accessible HTTPS endpoint to the webhook.
• Webhook Auth Method: Select either HMAC or OAuth (optional but strongly recommended for security).
• Events: Select Transcription completed from the options.

Submitting a transcription with webhook delivery

Python

from dotenv import load_dotenv
from elevenlabs.client import ElevenLabs

load_dotenv()

elevenlabs = ElevenLabs(
    api_key=os.getenv("ELEVENLABS_API_KEY"),
)

def transcribe_with_webhook(audio_file):
    try:
        result = elevenlabs.speech_to_text.convert(
            file=audio_file,
            model_id="scribe_v2",
            webhook=True,
        )
        print(f"Transcription started: {result.request_id}")
        return result
    except Exception as e:
        print(f"Error starting transcription: {e}")
        raise e

When webhook=True, the request returns early without the transcript. The completed transcript is delivered to your endpoint as a POST request when processing finishes.

Implementing your webhook endpoint

import { ElevenLabsClient } from '@elevenlabs/elevenlabs-js';
import 'dotenv/config';
import express from 'express';

const elevenlabs = new ElevenLabsClient();
const app = express();
app.use(express.json());

const WEBHOOK_SECRET = process.env.WEBHOOK_SECRET;

app.post('/webhook/speech-to-text', (req, res) => {
  try {
    const signature = req.headers['elevenlabs-signature'];
    const payload = JSON.stringify(req.body);
    let event;

    try {
      // Verify the webhook signature.
      event = await elevenlabs.webhooks.constructEvent(payload, signature, WEBHOOK_SECRET);
    } catch (error) {
      return res.status(401).json({ error: 'Invalid signature' });
    }

    if (event.type === 'speech_to_text.completed') {
      const { requestId, status, text, language_code } = event.data;

      console.log(`Transcription ${requestId} completed`);
      console.log(`Language: ${language_code}`);
      console.log(`Text: ${text}`);

      processTranscription(requestId, text, language_code);
    } else if (status === 'failed') {
      console.error(`Transcription ${requestId} failed`);
      handleTranscriptionError(requestId);
    }

    res.status(200).json({ received: true });
  } catch (error) {
    console.error('Webhook error:', error);
    res.status(500).json({ error: 'Internal server error' });
  }
});

app.listen(3000, () => {
  console.log('Webhook server listening on port 3000');
});

Webhook payload structure

Your endpoint receives a POST with the following shape:

{
  "type": "speech_to_text_transcription",
  "data": {
    "request_id": "some-request-id-123",
    "webhook_metadata": {},
    "transcription": {
      "language_code": "en",
      "language_probability": 0.98,
      "text": "Hello world!",
      "words": [
        {
          "text": "Hello",
          "start": 0.0,
          "end": 0.5,
          "type": "word",
          "speaker_id": "speaker_1"
        }
      ]
    }
  }
}

Webhook security best practices

When building with webhooks, there are a few security measures you can take to ensure events are delivered and processed correctly.

• Verify webhook signatures: Always verify webhook signatures using elevenlabs.webhooks.constructEvent() to confirm they originate from ElevenLabs.
• Use HTTPS endpoints: Webhook URLs must use HTTPS to protect data in transit from tampering.
• Return the appropriate HTTP status code: Return 200-299 for successful processing, 400-499 for client errors (these will not be retried), and 500-599 for server errors (these will be retried).
• Use a tunneling tool for local development: For local development, use a tunneling tool such as ngrok to expose your local server with a public HTTPS URL.

With these strategies, you'll be able to use webhooks with confidence.

Key takeaways for your Speech to Text API integration

A production Speech to Text API integration comes down to a handful of decisions.

Get these right and everything else follows:

• Choose the mode by use case: Use batch (scribe_v2) when the audio is complete before processing starts. Use realtime (scribe_v2_realtime) when you need to transcribe as audio is being produced, such as for agents, voice assistants, or live captions.
• Use keyterms for accuracy on specific vocabulary: Pass product names, technical terms, or unusual proper nouns as keyterms. The model uses context to apply them correctly without over-triggering.
• Let the API handle long files: Files over 8 minutes are automatically parallelized, meaning you don't need to chunk them yourself. Files up to 10 hours and 3 GB are supported in standard mode.
• Use webhooks for async pipelines: For long-running jobs or high-volume processing, webhook=True lets you submit and move on. Verify the signature on every incoming webhook payload.
• Enable no-verbatim mode for clean output: If your use case is subtitles, summaries, or downstream NLP processing, no_verbatim=True removes filler words and disfluencies automatically.
• Use multichannel mode for separated audio tracks: If your recording has a channel per speaker (call center recordings, interview setups), multichannel transcription produces more accurate speaker attribution than diarization on a mixed file. Note the 1-hour duration limit in this mode.

If you'd like even more information, explore the full API reference as a starting point.

Build your Speech to Text integration with ElevenAPI

After reading through this guide, you have every pattern you need for a production Speech to Text API integration. Across batch and realtime transcription, and including advanced features like keyterm prompting and async delivery, you'll be ready to launch your app with STT connected.

Get started by learning more about the Speech to Text API or sign up to make your first call with ElevenAPI today.

FAQs about STT API integration