> ## Documentation Index
> Fetch the complete documentation index at: https://assembly-preview.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming

> Learn how to transcribe live audio streams

# Streaming Speech-to-Text

AssemblyAI's Streaming Speech-to-Text (STT) allows you to transcribe live audio streams with high accuracy and low latency. By streaming your audio data to our secure WebSocket API, you can receive transcripts back within a few hundred milliseconds.

<Info>
  Supported languages

  Streaming Speech-to-Text is only available for English. See [Supported languages](/docs/getting-started/supported-languages).
</Info>

## Getting started

Get started with any of our official SDKs:

<Info>
  Getting Started Guides

  * [Python](/docs/getting-started/transcribe-streaming-audio-from-a-microphone/python)
  * [JavaScript / TypeScript](/docs/getting-started/transcribe-streaming-audio-from-a-microphone/typescript)
  * [Go](/docs/getting-started/transcribe-streaming-audio-from-a-microphone/go)
  * [Java](/docs/getting-started/transcribe-streaming-audio-from-a-microphone/java)
  * [C#](/docs/getting-started/transcribe-streaming-audio-from-a-microphone/csharp)
</Info>

If your programming language isn't supported yet, see the WebSocket API:

* [Streaming API reference](https://assemblyai.com/docs/api-reference/streaming)
* [Python guide on using Streaming Speech-to-Text](/docs/guides/real-time-streaming-transcription)

## Audio requirements

The audio format must conform to the following requirements:

* PCM16 or Mu-law encoding (See [Specify the encoding](#specify-the-encoding))
* A sample rate that matches the value of the supplied `sample_rate` parameter
* Single-channel
* 100 to 2000 milliseconds of audio per message

<Tip>
  Audio segments with a duration between 100 ms and 450 ms produce the best results in transcription accuracy.
</Tip>

## Specify the encoding

By default, transcriptions expect [PCM16 encoding](http://trac.ffmpeg.org/wiki/audio%20types). If you want to use [Mu-law encoding](http://trac.ffmpeg.org/wiki/audio%20types), you must set the `encoding` parameter to `aai.AudioEncoding.pcm_mulaw`:

<CodeGroup>
  ```python Python theme={"system"}
  transcriber = aai.RealtimeTranscriber(
  ...,
  encoding=aai.AudioEncoding.pcm_mulaw
  )
  ```

  ```typescript Typescript theme={"system"}
  const rt = client.realtime.transcriber({
    ...,
    encoding: 'pcm_mulaw'
  })
  ```

  ```go Go theme={"system"}
  client := aai.NewRealTimeClientWithOptions(
      aai.WithRealTimeAPIKey(apiKey),
      aai.WithHandler(handler),
      aai.WithRealTimeEncoding(aai.RealTimeEncodingPCMMulaw),
  )
  ```

  ```java Java theme={"system"}
  var realtimeTranscriber = RealtimeTranscriber.builder()
    ...
    .encoding(AudioEncoding.PCM_MULAW)
    .build();
  ```

  ```csharp C# theme={"system"}
  await using var transcriber = new RealtimeTranscriber(new RealtimeTranscriberOptions
  {
      ...
      Encoding = AudioEncoding.PcmMulaw
  });
  ```
</CodeGroup>

| Encoding            | SDK Parameter                 | Description                      |
| ------------------- | ----------------------------- | -------------------------------- |
| **PCM16** (Default) | `aai.AudioEncoding.pcm_s16le` | PCM signed 16-bit little-endian. |
| **Mu-law**          | `aai.AudioEncoding.pcm_mulaw` | PCM Mu-law.                      |

## Add custom vocabulary

You can add up to 2500 characters of custom vocabulary to boost their transcription probability.

For this, create a list of strings and set the `word_boost` parameter:

<CodeGroup>
  ```python Python theme={"system"}
  transcriber = aai.RealtimeTranscriber(
      ...,
      word_boost=["aws", "azure", "google cloud"]
  )
  ```

  ```typescript Typescript theme={"system"}
  const rt = client.realtime.transcriber({
    ...,
    wordBoost:['aws', 'azure', 'google cloud']
  })
  ```

  ```go Go theme={"system"}
  client := aai.NewRealTimeClientWithOptions(
      aai.WithRealTimeAPIKey(apiKey),
      aai.WithHandler(handler),
      aai.WithRealTimeWordBoost([]string{"aws", "azure", "google cloud"}),
  )
  ```

  ```java Java theme={"system"}
  var realtimeTranscriber = RealtimeTranscriber.builder()
    ...
    .wordBoost(List.of("aws", "azure", "google cloud"))
    .build();
  ```

  ```csharp C# theme={"system"}
  await using var transcriber = new RealtimeTranscriber(new RealtimeTranscriberOptions
  {
      ...,
      WordBoost = ["aws", "azure", "google cloud"]
  });
  ```
</CodeGroup>

<Info>
  If you're not using one of the SDKs, you must ensure that the `word_boost` parameter is a JSON array that is URL encoded. See this [code example](/docs/guides/real-time-streaming-transcription#adding-custom-vocabulary).
</Info>

## Authenticate with a temporary token

If you need to authenticate on the client, you can avoid exposing your API key by using temporary authentication tokens. You should generate this token on your server and pass it to the client.

<Steps>
  <Step>
    To generate a temporary token, call `aai.RealtimeTranscriber.create_temporary_token()`.
    Use the `expires_in` parameter to specify how long the token should be valid for, in seconds.

    <CodeGroup>
      ```python Python theme={"system"}
      token = aai.RealtimeTranscriber.create_temporary_token(
          expires_in=60
      )
      ```

      ```typescript Typescript theme={"system"}
      const token = await client.realtime.createTemporaryToken({ expires_in: 60 })
      ```

      ```go Go theme={"system"}
      client := aai.NewClient("YOUR_API_KEY")

      resp, _ := client.RealTime.CreateTemporaryToken(ctx, 60)
      ```

      ```java Java theme={"system"}
      var tokenResponse = client.realtime().createTemporaryToken(CreateRealtimeTemporaryTokenParams.builder()
        .expiresIn(60)
        .build()
      );
      ```

      ```csharp C# theme={"system"}
      var tokenResponse = await client.Realtime.CreateTemporaryTokenAsync(expiresIn: 60);
      ```
    </CodeGroup>

    <Info>
      Note

      The expiration time must be a value between 60 and 360000 seconds.
    </Info>
  </Step>

  <Step>
    The client should retrieve the token from the server and use the token to authenticate the transcriber.

    <Info>
      Note

      Each token has a one-time use restriction and can only be used for a single session.

      To use it, specify the `token` parameter when initializing the streaming transcriber.
    </Info>

    <CodeGroup>
      ```python Python theme={"system"}
      transcriber = aai.RealtimeTranscriber(
          ...,
          token=token
      )
      ```

      ```typescript Typescript theme={"system"}
      import { RealtimeTranscriber } from 'assemblyai';

      // TODO: implement getToken to retrieve token from server
      const token = await getToken();

      const rt = new RealtimeTranscriber({
        ...,
        token
      })
      ```

      ```go Go theme={"system"}
      client := aai.NewRealTimeClientWithOptions(
          aai.WithRealTimeAuthToken(resp.Token),
          aai.WithHandler(handler),
      )
      ```

      ```java Java theme={"system"}
      var realtimeTranscriber = RealtimeTranscriber.builder()
        ...
        .token(tokenResponse.getToken())
        .build();
      ```

      ```csharp C# theme={"system"}
      await using var transcriber = new RealtimeTranscriber(new RealtimeTranscriberOptions
      {
          Token = tokenResponse.Token,
          ...
      });
      ```
    </CodeGroup>
  </Step>
</Steps>

## Manually end current utterance

To manually end an utterance, call `force_end_utterance()`:

<CodeGroup>
  ```python Python theme={"system"}
  transcriber.force_end_utterance()
  ```

  ```typescript Typescript theme={"system"}
  rt.forceEndUtterance()
  ```

  ```go Go theme={"system"}
  client.ForceEndUtterance(ctx)
  ```

  ```java Java theme={"system"}
  realtimeTranscriber.forceEndUtterance()
  ```

  ```csharp C# theme={"system"}
  await transcriber.ForceEndUtteranceAsync();
  ```
</CodeGroup>

Manually ending an utterance immediately produces a final transcript.

## Configure the threshold for automatic utterance detection

You can configure the threshold for how long to wait before ending an utterance.

To change the threshold, you can specify the `end_utterance_silence_threshold` parameter when initializing the streaming transcriber.

After the session has started, you can change the threshold by calling `configure_end_utterance_silence_threshold()`.

<CodeGroup>
  ```python Python theme={"system"}
  transcriber = aai.RealtimeTranscriber(
      ...,
      end_utterance_silence_threshold=500
  )

  transcriber.configure_end_utterance_silence_threshold(300)
  ```

  ```typescript Typescript theme={"system"}
  const rt = client.realtime.transcriber({
    ...,
    endUtteranceSilenceThreshold: 500
  })

  // after connecting

  rt.configureEndUtteranceSilenceThreshold(300)
  ```

  ```go Go theme={"system"}
  client.SetEndUtteranceSilenceThreshold(ctx, 500)
  ```

  ```java Java theme={"system"}
  var realtimeTranscriber = RealtimeTranscriber.builder()
    ...
    .endUtteranceSilenceThreshold(500)
    .build();

  // after connecting

  realtimeTranscriber.configureEndUtteranceSilenceThreshold(300)
  ```

  ```csharp C# theme={"system"}
  await transcriber.ConfigureEndUtteranceThresholdAsync(500);
  ```
</CodeGroup>

<Info>
  Note

  By default, Streaming Speech-to-Text ends an utterance after 700 milliseconds of silence. You can configure the duration threshold any number of times during a session after the session has started. The valid range is between 0 and 20000.
</Info>

## Disable partial transcripts

If you're only using the final transcript, you can disable partial transcripts to reduce network traffic.

To disable partial transcripts, set the `disable_partial_transcripts` parameter to `True`.

<CodeGroup>
  ```python Python theme={"system"}
  transcriber = aai.RealtimeTranscriber(
      ...,
      disable_partial_transcripts=True
  )
  ```

  ```typescript Typescript theme={"system"}
  const rt = client.realtime.transcriber({
    ...,
    disablePartialTranscripts: true
  })
  ```

  ```go Go theme={"system"}
  // Partial transcripts are disabled by default.
  ```

  ```java Java theme={"system"}
  var realtimeTranscriber = RealtimeTranscriber.builder()
    ...
    .disablePartialTranscripts()
    .build();
  ```

  ```csharp C# theme={"system"}
  await using var transcriber = new RealtimeTranscriber(new RealtimeTranscriberOptions
  {
      ...,
      DisablePartialTranscripts = true
  });
  ```
</CodeGroup>

## Enable extra session information

If you enable extra session information, the client receives a `RealtimeSessionInformation` message right before receiving the session termination message.

To enable it, define a callback function to handle the event and cofigure the `on_extra_session_information` parameter.

<CodeGroup>
  ```python Python theme={"system"}
  # Define a callback to handle the session information message
  def on_extra_session_information(data: aai.RealtimeSessionInformation):
      print(data.audio_duration_seconds)

  # Configure the RealtimeTranscriber
  transcriber = aai.RealtimeTranscriber(
      ...,
      on_extra_session_information=on_extra_session_information,
  )
  ```

  ```typescript Typescript theme={"system"}
  const rt = client.realtime.transcriber({
    ...
  })

  rt.on('session_information', (info: SessionInformation) => console.log(info));
  ```

  ```go Go theme={"system"}
  transcriber := &aai.RealTimeTranscriber{
      OnSessionInformation: func(event aai.SessionInformation) {
          fmt.Println(event.AudioDurationSeconds)
      }
  }
  client := aai.NewRealTimeClientWithOptions(
      aai.WithRealTimeAPIKey("YOUR_API_KEY"),
      aai.WithRealTimeTranscriber(transcriber),
  )
  ```

  ```java Java theme={"system"}
  var realtimeTranscriber = RealtimeTranscriber.builder()
    ...
    .onSessionInformation((info) -> System.out.println(info.getAudioDurationSeconds()))
    .build()
  ```

  ```csharp C# theme={"system"}
  transcriber.SessionInformationReceived.Subscribe(info =>
  {
      Console.WriteLine("Session information:\n- duration: {0}", info.AudioDurationSeconds);
  });
  ```
</CodeGroup>

## Learn more

To learn about using Streaming Speech-to-Text, see the following resources:

* [Blog post: Automatically Transcribe Zoom Calls in Real Time](https://www.assemblyai.com/blog/how-to-automatically-transcribe-zoom-calls/)
* [Blog post: Transcribe Twilio Phone Calls](https://www.assemblyai.com/blog/transcribe-twilio-phone-calls-in-real-time-with-assemblyai/)
* [Blog post: Real Time Speech Recognition with Python and PyAudio](https://www.assemblyai.com/blog/real-time-speech-recognition-with-python/)
* [GitHub: End-to-end examples](https://github.com/AssemblyAI-Community/docs-snippets/tree/main/real-time)
* [GitHub: Cookbook examples](https://github.com/AssemblyAI/cookbook/tree/master/real-time)
* [GitHub: Use Express.js for Streaming Speech-to-Text](https://github.com/AssemblyAI/realtime-transcription-browser-js-example)
* [Streaming API reference](https://assemblyai.com/docs/api-reference/streaming)