> ## Documentation Index > Fetch the complete documentation index at: https://assembly-preview.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Speaker Diarization > Learn how to detect multiple speakers in an audio The Speaker Diarization model lets you detect multiple speakers in an audio file and what each speaker said. If you enable Speaker Diarization, the resulting transcript will return a list of *utterances*, where each utterance corresponds to an uninterrupted segment of speech from a single speaker. Speaker Diarization and multichannel Speaker Diarization doesn't support multichannel transcription. Enabling both Speaker Diarization and [multichannel](/docs/speech-to-text/speech-recognition#multichannel-transcription) will result in an error. ## Quickstart To enable Speaker Diarization, set `speaker_labels` to `true` in the transcription config. ```python Python theme={"system"} import assemblyai as aai aai.settings.api_key = "YOUR_API_KEY" # You can use a local filepath: # audio_file = "./example.mp3" # Or use a publicly-accessible URL: audio_file = ( "https://assembly.ai/wildfires.mp3" ) config = aai.TranscriptionConfig( speaker_labels=True, ) transcript = aai.Transcriber().transcribe(audio_file, config) for utterance in transcript.utterances: print(f"Speaker {utterance.speaker}: {utterance.text}") ``` ```typescript Typescript theme={"system"} import { AssemblyAI } from 'assemblyai' const client = new AssemblyAI({ apiKey: 'YOUR_API_KEY' }) // You can use a local filepath: // const audioFile = "./example.mp3" // Or use a publicly-accessible URL: const audioFile = 'https://assembly.ai/wildfires.mp3' const params = { audio: audioFile, speaker_labels: true } const run = async () => { const transcript = await client.transcripts.transcribe(params) for (const utterance of transcript.utterances!) { console.log(`Speaker ${utterance.speaker}: ${utterance.text}`) } } run() ``` ```go Go theme={"system"} package main import ( "context" "fmt" "os" aai "github.com/AssemblyAI/assemblyai-go-sdk" ) func main() { ctx := context.Background() client := aai.NewClient("YOUR_API_KEY") // You can use a local file: /* f, err := os.Open("./example.mp3") [error handling here] transcript, err := client.Transcripts.TranscribeFromReader(ctx, f, &aai.TranscriptOptionalParams{ SpeakerLabels: aai.Bool(true), }) */ // Or use a publicly-accessible URL: audioURL := "https://assembly.ai/wildfires.mp3" transcript, _ := client.Transcripts.TranscribeFromURL(ctx, audioURL, &aai.TranscriptOptionalParams{ SpeakerLabels: aai.Bool(true), }) for _, utterance := range transcript.Utterances { fmt.Printf("Speaker %s: %s\n", aai.ToString(utterance.Speaker), aai.ToString(utterance.Text), ) } } ``` ```java Java theme={"system"} import com.assemblyai.api.AssemblyAI; import com.assemblyai.api.resources.transcripts.types.*; public final class App { public static void main(String[] args) { AssemblyAI client = AssemblyAI.builder() .apiKey("YOUR_API_KEY") .build(); var params = TranscriptOptionalParams.builder() .speakerLabels(true) .build(); // You can use a local file: /* Transcript transcript = aai.transcripts().transcribe( new File("./example.mp3"), params); */ // Or use a publicly-accessible URL: String audioUrl = "https://assembly.ai/wildfires.mp3"; Transcript transcript = client.transcripts().transcribe(audioUrl, params); transcript.getUtterances().get().forEach(utterance -> System.out.println("Speaker " + utterance.getSpeaker() + ": " + utterance.getText()) ); } } ``` ```csharp C# theme={"system"} using AssemblyAI; using AssemblyAI.Transcripts; var client = new AssemblyAIClient("YOUR_API_KEY"); // You can use a local file: /* var transcript = await client.Transcripts.TranscribeAsync( new FileInfo("./example.mp3"), new TranscriptOptionalParams { SpeakerLabels = true } ); */ // Or use a publicly-accessible URL: const string audioUrl = "https://assembly.ai/wildfires.mp3"; var transcript = await client.Transcripts.TranscribeAsync(new TranscriptParams { AudioUrl = audioUrl, SpeakerLabels = true }); foreach (var utterance in transcript.Utterances!) { Console.WriteLine($"Speaker {utterance.Speaker}: {utterance.Text}"); } ``` ```ruby Ruby theme={"system"} require 'assemblyai' client = AssemblyAI::Client.new(api_key: 'YOUR_API_KEY') # You can upload and transcribe a local file: # uploaded_file = client.files.upload(file: '/path/to/your/file') # transcript = client.transcripts.transcribe(audio_url: uploaded_file.upload_url, speaker_labels: true) # Or use a publicly-accessible URL: audio_url = 'https://assembly.ai/wildfires.mp3' transcript = client.transcripts.transcribe( audio_url: audio_url, speaker_labels: true ) transcript.utterances.each do |utterance| printf('Speaker %s: %s', speaker: utterance.speaker, text: utterance.text) end ``` ### Example output ```text theme={"system"} Speaker A: Smoke from hundreds of wildfires in Canada is triggering air quality alerts throughout the US. Skylines from Maine to Maryland to Minnesota are gray and smoggy. And in some places, the air quality warnings include the warning to stay inside. We wanted to better understand what's happening here and why, so we called Peter DiCarlo, an associate professor in the Department of Environmental Health and Engineering at Johns Hopkins University. Good morning, professor. Speaker B: Good morning. Speaker A: So what is it about the conditions right now that have caused this round of wildfires to affect so many people so far away? Speaker B: Well, there's a couple of things. The season has been pretty dry already, and then the fact that we're getting hit in the US. Is because there's a couple of weather systems that are essentially channeling the smoke from those Canadian wildfires through Pennsylvania into the Mid Atlantic and the Northeast and kind of just dropping the smoke there. Speaker A: So what is it in this haze that makes it harmful? And I'm assuming it is. ... ``` ## Set number of speakers If you know the number of speakers in advance, you can improve the diarization performance by setting the `speakers_expected` parameter. ```python Python theme={"system"} config = aai.TranscriptionConfig( speaker_labels=True, speakers_expected=3 ) ``` ```typescript Typescript theme={"system"} const params = { audio: audioUrl, speaker_labels: true, speakers_expected: 3 } ``` ```go Go theme={"system"} transcript, _ := client.Transcripts.TranscribeFromURL(ctx, audioURL, &aai.TranscriptOptionalParams{ SpeakerLabels: aai.Bool(true), SpeakersExpected: aai.Int64(3), }) ``` ```java Java theme={"system"} var params = TranscriptOptionalParams.builder() .speakerLabels(true) .speakersExpected(3) .build(); ``` ```csharp C# theme={"system"} var transcript = await client.Transcripts.TranscribeAsync(new TranscriptParams { AudioUrl = audioUrl, SpeakerLabels = true, SpeakersExpected = 3 }); ``` ```ruby Ruby theme={"system"} transcript = client.transcripts.transcribe( audio_url: audio_url, speaker_labels: true, speakers_expected: 3 ) ``` The `speakers_expected` parameter is ignored for audio files with a duration less than 2 minutes. ## API reference ### Request ```sh theme={"system"} curl https://api.assemblyai.com/v2/transcript \ --header "Authorization: YOUR_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "audio_url": "YOUR_AUDIO_URL", "speaker_labels": true, "speakers_expected": 3 }' ``` | Key | Type | Description | | ------------------ | ------- | -------------------------------------------------- | | `speaker_labels` | boolean | Enable Speaker Diarization. | | `speaker_expected` | number | [Set number of speakers](#set-number-of-speakers). | ### Response ```text theme={"system"} {utterances:[...]} ``` | Key | Type | Description | | ----------------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `utterances` | array | A turn-by-turn temporal sequence of the transcript, where the i-th element is an object containing information about the i-th utterance in the audio file. | | `utterances[i].confidence` | number | The confidence score for the transcript of this utterance. | | `utterances[i].end` | number | The ending time, in milliseconds, of the utterance in the audio file. | | `utterances[i].speaker` | string | The speaker of this utterance, where each speaker is assigned a sequential capital letter. For example, "A" for Speaker A, "B" for Speaker B, and so on. | | `utterances[i].start` | number | The starting time, in milliseconds, of the utterance in the audio file. | | `utterances[i].text` | string | The transcript for this utterance. | | `utterances[i].words` | array | A sequential array for the words in the transcript, where the j-th element is an object containing information about the j-th word in the utterance. | | `utterances[i].words[j].text` | string | The text of the j-th word in the i-th utterance. | | `utterances[i].words[j].start` | number | The starting time for when the j-th word is spoken in the i-th utterance, in milliseconds. | | `utterances[i].words[j].end` | number | The ending time for when the j-th word is spoken in the i-th utterance, in milliseconds. | | `utterances[i].words[j].confidence` | number | The confidence score for the transcript of the j-th word in the i-th utterance. | | `utterances[i].words[j].speaker` | string | The speaker who uttered the j-th word in the i-th utterance. | The response also includes the request parameters used to generate the transcript. ## Frequently asked questions To improve the performance of the Speaker Diarization model, it's recommended to ensure that each speaker speaks for at least 30 seconds uninterrupted. Avoiding scenarios where a person only speaks a few short phrases like “Yeah”, “Right”, or “Sounds good” can also help. If possible, avoiding cross-talking can also improve performance. The upper limit on the number of speakers for Speaker Diarization is 10. The accuracy of the Speaker Diarization model depends on several factors, including the quality of the audio, the number of speakers, and the length of the audio file. Ensuring that each speaker speaks for at least 30 seconds uninterrupted and avoiding scenarios where a person only speaks a few short phrases can improve accuracy. However, it's important to note that the model isn't perfect and may make mistakes, especially in more challenging scenarios. ## Troubleshooting The speaker diarization may be performing poorly if a speaker only speaks once or infrequently throughout the audio file. Additionally, if the speaker speaks in short or single-word utterances, the model may struggle to create separate clusters for each speaker. Lastly, if the speakers sound similar, there may be difficulties in accurately identifying and separating them. Background noise, cross-talk, or an echo may also cause issues.