> ## Documentation Index
> Fetch the complete documentation index at: https://assembly-preview.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Apply LLMs to audio files

> Learn how to leverage LLMs for speech using LeMUR.

## Overview

A Large Language Model (LLM) is a machine learning model that uses natural language processing (NLP) to generate text. [LeMUR](https://assemblyai.com/docs/api-reference/lemur) is a framework that lets you apply LLMs to audio transcripts, for example to ask questions about a call, or to summarize a meeting.

By the end of this tutorial, you'll be able to use LeMUR to summarize an audio file.

Here's the full sample code for what you'll build in this tutorial:

<CodeGroup>
  ```python Python theme={"system"}
  import assemblyai as aai

  aai.settings.api_key = "YOUR_API_KEY"

  transcriber = aai.Transcriber()

  # You can use a local filepath:
  # audio_file = "./example.mp3"

  # Or use a publicly-accessible URL:
  audio_file = (
      "https://assembly.ai/sports_injuries.mp3"
  )
  transcript = transcriber.transcribe(audio_file)

  prompt = "Provide a brief summary of the transcript."

  result = transcript.lemur.task(
      prompt, final_model=aai.LemurModel.claude3_5_sonnet
  )

  print(result.response)
  ```

  ```typescript TypeScript theme={"system"}
  import { AssemblyAI } from 'assemblyai'

  const client = new AssemblyAI({
    apiKey: 'YOUR_API_KEY'
  })

  // You can use a local filepath:
  // const audioFile = "./example.mp3"

  // Or use a publicly-accessible URL:
  const audioFile =
    'https://assembly.ai/sports_injuries.mp3'

  const run = async () => {
    const transcript = await client.transcripts.transcribe({ audio: audioFile })

    const prompt = 'Provide a brief summary of the transcript.'

    const { response } = await client.lemur.task({
      transcript_ids: [transcript.id],
      prompt,
      final_model: 'anthropic/claude-3-5-sonnet'
    })

    console.log(response)
  }

  run()
  ```

  ```go Go theme={"system"}
  package main

  import (
      "context"
      "fmt"

      aai "github.com/AssemblyAI/assemblyai-go-sdk"
  )

  func main() {
      ctx := context.Background()

      client := aai.NewClient("YOUR_API_KEY")

      // You can use a local file:
      /*
      f, err := os.Open("./example.mp3")
      [error handling here]
      transcript, err := client.Transcripts.TranscribeFromReader(ctx, f, params)
      */

      // Or use a publicly-accessible URL:
      audioURL := "https://assembly.ai/sports_injuries.mp3"

      transcript, _ := client.Transcripts.TranscribeFromURL(ctx, audioURL, nil)

      prompt := "Provide a brief summary of the transcript."

      var params aai.LeMURTaskParams
      params.Prompt = aai.String(prompt)
      params.TranscriptIDs = []string{aai.ToString(transcript.ID)}
      params.FinalModel = "anthropic/claude-3-5-sonnet"

      result, _ := client.LeMUR.Task(ctx, params)

      fmt.Println(*result.Response)
  }
  ```

  ```java Java theme={"system"}
  import com.assemblyai.api.AssemblyAI;
  import com.assemblyai.api.resources.lemur.requests.LemurTaskParams;
  import com.assemblyai.api.resources.transcripts.types.Transcript;
  import java.util.List;

  public final class App {
      public static void main(String... args) {

          AssemblyAI client = AssemblyAI.builder()
                  .apiKey("YOUR_API_KEY")
                  .build();

          // You can use a local file:
          /*
          Transcript transcript = aai.transcripts().transcribe(
                  new File("./example.mp3"), params);
          */

          // Or use a publicly-accessible URL:
          String audioUrl = "https://assembly.ai/sports_injuries.mp3";
          Transcript transcript = client.transcripts().transcribe(audioUrl);

          var params = LemurTaskParams.builder()
                  .prompt("Provide a brief summary of the transcript.")
                  .transcriptIds(List.of(transcript.getId()))
                  .finalModel(LemurModel.ANTHROPIC_CLAUDE3_5_SONNET)
                  .build();

          var result = client.lemur().task(params);

          System.out.println(result.getResponse());
      }
  }
  ```

  ```csharp C# theme={"system"}
  using AssemblyAI;
  using AssemblyAI.Lemur;
  using AssemblyAI.Transcripts;

  var client = new AssemblyAIClient("YOUR_API_KEY");

  // You can use a local file:
  /*
  var transcript = await client.Transcripts.TranscribeAsync(
      new FileInfo("./example.mp3")
  );
  */

  // Or use a publicly-accessible URL:
  const string audioUrl = "https://assembly.ai/sports_injuries.mp3";
  var transcript = await client.Transcripts.TranscribeAsync(new TranscriptParams
  {
      AudioUrl = audioUrl
  });

  var lemurTaskParams = new LemurTaskParams
  {
      Prompt = "Provide a brief summary of the transcript.",
      TranscriptIds = [transcript.Id],
      FinalModel = LemurModel.AnthropicClaude3_5_Sonnet
  };

  var response = await client.Lemur.TaskAsync(lemurTaskParams);

  Console.WriteLine(response.Response);
  ```

  ```ruby Ruby theme={"system"}
  require 'assemblyai'

  client = AssemblyAI::Client.new(api_key: 'YOUR_API_KEY')

  # You can upload and transcribe a local file:
  # uploaded_file = client.files.upload(file: '/path/to/your/file')
  # transcript = client.transcripts.transcribe(audio_url: uploaded_file.upload_url, speaker_labels: true)

  # Or use a publicly-accessible URL:
  audio_url = 'https://assembly.ai/sports_injuries.mp3'

  transcript = client.transcripts.transcribe(audio_url: audio_url)

  result = client.lemur.task(
    transcript_ids: [transcript.id],
    prompt: 'Provide a brief summary of the transcript.',
    final_model: AssemblyAI::Lemur::LemurModel::ANTHROPIC_CLAUDE3_5_SONNET
  )

  puts result.response
  ```
</CodeGroup>

If you run the code above, you'll see the following output:

```
The transcript describes several common sports injuries - runner's knee,sprained ankle, meniscus tear, rotator cuff tear, and ACL tear. It providesdefinitions, causes, and symptoms for each injury. The transcript seems to benarrating sports footage and describing injuries as they occur to the athletes.Overall, it provides an overview of these common sports injuries that can resultfrom overuse or sudden trauma during athletic activities
```

## Before you begin

To complete this tutorial, you need:

* [Python](https://www.python.org/), [TypeScript](https://www.typescriptlang.org/), [Go](https://go.dev), Java, [.NET](https://dotnet.microsoft.com/en-us/download), or [Ruby](https://www.ruby-lang.org/en/documentation/installation/) installed.
* An .
* Basic understanding of how to [Transcribe an audio file](/docs/getting-started/transcribe-an-audio-file).

## Step 1: Install the SDK[​](#step-1-install-the-sdk "Direct link to Step 1: Install the SDK")

Install the package via pip:

<Tabs>
  <Tab title="Python">
    ```bash theme={"system"}
    pip install assemblyai
    ```
  </Tab>

  <Tab title="TypeScript">
    ```bash theme={"system"}
    npm install assemblyai
    ```
  </Tab>

  <Tab title="Go">
    ```bash theme={"system"}
    go get github.com/AssemblyAI/assemblyai-go-sdk
    ```
  </Tab>

  <Tab title="Java">
    <Tabs>
      <Tab title="Maven">
        ```xml theme={"system"}
        <dependency>
            <groupId>com.assemblyai</groupId>
            <artifactId>assemblyai-java</artifactId>
            <version>ASSEMBLYAI_SDK_VERSION</version>
        </dependency>
        ```
      </Tab>

      <Tab title="Gradle">
        ```Kotlin theme={"system"}
        dependencies {
            implementation 'com.assemblyai:assemblyai-java:ASSEMBLYAI_SDK_VERSION'
        }
        ```
      </Tab>
    </Tabs>
  </Tab>

  <Tab title="C#">
    <Tabs>
      <Tab title=".NET CLI">
        ```
        dotnet add package AssemblyAI
        ```
      </Tab>

      <Tab title="Package Manager">
        ```bash theme={"system"}
        Install-Package AssemblyAI
        ```
      </Tab>

      <Tab title="PackageReference">
        ```xml theme={"system"}
        <PackageReference Include="AssemblyAI" Version="ASSEMBLYAI_SDK_VERSION" />
        ```
      </Tab>
    </Tabs>
  </Tab>

  <Tab title="Ruby">
    ```bash theme={"system"}
    gem install assemblyai
    ```
  </Tab>
</Tabs>

## Step 2: Transcribe an audio file

LeMUR uses one or more transcripts as input to generate text output. In this step, you'll transcribe an audio file that you can later use to create a prompt for.

For more information about transcribing audio, see [Transcribe an audio file](/docs/getting-started/transcribe-an-audio-file).

<CodeGroup>
  ```python Python theme={"system"}
  import assemblyai as aai

  aai.settings.api_key = "YOUR_API_KEY"

  transcriber = aai.Transcriber()

  audio_url = "https://assembly.ai/sports_injuries.mp3"

  transcript = transcriber.transcribe(audio_url)
  ```

  ```typescript TypeScript theme={"system"}
  import { AssemblyAI } from 'assemblyai'

  const client = new AssemblyAI({
    apiKey: 'YOUR_API_KEY'
  })

  const audioUrl =
    'https://assembly.ai/sports_injuries.mp3'

  const run = async () => {
    const transcript = await client.transcripts.transcribe({ audio: audioUrl })
  }

  run()
  ```

  ```go Go theme={"system"}
  package main

  import (
      "context"

      aai "github.com/AssemblyAI/assemblyai-go-sdk"
  )

  func main() {
      ctx := context.Background()

      client := aai.NewClient("YOUR_API_KEY")

      audioURL := "https://assembly.ai/sports_injuries.mp3"

      transcript, _ := client.Transcripts.TranscribeFromURL(ctx, audioURL, nil)
  }
  ```

  ```java Java theme={"system"}
  import com.assemblyai.api.AssemblyAI;
  import com.assemblyai.api.resources.lemur.requests.LemurTaskParams;
  import com.assemblyai.api.resources.transcripts.types.Transcript;
  import java.util.List;

  public final class App {

  public static void main(String... args) {

          AssemblyAI client = AssemblyAI.builder()
                  .apiKey("YOUR_API_KEY")
                  .build();

          String audioUrl = "https://assembly.ai/sports_injuries.mp3";

          Transcript transcript = client.transcripts().transcribe(audioUrl);
      }
  }
  ```

  ```csharp C# theme={"system"}
  using AssemblyAI;
  using AssemblyAI.Lemur;
  using AssemblyAI.Transcripts;

  var client = new AssemblyAIClient("YOUR_API_KEY");

  const string audioUrl = "https://assembly.ai/sports_injuries.mp3";

  var transcript = await client.Transcripts.TranscribeAsync(new TranscriptParams
  {
      AudioUrl = audioUrl
  });
  ```

  ```ruby Ruby theme={"system"}
  require 'assemblyai'

  client = AssemblyAI::Client.new(api_key: 'YOUR_API_KEY')

  audio_url = 'https://assembly.ai/sports_injuries.mp3'

  transcript = client.transcripts.transcribe(audio_url: audio_url)
  ```
</CodeGroup>

<Tip>
  If you've already transcribed an audio file you want to use, you can get an existing transcript using its ID. You can find the ID for previously transcribed audio files in the .

  <CodeGroup>
    ```python Python theme={"system"}
    transcript = aai.Transcript.get_by_id("YOUR_TRANSCRIPT_ID")
    ```

    ```typescript TypeScript theme={"system"}
    const transcript = await client.transcripts.get('YOUR_TRANSCRIPT_ID')
    ```

    ```go Go theme={"system"}
    transcript, _ := client.Transcripts.Get(ctx, "YOUR_TRANSCRIPT_ID")
    ```

    ```java Java theme={"system"}
    Transcript transcript = client.transcripts().get("YOUR_TRANSCRIPT_ID");
    ```

    ```csharp C# theme={"system"}
    var transcript = await client.Transcripts.GetAsync("YOUR_TRANSCRIPT_ID");
    ```

    ```ruby Ruby theme={"system"}
    transcript = client.transcripts.get(transcript_id: 'YOUR_TRANSCRIPT_ID')
    ```
  </CodeGroup>
</Tip>

## Step 3: Prompt LeMUR to generate text output

In this step, you'll create a [Custom task](https://assemblyai.com/docs/api-reference/lemur/task) with LeMUR and use the transcript you created in the previous step as input.

The input to a custom task is called a *prompt*. A prompt is a text string that provides LeMUR with instructions on how to generate the text output.

For more techniques on how to build prompts, see [Improving your prompt](/docs/lemur/improving-your-prompt).

<Steps>
  <Step>
    Write a prompt with instructions on how LeMUR should generate the text output.

    <CodeGroup>
      ```python Python theme={"system"}
      prompt = "Provide a brief summary of the transcript."
      ```

      ```typescript TypeScript theme={"system"}
      const prompt = 'Provide a brief summary of the transcript.'
      ```

      ```go Go theme={"system"}
      prompt := "Provide a brief summary of the transcript."
      ```

      ```java Java theme={"system"}
      var params = LemurTaskParams.builder()
                   .prompt("Provide a brief summary of the transcript.")
                   .transcriptIds(List.of(transcript.getId()))
                   .finalModel(LemurModel.ANTHROPIC_CLAUDE3_5_SONNET)
                   .build();
      ```

      ```csharp C# theme={"system"}
      var prompt = "Provide a brief summary of the transcript.";
      ```

      ```ruby Ruby theme={"system"}
      prompt = 'Provide a brief summary of the transcript.'
      ```
    </CodeGroup>
  </Step>

  <Step>
    Create a custom task with LeMUR, using the transcript and prompt as input. The final model defines the LLM to use to process the task. For available models to choose from, see [Change the model type](/docs/lemur/customize-parameters#change-the-model-type).

    <CodeGroup>
      ```python Python theme={"system"}
      result = transcript.lemur.task(
       prompt, final_model=aai.LemurModel.claude3_5_sonnet
      )
      ```

      ```typescript TypeScript theme={"system"}
      const { response } = await client.lemur.task({
      transcript_ids: [transcript.id],
      prompt,
      final_model: 'anthropic/claude-3-5-sonnet'
      })
      ```

      ```go Go theme={"system"}
      var params aai.LeMURTaskParams
      params.Prompt = aai.String(prompt)
      params.TranscriptIDs = []string{aai.ToString(transcript.ID)}
      params.FinalModel = "anthropic/claude-3-5-sonnet"

      result, _ := client.LeMUR.Task(ctx, params)
      ```

      ```java Java theme={"system"}
      var result = client.lemur().task(params);
      ```

      ```csharp C# theme={"system"}
      var response = await client.Lemur.TaskAsync(new LemurTaskParams
      {
       Prompt = prompt,
       TranscriptIds = [transcript.Id],
       FinalModel = LemurModel.AnthropicClaude3_5_Sonnet
      });
      ```

      ```ruby Ruby theme={"system"}
      result = client.lemur.task(
      transcript_ids: [transcript.id],
      prompt: prompt,
      final_model: AssemblyAI::Lemur::LemurModel::ANTHROPIC_CLAUDE3_5_SONNET
      )
      ```
    </CodeGroup>
  </Step>

  <Step>
    Print the result.

    <CodeGroup>
      ```python Python theme={"system"}
      print(result.response)
      ```

      ```typescript TypeScript theme={"system"}
      console.log(response)
      ```

      ```go Go theme={"system"}
      fmt.Println(*result.Response)
      ```

      ```java Java theme={"system"}
      System.out.println(result.getResponse());
      ```

      ```csharp C# theme={"system"}
      Console.WriteLine(response.Response);
      ```

      ```ruby Ruby theme={"system"}
      puts result.response
      ```
    </CodeGroup>

    The output will look something like this:

    ```
     The transcript describes several common sports injuries - runner's knee, sprained ankle, meniscus tear, rotator cuff tear, and ACL tear. It provides definitions, causes, and symptoms for each injury. The transcript seems to be narrating sports footage and describing injuries as they occur to the athletes. Overall, it provides an overview of these common sports injuries that can result from overuse or sudden trauma during athletic activities
    ```
  </Step>
</Steps>

## Next steps

In this tutorial, you've learned how to generate LLM output based on your audio transcripts. The type of output depends on your prompt, so try exploring different prompts to see how they affect the output. Here's a few more prompts to try.

* "Provide an analysis of the transcript and offer areas to improve with exact quotes."
* "What's the main take-away from the transcript?"
* "Generate a set of action items from this transcript."

To learn more about how to apply LLMs to your transcripts, see the following resources:

* [Ask questions about your audio data using LeMUR](/docs/lemur/ask-questions)
* [Writing good prompts](/docs/lemur/improving-your-prompt)

## Need some help?

If you get stuck, or have any other questions, we'd love to help you out. Ask our support team in our [Discord server](https://discord.gg/aSMMpMadFh).