Summarize Your Audio Data

Learn how to summarize your audio using LeMUR

In this guide, you’ll learn how to use LeMUR to summarize your audio data with key takeaways.

If you want a Quickstart, see Apply LLMs to audio files.

Before you start

To use LeMUR, you need an AssemblyAI account with a credit card set up.

Basic summary example

If you want to send a custom prompt to the LLM, you can use the LeMUR Task and apply the model to your transcribed audio files.

To summarize the content in your audio data, define a summarization prompt and call transcript.lemur.task(). The underlying transcript is automatically used as additional context for the model.

Python

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

# Step 1: Transcribe an audio file.
# audio_file = "./local_file.mp3"
audio_file = "https://assembly.ai/sports_injuries.mp3"

transcriber = aai.Transcriber()
transcript = transcriber.transcribe(audio_file)

# Step 2: Define a summarization prompt.
prompt = "Provide a brief summary of the transcript."

# Step 3: Apply LeMUR.
result = transcript.lemur.task(
    prompt, final_model=aai.LemurModel.claude3_5_sonnet
)

print(result.response)

Example output

The transcript describes several common sports injuries - runner's knee,
sprained ankle, meniscus tear, rotator cuff tear, and ACL tear. It provides
definitions, causes, and symptoms for each injury. The transcript seems to be
narrating sports footage and describing injuries as they occur to the athletes.
Overall, it provides an overview of these common sports injuries that can result
from overuse or sudden trauma during athletic activities

Summary with specialized endpoint

The LeMUR Summary function requires no prompt engineering and facilitates more deterministic and structured outputs. You can use it with transcript.lemur.summarize().

You can add additional context to provide information that is not explicitly referenced in the audio data, as well as specify an answer format. For this, use the optional parameters context and answer_format.

Python

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

audio_url = "https://assembly.ai/meeting.mp4"
transcript = aai.Transcriber().transcribe(audio_url)

result = transcript.lemur.summarize(
    final_model=aai.LemurModel.claude3_5_sonnet,
    context="A GitLab meeting to discuss logistics",
    answer_format="TLDR"
)

print(result.response)

Custom summary example (Advanced)

In this example, we’ll run a custom LeMUR task with an advanced prompt to create custom summaries:

Cookbook: Custom summary with LeMUR Task

More summarization prompt examples

Try any of these prompts to get started:

Use case	Example prompt
Summaries	”Summarize key decisions and important points from the phone call transcript”
Summarize audio segments	”Summarize the key events of each chapter”

For more use cases and prompt examples, see LeMUR examples.

API reference

• LeMUR Task endpoint
• LeMUR Summary endpoint

Improve the results

To improve the results, see the following resources:

• Optimize your prompt with the prompt engineering guide.
• To alter the outcome, see Change model and parameters.