Ask questions about your audio data

In this guide, you’ll learn how to use LeMUR to ask questions and get answers about your audio data.

If you want a Quickstart, see Apply LLMs to audio files.

To use LeMUR, you need an with a credit card set up.

Basic Q&A example

If you want to send a custom prompt to the LLM, you can use the LeMUR Task and apply the model to your transcribed audio files.

To ask question about your audio data, define a prompt with your questions and call transcript.lemur.task(). The underlying transcript is automatically used as additional context for the model.

python

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

# Step 1: Transcribe an audio file.
# audio_file = "./local_file.mp3"
audio_file = "https://assembly.ai/sports_injuries.mp3"

transcriber = aai.Transcriber()
transcript = transcriber.transcribe(audio_file)

# Step 2: Define a prompt with your question(s).
prompt = "What is a runner's knee?"

# Step 3: Apply LeMUR.
result = transcript.lemur.task(
    prompt, final_model=aai.LemurModel.claude3_5_sonnet
)

print(result.response)

Example output

Based on the transcript, runner's knee is a condition characterizedby pain behind or around the kneecap. It is caused by overuse,muscle imbalance and inadequate stretching. Symptoms include painunder or around the kneecap and pain when walking.

Q&A with specialized endpoint

The LeMUR Question & Answer function requires no prompt engineering and facilitates more deterministic and structured outputs. You can use it with transcript.lemur.question().

To use it, define a list of aai.LemurQuestion objects. For each question, you can define additional context and specify either a answer_format or a list of answer_options. Additionally, you can define an overall context.

python

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

audio_url = "https://assembly.ai/meeting.mp4"
transcript = aai.Transcriber().transcribe(audio_url)

questions = [
    aai.LemurQuestion(
        question="What are the top level KPIs for engineering?",
        context="KPI stands for key performance indicator",
        answer_format="short sentence"),
    aai.LemurQuestion(
        question="How many days has it been since the data team has gotten updated metrics?",
        answer_options=["1", "2", "3", "4", "5", "6", "7", "more than 7"]),
]

result = transcript.lemur.question(
    final_model=aai.LemurModel.claude3_5_sonnet,
    questions,
    context="A GitLab meeting to discuss logistics"
)

for qa_response in result.response:
    print(f"Question: {qa_response.question}")
    print(f"Answer: {qa_response.answer}")

Custom Q&A example (Advanced)

This example shows how you can run a custom LeMUR task with an advanced prompt to create custom Q&A responses:

Cookbook: Custom Q&A with LeMUR Task

More Q&A prompt examples

Try any of these prompts to get started:

Use case	Example prompt
Question and answer	”Identify any patterns or trends based on the transcript”
Closed-ended questions	”Did the customer express a positive sentiment in the phone call?”
Sentiment analysis	”What was the emotional sentiment of the phone call?”

For more use cases and prompt examples, see LeMUR examples.

API reference

• LeMUR Task endpoint
• LeMUR Question & Answer endpoint

Improve the results

To improve the results, see the following resources:

• Optimize your prompt with the prompt engineering guide.
• To alter the outcome, see Change model and parameters.