Key Phrases

The Key Phrases model identifies significant words and phrases in your transcript and lets you extract the most important concepts or highlights from your audio or video file.

Quickstart

Enable Key Phrases by setting auto_highlights to true in the transcription config.

python

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

# audio_file = "./local_file.mp3"
audio_file = "https://assembly.ai/wildfires.mp3"

config = aai.TranscriptionConfig(auto_highlights=True)

transcript = aai.Transcriber().transcribe(audio_file, config)

for result in transcript.auto_highlights.results:
    print(f"Highlight: {result.text}, Count: {result.count}, Rank: {result.rank}, Timestamps: {result.timestamps}")

Example output

Highlight: air quality alerts, Count: 1, Rank: 0.08, Timestamps: [Timestamp(start=3978, end=5114)]
Highlight: wide ranging air quality consequences, Count: 1, Rank: 0.08, Timestamps: [Timestamp(start=235388, end=238838)]
Highlight: more fires, Count: 1, Rank: 0.07, Timestamps: [Timestamp(start=184716, end=185186)]
...

API reference

Request

curl https://api.assemblyai.com/v2/transcript \
--header "Authorization: YOUR_API_KEY" \
--header "Content-Type: application/json" \
--data '{
  "audio_url": "YOUR_AUDIO_URL",
  "auto_highlights": true
}'

Key	Type	Description
auto_highlights	boolean	Enable Key Phrases.

Response

{
  "auto_highlights_result": {
    "status": "success",
    "results": [...]
  }
}

Key	Type	Description
auto_highlights_result	object	The result of the Key Phrases model.
auto_highlights_result.status	string	Is either success, or unavailable in the rare case that the Key Phrases model failed.
auto_highlights_result.results	array	A temporally-sequential array of key phrases.
auto_highlights_result.results[i].count	number	The total number of times the i-th key phrase appears in the audio file.
auto_highlights_result.results[i].rank	number	The total relevancy to the overall audio file of this key phrase. A greater number means that the key phrase is more relevant.
auto_highlights_result.results[i].text	string	The text itself of the key phrase.
auto_highlights_result.results[i].timestamps[j].start	number	The starting time of the j-th appearance of the i-th key phrase.
auto_highlights_result.results[i].timestamps[j].end	number	The ending time of the j-th appearance of the i-th key phrase.

The response also includes the request parameters used to generate the transcript.

Frequently Asked Questions

How does the Key Phrases model identify important phrases in my transcription?

The Key Phrases model uses natural language processing and machine learning algorithms to analyze the frequency and distribution of words and phrases in your transcription. The algorithm identifies key phrases based on their relevancy score, which takes into account factors such as the number of times a phrase occurs, the distance between occurrences, and the overall length of the transcription.

What is the difference between the Key Phrases model and the Topic Detection model?

The Key Phrases model is designed to identify important phrases and words in your transcription, whereas the Topic Detection model is designed to categorize your transcription into predefined topics. While both models use natural language processing and machine learning algorithms, they have different goals and approaches to analyzing your text.

Can the Key Phrases model handle misspelled or unrecognized words?

Yes, the Key Phrases model can handle misspelled or unrecognized words to some extent. However, the accuracy of the detection may depend on the severity of the misspelling or the obscurity of the word. It’s recommended to provide high-quality, relevant audio files with accurate transcriptions for the best results.

What are some limitations of the Key Phrases model?

Some limitations of the Key Phrases model include its limited understanding of context, which may lead to inaccuracies in identifying the most important phrases in certain cases, such as text with heavy use of jargon or idioms. Additionally, the model assigns higher scores to words or phrases that occur more frequently in the text, which may lead to an over-representation of common words and phrases that may not be as important in the context of the text. Finally, the Key Phrases model is a general-purpose algorithm that can’t be easily customized or fine-tuned for specific domains, meaning it may not perform as well for specialized texts where certain keywords or concepts may be more important than others.

How can I optimize the performance of the Key Phrases model?

To optimize the performance of the Key Phrases model, it’s recommended to provide high-quality, relevant audio files with accurate transcriptions, to review and adjust the model’s configuration parameters, such as the confidence threshold for key phrase detection, and to refer to the list of identified key phrases to guide the analysis. It may also be helpful to consider adding additional training data to the model or consulting with AssemblyAI support for further assistance.