Edit

Share via

Summarize text with the ai.summarize function

The ai.summarize function uses generative AI to produce summaries of input text. It uses only a single line of code. The function can either summarize values from one column of a DataFrame or values across all the columns.

AI functions improve data engineering by using the power of large language models in Microsoft Fabric. To learn more, see this overview article.

Important

This feature is in preview, for use in the Fabric Runtime 1.3 and later.

  • Review the prerequisites in this overview article, including the library installations that are temporarily required to use AI functions.
  • By default, the gpt-4o-mini model currently powers AI functions. Learn more about billing and consumption rates.
  • Although the underlying model can handle several languages, most of the AI functions are optimized for use on English-language texts.
  • During the initial rollout of AI functions, users are temporarily limited to 1,000 requests per minute with the built-in AI endpoint in Fabric.

Use ai.summarize with pandas

The ai.summarize function extends the pandas Series class. To summarize each row value from that column alone, call the function on a pandas DataFrame text column. You can also call the ai.summarize function on an entire DataFrame to summarize values across all the columns.

The function returns a pandas Series that contains summaries, which can be stored in a new DataFrame column.

Syntax

df["summaries"] = df["text"].ai.summarize()

Parameters

None

Returns

The function returns a pandas Series that contains summaries for each input text row. If the input text is null, the result is null.

Example

# This code uses AI. Always review output for mistakes. 
# Read terms: https://azure.microsoft.com/support/legal/preview-supplemental-terms/.

df= pd.DataFrame([
        ("Microsoft Teams", "2017",
        """
        The ultimate messaging app for your organization—a workspace for real-time 
        collaboration and communication, meetings, file and app sharing, and even the 
        occasional emoji! All in one place, all in the open, all accessible to everyone.
        """),
        ("Microsoft Fabric", "2023",
        """
        An enterprise-ready, end-to-end analytics platform that unifies data movement, 
        data processing, ingestion, transformation, and report building into a seamless, 
        user-friendly SaaS experience. Transform raw data into actionable insights.
        """)
    ], columns=["product", "release_year", "description"])

df["summaries"] = df["description"].ai.summarize()
display(df)

Use ai.summarize with PySpark

The ai.summarize function is also available for Spark DataFrames. If you specify the name of an existing input column as a parameter, the function summarizes each value from that column alone. Otherwise, the function summarizes values across all columns of the DataFrame, row by row.

The function returns a new DataFrame with summaries for each input text row, from a single column or across all the columns, stored in an output column.

Syntax

df.ai.summarize(input_col="text", output_col="summaries")

Parameters

Name Description
input_col
Optional		 A string that contains the name of an existing column with input text values to summarize. If you don't set this parameter, the function summarizes values across all columns in the DataFrame, instead of values from a specific column.
output_col
Optional		 A string that contains the name of a new column to store summaries for each input text row. If you don't set this parameter, a default name generates for the output column.
error_col
Optional		 A string that contains the name of a new column to store any OpenAI errors that result from processing each input text row. If you don't set this parameter, a default name generates for the error column. If an input row has no errors, the value in this column is null.

Returns

The function returns a Spark DataFrame that includes a new column that contains summarized text for each input text row. If the input text is null, the result is null. If no input column is specified, the function summarizes values across all columns in the DataFrame.

Example

# This code uses AI. Always review output for mistakes. 
# Read terms: https://azure.microsoft.com/support/legal/preview-supplemental-terms/.

df = spark.createDataFrame([
        ("Microsoft Teams", "2017",
        """
        The ultimate messaging app for your organization—a workspace for real-time 
        collaboration and communication, meetings, file and app sharing, and even the 
        occasional emoji! All in one place, all in the open, all accessible to everyone.
        """,),
        ("Microsoft Fabric", "2023",
        """
        An enterprise-ready, end-to-end analytics platform that unifies data movement, 
        data processing, ingestion, transformation, and report building into a seamless, 
        user-friendly SaaS experience. Transform raw data into actionable insights.
        """,)
    ], ["product", "release_year", "description"])

summaries = df.ai.summarize(input_col="description", output_col="summaries")
display(summaries)

Related content