Reasoning is important aspect for advanced LLM in business – such as, in autonomous agents.
Today, a lot of reasoning techniques are introduced in many papers, – such as, ReAct, ReWOO, LLM Compiler, and more.
In this blog post, I’ll show you how to implement custom Planner (which performs custom structured planning) for advanced reasoning in Semantic Kernel.
Note : To implement ReAct (Reasoning+Acting) planning in Semantic Kernel, you can use built-in Stepwise Planner.
Planner – Deep Dive
Before starting the implementation of custom planner, let’s see the brief outline of components in Semantic Kernel – Plugins, Functions, and Planners.
The core component in Semantic Kernel is plugin.
A plugin in Semantic Kernel defines a set of semantic functions and native functions.
The semantic function performs LLM prompting and the native function runs native code.
When some task is given, the Planner decomposes into subtasks (in which available plugins and functions are formulated) to generate a goal-oriented plan to achieve the given task.
From : Semantic Kernel developer blog
In order to understand Planner, let’s see the tutorial example, 05-using-the-planner.ipynb, in official GitHub repository.
In this example, the following 3 built-in plugins (SummarizePlugin, WriterPlugin, TextPlugin) are imported into kernel.
from semantic_kernel.core_plugins.text_plugin import TextPlugin
plugins_directory = "../../samples/plugins/"
summarize_plugin = kernel.import_plugin_from_prompt_directory(plugins_directory, "SummarizePlugin")
writer_plugin = kernel.import_plugin_from_prompt_directory(plugins_directory, "WriterPlugin")
text_plugin = kernel.import_plugin_from_object(TextPlugin(), "TextPlugin")Next we build the built-in BasicPlanner as follows.
This Planner builds a plan, in which all functions in plugins are executed sequentially.
from semantic_kernel.planners.basic_planner import BasicPlanner
service_id = "default"
planner = BasicPlanner(service_id)Now let’s build a plan for the following task by using BasicPlanner.
“Tomorrow is Valentine’s day. I need to come up with a few short poems.
She likes Shakespeare so write using his style. She speaks French so write it in French.
Convert the text to uppercase.”
As you can see the result below, the plan generated by BasicPlanner includes the following 5 functions, and these functions are then called sequentially. Each function gets the output of previous function as an input. :
WriterPlugin.Brainstorm: This function asks LLM for ideas with bullet’s points. (semantic function)WriterPlugin.ShortPoem: This function asks LLM to create a poem. (semantic function)WriterPlugin.Shakespeare: This function asks LLM to convert text into Shakespeare-style text. (semantic function)WriterPlugin.Translate: This function asks LLM to translate text. (semantic function)TextPlugin.uppercase: This function changes text to uppercase. (native function)
ask = """
Tomorrow is Valentine's day. I need to come up with a few short poems.
She likes Shakespeare so write using his style. She speaks French so write it in French.
Convert the text to uppercase."""
new_plan = await planner.create_plan(goal=ask, kernel=kernel)
print(new_plan.generated_plan)output
{
"input": "Valentine's Day Poems",
"subtasks": [
{"function": "WriterPlugin.Brainstorm"},
{"function": "WriterPlugin.ShortPoem"},
{"function": "WriterPlugin.Shakespeare"},
{"function": "WriterPlugin.Translate", "args": {"language": "French"}},
{"function": "TextPlugin.uppercase"}
]
}Note : The above input’s text (“Valentine’s Day Poems”) was automatically generated by
SummarizePlugin.
When you run this plan, you can eventually get a poem for Valentine’s ideas in uppercase French.
results = await planner.execute_plan(new_plan, kernel)
print(results)output
ASSUREZ-VOUS D'UTILISER UNIQUEMENT LE FRANÇAIS.
Ô BELLES ROSES D'UN ROUGE CRAMOISI,
ET VIOLETTES D'UN BLEU VIOLET,
LA DOUCEUR COMME DU SUCRE IMBUE,
ET DANS TA GRÂCE, JE TE VOIS VRAIE.
MAIS OH, AMOUR, COMME TU ES TUMULTUEUX,
RAGEANT COMME UNE TEMPÊTE, TUMULTUEUX.
MON CŒUR BAT LA CHAMADE ET S'AGITE,
DANS TES TOURMENTS, JE SUIS IMPUISSANT.
TU ES MON SOLEIL, MA LUNE BRILLANTE,
POURTANT PARFOIS, TU ME METS EN DIFFICULTÉ.
L'AMOUR, UN LUTIN CAPRICIEUX ET FANTASQUE,
ME LAISSE SOUVENT DANS UN TRISTE ÉTAT.What is Planner doing internally ?
Simply put, BasicPlanner is a simple Python class which invokes kernel functions for reasoning and execution.
First, it submits the following prompt to LLM (in this case, OpenAI GPT) by running semantic function in create_plan() method.
As you can see the below, the prompt for LLM includes a few-shot exemplars and also what kind of functions are available.
prompt (input)
You are a planner for the Semantic Kernel.
Your job is to create a properly formatted JSON plan step by step, to satisfy the goal given.
Create a list of subtasks based off the [GOAL] provided.
Each subtask must be from within the [AVAILABLE FUNCTIONS] list. Do not use any functions that are not in the list.
Base your decisions on which functions to use from the description and the name of the function.
Sometimes, a function may take arguments. Provide them if necessary.
The plan should be as short as possible.
For example:
[AVAILABLE FUNCTIONS]
EmailConnector.LookupContactEmail
description: looks up the a contact and retrieves their email address
args:
- name: the name to look up
WriterPlugin.EmailTo
description: email the input text to a recipient
args:
- input: the text to email
- recipient: the recipient's email address. Multiple addresses may be included if separated by ';'.
WriterPlugin.Translate
description: translate the input to another language
args:
- input: the text to translate
- language: the language to translate to
WriterPlugin.Summarize
description: summarize input text
args:
- input: the text to summarize
FunPlugin.Joke
description: Generate a funny joke
args:
- input: the input to generate a joke about
[GOAL]
"Tell a joke about cars. Translate it to Spanish"
[OUTPUT]
{
"input": "cars",
"subtasks": [
{"function": "FunPlugin.Joke"},
{"function": "WriterPlugin.Translate", "args": {"language": "Spanish"}}
]
}
[AVAILABLE FUNCTIONS]
WriterPlugin.Brainstorm
description: Brainstorm ideas
args:
- input: the input to brainstorm about
EdgarAllenPoePlugin.Poe
description: Write in the style of author Edgar Allen Poe
args:
- input: the input to write about
WriterPlugin.EmailTo
description: Write an email to a recipient
args:
- input: the input to write about
- recipient: the recipient's email address.
WriterPlugin.Translate
description: translate the input to another language
args:
- input: the text to translate
- language: the language to translate to
[GOAL]
"Tomorrow is Valentine's day. I need to come up with a few date ideas.
She likes Edgar Allen Poe so write using his style.
E-mail these ideas to my significant other. Translate it to French."
[OUTPUT]
{
"input": "Valentine's Day Date Ideas",
"subtasks": [
{"function": "WriterPlugin.Brainstorm"},
{"function": "EdgarAllenPoePlugin.Poe"},
{"function": "WriterPlugin.EmailTo", "args": {"recipient": "significant_other"}},
{"function": "WriterPlugin.Translate", "args": {"language": "French"}}
]
}
[AVAILABLE FUNCTIONS]
SummarizePlugin.Topics
description: Analyze given text or document and extract key topics worth remembering
args:
- input:
SummarizePlugin.MakeAbstractReadable
description: Given a scientific white paper abstract, rewrite it to make it more readable
args:
- input:
SummarizePlugin.Summarize
description: Summarize given text or any text document
args:
- input: Text to summarize
SummarizePlugin.Notegen
description: Automatically generate compact notes for any text or text document.
args:
- input:
WriterPlugin.Acronym
description: Generate an acronym for the given concept or phrase
args:
- input:
WriterPlugin.StoryGen
description: Generate a list of synopsis for a novel or novella with sub-chapters
args:
- input:
WriterPlugin.AcronymGenerator
description: Given a request to generate an acronym from a string, generate an acronym and provide the acronym explanation.
args:
- INPUT:
WriterPlugin.TwoSentenceSummary
description: Summarize given text in two sentences or less
args:
- input:
WriterPlugin.Brainstorm
description: Given a goal or topic description generate a list of ideas
args:
- input: A topic description or goal.
WriterPlugin.NovelOutline
description: Generate a list of chapter synopsis for a novel or novella
args:
- input: What the novel should be about.
- chapterCount: The number of chapters to generate.
- endMarker: The marker to use to end each chapter.
WriterPlugin.EmailTo
description: Turn bullet points into an email to someone, using a polite tone
args:
- to:
- input:
- sender:
WriterPlugin.ShortPoem
description: Turn a scenario into a short and entertaining poem.
args:
- input: The scenario to turn into a poem.
WriterPlugin.Translate
description: Translate the input into a language of your choice
args:
- input: Text to translate
- language: Language to translate to
WriterPlugin.NovelChapterWithNotes
description: Write a chapter of a novel using notes about the chapter to write.
args:
- input: What the novel should be about.
- theme: The theme of this novel.
- notes: Notes useful to write this chapter.
- previousChapter: The previous chapter synopsis.
- chapterIndex: The number of the chapter to write.
WriterPlugin.AcronymReverse
description: Given a single word or acronym, generate the expanded form matching the acronym letters.
args:
- INPUT:
WriterPlugin.NovelChapter
description: Write a chapter of a novel.
args:
- input: A synopsis of what the chapter should be about.
- theme: The theme or topic of this novel.
- previousChapter: The synopsis of the previous chapter.
- chapterIndex: The number of the chapter to write.
WriterPlugin.EmailGen
description: Write an email from the given bullet points
args:
- input:
WriterPlugin.Rewrite
description: Automatically generate compact notes for any text or text document
args:
- style:
- input:
WriterPlugin.TellMeMore
description: Summarize given text or any text document
args:
- conversationtype:
- input:
- focusarea:
- previousresults:
WriterPlugin.EnglishImprover
description: Translate text to English and improve it
args:
- INPUT:
WriterPlugin.Shakespeare
description: Convert input to Shakespeare-style text.
args:
- input:
TextPlugin.lowercase
description: Convert a string to lowercase.
args:
- input:
TextPlugin.trim
description: Trim whitespace from the start and end of a string.
args:
- input:
TextPlugin.trim_end
description: Trim whitespace from the end of a string.
args:
- input:
TextPlugin.trim_start
description: Trim whitespace from the start of a string.
args:
- input:
TextPlugin.uppercase
description: Convert a string to uppercase.
args:
- input:
PlannerPlugin.CreatePlan
args:
- available_functions:
- goal:
[GOAL]
Tomorrow is Valentine's day. I need to come up with a few short poems.
She likes Shakespeare so write using his style. She speaks French so write it in French.
Convert the text to uppercase.
[OUTPUT]In accordance with the above exemplars, LLM then replies the following JSON plan.
completion (output)
{
"input": "Valentine's Day Poems",
"subtasks": [
{"function": "WriterPlugin.Brainstorm"},
{"function": "WriterPlugin.ShortPoem"},
{"function": "WriterPlugin.Shakespeare"},
{"function": "WriterPlugin.Translate", "args": {"language": "French"}},
{"function": "TextPlugin.uppercase"}
]
}Finally, the execute_plan() method in BasicPlanner will parse this plan (JSON) and then invoke each functions sequentially.
As I have mentioned above, each function gets the output of previous function as an input.
output of WriterPlugin.Brainstorm
1. Roses are red, violets are blue, sugar is sweet, and so are you.
2. You are the sunshine in my day, the moonlight in my night, and the beat in my heart.
3. Love is not just a feeling, it's a choice we make every day.
4. You are the missing piece in my puzzle, the melody in my song, and the love in my life.
5. I love you more than words can say, more than actions can show, and more than you will ever know.
6. You are the reason I wake up with a smile, the reason I go to bed with a happy heart, and the reason I believe in love.
7. You are my forever Valentine, my soulmate, and my best friend.
8. Love is not about finding someone perfect, it's about finding someone who is perfect for you.
9. You make my heart skip a beat, my world brighter, and my life complete.
##END##output of WriterPlugin.ShortPoem
Roses are red, violets are blue,
Sugar is sweet, and so are you.
But let's be real, we all know,
Love is messy, and it can blow.
You're my sunshine, my moonlight too,
But sometimes you make me feel blue.
Love is aoutput of WriterPlugin.Shakespeare
Oh, fair roses of crimson hue,
And violets of violet blue,
Sweetness like sugar doth imbue,
And in thy grace, I see thee true.
But oh, love, how tumultuous,
Raging like storm, tempestuous.
Mine heart doth flutter and fuss,
In thy throes, I am rendered helpless.
Thou art my sun, my moon alight,
Yet at times, thou bringeth me plight.
Love, a fickle and wayward sprite,
Doth oft leave me in sorry plight.output of WriterPlugin.Translate
Assurez-vous d'utiliser UNIQUEMENT le français.
Ô belles roses d'un rouge cramoisi,
Et violettes d'un bleu violet,
La douceur comme du sucre imbue,
Et dans ta grâce, je te vois vraie.
Mais oh, amour, comme tu es tumultueux,
Rageant comme une tempête, tumultueux.
Mon cœur bat la chamade et s'agite,
Dans tes tourments, je suis impuissant.
Tu es mon soleil, ma lune brillante,
Pourtant parfois, tu me mets en difficulté.
L'amour, un lutin capricieux et fantasque,
Me laisse souvent dans un triste état.output of TextPlugin.uppercase
ASSUREZ-VOUS D'UTILISER UNIQUEMENT LE FRANÇAIS.
Ô BELLES ROSES D'UN ROUGE CRAMOISI,
ET VIOLETTES D'UN BLEU VIOLET,
LA DOUCEUR COMME DU SUCRE IMBUE,
ET DANS TA GRÂCE, JE TE VOIS VRAIE.
MAIS OH, AMOUR, COMME TU ES TUMULTUEUX,
RAGEANT COMME UNE TEMPÊTE, TUMULTUEUX.
MON CŒUR BAT LA CHAMADE ET S'AGITE,
DANS TES TOURMENTS, JE SUIS IMPUISSANT.
TU ES MON SOLEIL, MA LUNE BRILLANTE,
POURTANT PARFOIS, TU ME METS EN DIFFICULTÉ.
L'AMOUR, UN LUTIN CAPRICIEUX ET FANTASQUE,
ME LAISSE SOUVENT DANS UN TRISTE ÉTAT.For details about implementation, let’s see here, the source code of BasicPlanner.
In some cases, a single context will be shared across all plugins and functions in a sequential plan.
For instance, in the restaurant reservations, the information – such as, the datetime, the number of persons, meals, or dietary requirements – may be collected through functions. All these information will then be saved in a single context as key-value pairs, and are finally submitted to reservation’s function.
You can design your own planner as you like.
From : Semantic Kernel Overview
Implement custom Planner – Structured Planning Example
Now I implement a simple Planner example to run simple calculations of company’s invoices. (This example is the same as this post, which is built by LangChain with ReAct prompting pattern.)
In this example, I’ll define the following native functions.
- GetInvoice(name_of_company) :
This extracts and returns the invoice amount of name_of_company from database. - Diff(value1, value2) :
This is a simple calculator and returns the difference between value1 and value2. - Total(values) :
This is also a simple calculator and returns the sum of integer’s list values.
When I ask for the following question (task) :
“How much is the difference between the total of company C, F and the total of company A, E ?”
we expect to break down into the following subtasks by LLM’s reasoning :
- Get invoice amount for company C and F. (GetInvoice function)
- Calculate total amount of above C and F. (Total function)
- Get invoice amount for company A and E. (GetInvoice function)
- Calculate total amount of above A and E. (Total function)
- Calculate the difference between previous total C, F and total A, E. (Diff function)
While built-in BasicPlanner can generate a plan for sequential execution, our custom Planner will create the structured plans.
For instance, when we ask for “the difference between company A and company B“, first it should invoke GetInvoice("A") and GetInvoice("B"), and next these results are fed into the function Diff(value1, value2).
In our custom Planner, each inputs and outputs can be nested in a tree structure with any depth.
Now let’s implement a custom Planner.
First, I’ll configure to use Azure OpenAI chat completion endpoint as follows. (Please prepare .env file in advance.)
import semantic_kernel as sk
import semantic_kernel.connectors.ai.open_ai as sk_oai
kernel = sk.Kernel()
deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()
service_id = "default"
kernel.add_service(
sk_oai.AzureChatCompletion(
service_id=service_id,
deployment_name=deployment,
endpoint=endpoint,
api_key=api_key
),
)Next I prepare a custom plugin as follows.
As you can see below, our custom plugin consists of 3 native functions – get_invoice(), diff(), and total().
from semantic_kernel.functions.kernel_function_decorator import kernel_function
class DemoPlugin:
def __init__(self):
super().__init__()
self.company_dic = {
"A": 2000,
"B": 1500,
"C": 20000,
"D": 6700,
"E": 1000,
"F": 4100,
}
@kernel_function(
description="Get invoice amount of trading company",
name="GetInvoice",
)
def get_invoice(self, company_name: str) -> int:
return self.company_dic[company_name]
@kernel_function(
description="Get diffrence",
name="Diff"
)
def diff(self, value1: int, value2: int) -> int:
return abs(value1 - value2)
@kernel_function(
description="Get total",
name="Total"
)
def total(self, values: list) -> int:
return sum(values)Note : You can use built-in
semantic_kernel.core_plugins.MathPluginfor primitive arithmetic operations.
For the purpose of your learning, here I have built a custom plugin from scratch.
Now let’s implement a custom Planner as follows.
With LLM reasoning, the following create_plan() method generates a structured JSON plan.
In execute_plan() method, a generated tree structure (plan) is then parsed and executed.
import json
from typing import Dict, Any
from semantic_kernel.kernel import Kernel
from semantic_kernel.functions.kernel_arguments import KernelArguments
from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings
from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig
PROMPT = """
[GOAL]
How much is the difference between the invoice of company A and company B ?
[OUTPUT]
{
"input": "How much is the difference between the invoice of company A and company B ?",
"subtasks":
{
"function": "DemoPlugin.Diff",
"args":
{
"value1":
{
"function": "DemoPlugin.GetInvoice",
"args": {"company_name": "A"}
},
"value2":
{
"function": "DemoPlugin.GetInvoice",
"args": {"company_name": "B"}
}
}
}
}
[GOAL]
How much is the total invoice amount of company B and D ?
[OUTPUT]
{
"input": "How much is the total invoice amount of company B and D ?",
"subtasks":
{
"function": "DemoPlugin.Total",
"args":
{
"values":
[
{
"function": "DemoPlugin.GetInvoice",
"args": {"company_name": "B"}
},
{
"function": "DemoPlugin.GetInvoice",
"args": {"company_name": "D"}
}
]
}
}
}
[GOAL]
How much is the difference between company C and the total invoice amount of company A, D ?
[OUTPUT]
{
"input": "How much is the difference between company C and the total invoice amount of company A, D ?",
"subtasks":
{
"function": "DemoPlugin.Diff",
"args":
{
"value1":
{
"function": "DemoPlugin.GetInvoice",
"args": {"company_name": "C"}
},
"value2":
{
"function": "DemoPlugin.Total",
"args":
{
"values":
[
{
"function": "DemoPlugin.GetInvoice",
"args": {"company_name": "A"}
},
{
"function": "DemoPlugin.GetInvoice",
"args": {"company_name": "D"}
}
]
}
}
}
}
}
[GOAL]
{{$goal}}
[OUTPUT]
"""
class DemoPlanner:
def __init__(self, service_id: str) -> None:
self.service_id = service_id
async def create_plan(
self,
goal: str,
kernel: Kernel,
prompt: str = PROMPT,
) -> Dict:
# Create the kernel function for running prompt
exec_settings = PromptExecutionSettings(
service_id=self.service_id,
max_tokens=1024,
temperature=0.0,
)
prompt_template_config = PromptTemplateConfig(
template=prompt,
execution_settings=exec_settings,
)
function = kernel.create_function_from_prompt(
plugin_name="PlannerPlugin",
function_name="CreatePlan",
prompt_template_config=prompt_template_config,
)
# Invoke and create plan
generated_plan = await function.invoke(
kernel, KernelArguments(goal=goal)
)
return json.loads(str(generated_plan))
async def execute_plan(self, plan: Dict, kernel: Kernel) -> str:
task = plan["subtasks"]
result = await self.parse_task(task, kernel)
return result
async def parse_task(self, task, kernel) -> Any:
if isinstance(task, dict):
if "function" in task.keys():
#
# When it's function
#
plugin_name, function_name = task["function"].split(".")
kernel_function = kernel.func(plugin_name, function_name)
args = task.get("args", None)
args = await self.parse_task(args, kernel)
func_res = await kernel_function.invoke(kernel, args)
task = func_res.value
else:
#
# When it's dictionary
#
for key in task.keys():
task[key] = await self.parse_task(task[key], kernel)
elif isinstance(task, list):
#
# When it's list
#
for i, item in enumerate(task):
task[i] = await self.parse_task(item, kernel)
# Otherwise (not iterable), do nothing ...
return taskNote : For the simplicity, I have used few-shot exemplars which includes above 3 functions. But please use zero-shot approach in prompt, in case that available functions are changed in the future.
See here for details.
Now let’s run our custom Planner.
First, I create a structured plan.
As you can see the output, I can get the correct reasoning structure in JSON.
kernel.import_plugin_from_object(DemoPlugin(), "DemoPlugin")
planner = DemoPlanner(service_id)
ask = "How much is the difference between the total of company C, F and the total of company A, E ?"
plan = await planner.create_plan(ask, kernel)
print(plan)output
{
'input': 'How much is the difference between the total of company C, F and the total of company A, E ?',
'subtasks': {
'function': 'DemoPlugin.Diff',
'args': {
'value1': {
'function': 'DemoPlugin.Total',
'args': {
'values': [
{
'function': 'DemoPlugin.GetInvoice',
'args': {
'company_name': 'C'
}
},
{
'function': 'DemoPlugin.GetInvoice',
'args': {
'company_name': 'F'
}
}
]
}
},
'value2': {
'function': 'DemoPlugin.Total',
'args': {
'values': [
{
'function': 'DemoPlugin.GetInvoice',
'args': {
'company_name': 'A'
}
},
{
'function': 'DemoPlugin.GetInvoice',
'args': {
'company_name': 'E'
}
}
]
}
}
}
}
}When you execute this plan, you can get the right answer by running above custom plugin (DemoPlugin).
result = await planner.execute_plan(plan, kernel)
print(result)output
21100Here I have implemented a simple custom Planner, but please see the source code for built-in StepwisePlanner (see here) for implementing ReAct prompting.
Note (Added on June 2023) : You can now use a new function-enabled model (
gpt-4-0613andgpt-3.5-turbo-0613) for more reliable reasoning by function calling.
See here for details.
Apr 2024 : Updated source code to the latest version
Categories: Uncategorized
tsmatz 
Very insightful intro to ReAct. Glad I could find it. It was hard to google it due to lots of search results referring for React.js 🙂
LikeLike