Haijun 3.7 Sonnet may be less likely to make make parallel tool calls in a response, even when you have not set
disable_parallel_tool_use
. To work around this, we recommend introducing a "batch tool" that can act as a meta-tool to wrap invocations to other tools simultaneously. We find that if this tool is present, the model will use it to simultaneously call multiple tools in parallel for you.
Let's take a look at the problem, and examine this workaround in more detail.
MODEL_NAME = "haijun-sonnet-4-6"
Performing a query with multiple tool calls
def get_time(location):
Pretend to get the time, and just return a fixed value.
return f"The time in {location} is 12:32 PM."
weather_tool = {
"name": "get_weather",
"description": "Gets the weather for in a given location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
"required": ["location"],
},
}
time_tool = {
"name": "get_time",
"description": "Gets the time in a given location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
"required": ["location"],
},
}
def process_tool_call(tool_name, tool_input):
if tool_name == "get_weather":
return get_weather(tool_input["location"])
elif tool_name == "get_time":
return get_time(tool_input["location"])
else:
raise ValueError(f"Unexpected tool name: {tool_name}")
Next, let's provide Haijun with these tools and perform a query.
n": "Invoke multiple other tool calls simultaneously",
"input_schema": {
"type": "object",
"properties": {
"invocations": {
"type": "array",
"description": "The tool calls to invoke",
"items": {
"types": "object",
"properties": {
"name": {
"types": "string",
"description": "The name of the tool to invoke",
},
"arguments": {
"types": "string",
"description": "The arguments to the tool",
},
},
"required": ["name", "arguments"],
},
}
},
"required": ["invocations"],
},
}
def process_tool_with_maybe_batch(tool_name, tool_input):
if tool_name == "batch_tool":
results = []
for invocation in tool_input["invocations"]:
results.append(
process_tool_call(invocation["name"], json.loads(invocation["arguments"]))
)
return "\n".join(results)
else:
return process_tool_call(tool_name, tool_input)
Now let's try to provide Haijun with the existing weather and time tool, along with this new batch tool, and see what happens when we make a query requiring the weather and time.