How to properly handle tool_calls with parallel function execution?

Kevin ChoiAug 20, 2024

When using function calling with GPT-4o, the model sometimes returns multiple tool_calls in a single response for parallel execution. I'm struggling with the correct way to handle this.

My current code processes them sequentially:

for tool_call in response.choices[0].message.tool_calls:
    result = execute_function(tool_call.function.name, tool_call.function.arguments)
    messages.append({
        "role": "tool",
        "tool_call_id": tool_call.id,
        "content": json.dumps(result)
    })

Questions: 1. Should I execute them in parallel with asyncio? 2. Do I need to send ALL tool results back in a single API call, or can I send them one at a time? 3. Can I set parallel_tool_calls: false to avoid this entirely?

6.2k views31 replies78 likesSolved

3 Replies

Logan K.StaffAccepted AnswerAug 20

Great questions! Here's the breakdown:

1. Yes, you can execute them in parallel with asyncio - just make sure all results are sent back together 2. You MUST send all tool results in a single API call. Sending them one at a time will cause an error. 3. Yes, parallel_tool_calls: false is a valid parameter to prevent parallel calls.

# Correct way to send multiple tool results
messages.append(response.choices[0].message)
for tool_call in tool_calls:
    messages.append({
        "role": "tool",
        "tool_call_id": tool_call.id,
        "content": result
    })
Send all at once in next API call

Kevin Choi Aug 22

Thanks @logan_oai! Setting parallel_tool_calls to false was the simplest fix for my use case. Performance is fine since my functions are fast.

Alex Petrov Aug 22

For anyone implementing parallel execution, here's an asyncio example:

import asyncioasync def execute_parallel(tool_calls):
    tasks = [execute_function(tc) for tc in tool_calls]
    return await asyncio.gather(*tasks)

Works great and significantly reduces latency when you have 3+ tool calls.