Graph based control flow — for the most complex cases, a graph-based state machine can be used to control the execution of multiple agents
Of course, you can combine multiple strategies in a single application.
Agent delegation
"Agent delegation" refers to the scenario where an agent delegates work to another agent, then takes back control when the delegate agent (the agent called from within a tool) finishes.
Since agents are stateless and designed to be global, you do not need to include the agent itself in agent dependencies.
You'll generally want to pass ctx.usage to the usage keyword argument of the delegate agent run so usage within that run counts towards the total usage of the parent agent run.
Multiple models
Agent delegation doesn't need to use the same model for each agent. If you choose to use different models within a run, calculating the monetary cost from the final result.usage() of the run will not be possible, but you can still use UsageLimits to avoid unexpected costs.
agent_delegation_simple.py
frompydantic_aiimportAgent,RunContextfrompydantic_ai.usageimportUsageLimitsjoke_selection_agent=Agent('openai:gpt-4o',system_prompt=('Use the `joke_factory` to generate some jokes, then choose the best. ''You must return just a single joke.'),)joke_generation_agent=Agent('google-gla:gemini-1.5-flash',result_type=list[str])@joke_selection_agent.toolasyncdefjoke_factory(ctx:RunContext[None],count:int)->list[str]:r=awaitjoke_generation_agent.run(f'Please generate {count} jokes.',usage=ctx.usage,)returnr.dataresult=joke_selection_agent.run_sync('Tell me a joke.',usage_limits=UsageLimits(request_limit=5,total_tokens_limit=300),)print(result.data)#> Did you hear about the toothpaste scandal? They called it Colgate.print(result.usage())"""Usage( requests=3, request_tokens=204, response_tokens=24, total_tokens=228, details=None)"""
(This example is complete, it can be run "as is")
The control flow for this example is pretty simple and can be summarised as follows:
Agent delegation and dependencies
Generally the delegate agent needs to either have the same dependencies as the calling agent, or dependencies which are a subset of the calling agent's dependencies.
Initializing dependencies
We say "generally" above since there's nothing to stop you initializing dependencies within a tool call and therefore using interdependencies in a delegate agent that are not available on the parent, this should often be avoided since it can be significantly slower than reusing connections etc. from the parent agent.
agent_delegation_deps.py
fromdataclassesimportdataclassimporthttpxfrompydantic_aiimportAgent,RunContext@dataclassclassClientAndKey:http_client:httpx.AsyncClientapi_key:strjoke_selection_agent=Agent('openai:gpt-4o',deps_type=ClientAndKey,system_prompt=('Use the `joke_factory` tool to generate some jokes on the given subject, ''then choose the best. You must return just a single joke.'),)joke_generation_agent=Agent('gemini-1.5-flash',deps_type=ClientAndKey,result_type=list[str],system_prompt=('Use the "get_jokes" tool to get some jokes on the given subject, ''then extract each joke into a list.'),)@joke_selection_agent.toolasyncdefjoke_factory(ctx:RunContext[ClientAndKey],count:int)->list[str]:r=awaitjoke_generation_agent.run(f'Please generate {count} jokes.',deps=ctx.deps,usage=ctx.usage,)returnr.data@joke_generation_agent.toolasyncdefget_jokes(ctx:RunContext[ClientAndKey],count:int)->str:response=awaitctx.deps.http_client.get('https://example.com',params={'count':count},headers={'Authorization':f'Bearer {ctx.deps.api_key}'},)response.raise_for_status()returnresponse.textasyncdefmain():asyncwithhttpx.AsyncClient()asclient:deps=ClientAndKey(client,'foobar')result=awaitjoke_selection_agent.run('Tell me a joke.',deps=deps)print(result.data)#> Did you hear about the toothpaste scandal? They called it Colgate.print(result.usage())""" Usage( requests=4, request_tokens=309, response_tokens=32, total_tokens=341, details=None, ) """
(This example is complete, it can be run "as is" — you'll need to add asyncio.run(main()) to run main)
This example shows how even a fairly simple agent delegation can lead to a complex control flow:
Programmatic agent hand-off
"Programmatic agent hand-off" refers to the scenario where multiple agents are called in succession, with application code and/or a human in the loop responsible for deciding which agent to call next.
Here agents don't need to use the same deps.
Here we show two agents used in succession, the first to find a flight and the second to extract the user's seat preference.
programmatic_handoff.py
fromtypingimportLiteral,UnionfrompydanticimportBaseModel,Fieldfromrich.promptimportPromptfrompydantic_aiimportAgent,RunContextfrompydantic_ai.messagesimportModelMessagefrompydantic_ai.usageimportUsage,UsageLimitsclassFlightDetails(BaseModel):flight_number:strclassFailed(BaseModel):"""Unable to find a satisfactory choice."""flight_search_agent=Agent[None,Union[FlightDetails,Failed]]('openai:gpt-4o',result_type=Union[FlightDetails,Failed],# type: ignoresystem_prompt=('Use the "flight_search" tool to find a flight ''from the given origin to the given destination.'),)@flight_search_agent.toolasyncdefflight_search(ctx:RunContext[None],origin:str,destination:str)->Union[FlightDetails,None]:# in reality, this would call a flight search API or# use a browser to scrape a flight search websitereturnFlightDetails(flight_number='AK456')usage_limits=UsageLimits(request_limit=15)asyncdeffind_flight(usage:Usage)->Union[FlightDetails,None]:message_history:Union[list[ModelMessage],None]=Nonefor_inrange(3):prompt=Prompt.ask('Where would you like to fly from and to?',)result=awaitflight_search_agent.run(prompt,message_history=message_history,usage=usage,usage_limits=usage_limits,)ifisinstance(result.data,FlightDetails):returnresult.dataelse:message_history=result.all_messages(result_tool_return_content='Please try again.')classSeatPreference(BaseModel):row:int=Field(ge=1,le=30)seat:Literal['A','B','C','D','E','F']# This agent is responsible for extracting the user's seat selectionseat_preference_agent=Agent[None,Union[SeatPreference,Failed]]('openai:gpt-4o',result_type=Union[SeatPreference,Failed],# type: ignoresystem_prompt=("Extract the user's seat preference. "'Seats A and F are window seats. ''Row 1 is the front row and has extra leg room. ''Rows 14, and 20 also have extra leg room. '),)asyncdeffind_seat(usage:Usage)->SeatPreference:message_history:Union[list[ModelMessage],None]=NonewhileTrue:answer=Prompt.ask('What seat would you like?')result=awaitseat_preference_agent.run(answer,message_history=message_history,usage=usage,usage_limits=usage_limits,)ifisinstance(result.data,SeatPreference):returnresult.dataelse:print('Could not understand seat preference. Please try again.')message_history=result.all_messages()asyncdefmain():usage:Usage=Usage()opt_flight_details=awaitfind_flight(usage)ifopt_flight_detailsisnotNone:print(f'Flight found: {opt_flight_details.flight_number}')#> Flight found: AK456seat_preference=awaitfind_seat(usage)print(f'Seat preference: {seat_preference}')#> Seat preference: row=1 seat='A'
(This example is complete, it can be run "as is" — you'll need to add asyncio.run(main()) to run main)
The control flow for this example can be summarised as follows:
Pydantic Graphs
See the graph documentation on when and how to use graphs.
Examples
The following examples demonstrate how to use dependencies in PydanticAI: