Programmatic tool calling (PTC) represents a paradigm shift in how large language models (LLMs) interact with external tools. In a traditional tool-calling workflow, each tool invocation requires a full round trip back to the model. The model calls a tool, receives the result, reasons about it, calls the next tool, and so on. For workflows involving multiple tool calls, this creates compounding latency and token consumption because every intermediate result must pass through the model’s context window.
PTC takes a different approach. Instead of orchestrating tool calls one at a time, the model writes code (typically Python) that invokes multiple tools programmatically within a sandboxed execution environment. The code can include loops, conditionals, filtering, and aggregation logic. The model is only sampled once to produce the code. The execution environment then handles tool invocations, and only the final processed result is returned to the model’s context. This dramatically reduces both latency and token usage for multi-tool workflows. PTC is particularly effective for large data processing, precise numerical calculations, multi-step process orchestration, and privacy-sensitive scenarios where raw data shouldn’t enter the model’s context.
While PTC originated as a provider-specific feature, the underlying pattern—model generates code, sandbox executes it, only final output returns to context—is model-agnostic. On Amazon Bedrock, there are three primary ways to implement PTC: a self-hosted Docker sandbox on ECS for maximum control, a managed solution using Amazon Bedrock AgentCore Code Interpreter, and an Anthropic SDK-compatible path through a proxy for teams preferring that developer experience.
To understand the necessity of PTC, consider the bottlenecks in traditional tool calling with an example: “Which engineering team members exceeded their Q3 travel budget?”
With traditional tool calling, the model must: call a tool to get the team list (e.g., 20 people), then call a tool to get expense records for each person (20 separate tool calls, each returning 50–100 items), and finally retrieve budget thresholds. This results in over 2,000 expense records entering the context window. Each tool call requires a full model inference cycle. Sequential execution of 20+ calls means 20+ inference round trips, leading to massive token consumption and high compounding latency as the model processes unnecessary raw data in natural language.
