Function calling is a capability of large language models that lets them request the execution of external code or tools by outputting structured arguments instead of generating free-form text. It bridges natural language prompts and deterministic software so an LLM can reliably act on databases, APIs, or other services. The model decides when a function is needed and supplies a machine-readable payload, while the calling application is responsible for actually running the function and feeding the result back to the model.
How Function Calling works
Developers register a set of function definitions with the model, typically as JSON schemas that describe each function's name, purpose, and parameters. When a user sends a prompt such as "What's the weather in Tokyo?", the model does not answer directly. Instead, it returns a structured object — for example, {"name": "get_weather", "arguments": {"city": "Tokyo"}}. The host application parses that object, executes the real function (often calling a third-party API), and appends the result to the conversation as a new message. The LLM then turns that result into a natural-language reply for the user.
Most providers expose the same flow through their chat completion APIs. The model never runs code itself; function calling is essentially a disciplined way to get a language model to "speak JSON" so deterministic software can act on it safely.
Why it matters
LLMs are good at language and reasoning but bad at arithmetic, real-time data, and side effects. Function calling turns a chat model into the reasoning layer of an agentic system: the model plans, the tools execute. This pattern powers AI assistants that can book calendars, query CRMs, run SQL, call internal microservices, or trigger robotic actions, all while the developer keeps tight control over what code actually runs. It also dramatically reduces hallucination for factual queries, because answers are grounded in real tool output rather than the model's parametric memory.
Key types of function-calling patterns
- Single-shot tool use: the model calls one function per turn, such as a calculator or web search.
- Parallel calls: the model requests several independent functions in one response, useful for fetching data from multiple sources at once.
- Multi-step / agentic loops: the application loops back tool results until the model decides no more calls are needed, enabling chains of actions.
- Constrained decoding: techniques that force the model's token-by-token output to conform to a JSON schema, improving reliability for strict APIs.
Function calling is now a standard capability across major model providers, and it is the foundation that most modern AI agents, copilots, and retrieval pipelines are built on.