Anatomy of a Tool-Using LLM

2026-05-07
Section 1 · TaxonomyThe simple case

The Tool-Using LLM: a closed loop with a clear edge.

One prompt in, one response out. The model picks a tool, calls it, and answers. This is the architecture most enterprises already operate — sometimes without realizing it.

The loop

Human

“Look up the order status for #4421.”

LLM

Picks tool: getOrder(id)

Tool / API

{ status: "shipped", eta: "2026-05-09" }

LLM → Human

“Order #4421 shipped. Arrives Saturday.”

What the model decides

1

Whether to use a tool

Or answer from its own context.

2

Which tool to invoke

From a registered list with schemas.

3

What arguments to pass

Extracted or inferred from the prompt.

4

How to phrase the answer

Given the tool's structured response.

Operating properties

Stateless per call
One shot, one return
Easy to cache
Trivially auditable
No persistent goals
No retry logic
No self-correction

Architect's takeaway: this is a function call wrapped in a model. Treat it like a function call. The same auth, rate-limiting, and observability you apply to any API client applies here. The hard problems start at the next slide.