18.2 Workflows and Agents: Tools, Loops, and Permission Boundaries

A fixed workflow is determined by code; an Agent lets the model dynamically choose the next step based on goals and environmental feedback. The latter is well-suited for tasks where paths can't be preenumerated, but it increases latency, cost, and unpredictability.

Start with the simplest structure that gets the job done: single model call, retrieval augmentation, fixed chains, or branched workflows, only resort to open loops when complexity is actually required by evaluation results.

Common Workflows

Prompt chaining

Chain tasks that are clearly decomposable and add programmatic checks in between:

Extract facts → schema validation → draft generation → citation check

Routing

First categorize, then hand off to dedicated prompts, tools, or models. The router must independently assess misrouting costs and provide fallbacks for low confidence.

Parallelization

Independent subtasks can be processed in parallel and then aggregated; multiple samples from the same task can be used for candidate comparison. Additional calls are only worthwhile if they genuinely improve the target metric.

Evaluator–optimizer

Generators are repeatedly refined according to clear criteria. Maximum iteration limits and stopping conditions must be set to avoid treating "the evaluator can still provide feedback" as a justification for endless improvement.

An Agent is a controlled loop

Runtime in concept:

Goal + Current Status
      ↓
Model selection: answer or tool call
      ↓
Strategy review / Manual approval
      ↓
Run the tool and document the observations
└──────────→ Next round

The ReAct paper demonstrates a paradigm of alternating reasoning and action. In engineering implementation, it's not necessary to expose or permanently store the model's private reasoning traces; instead, only auditable summaries of the plan, tool parameters, tool results, approval records, and final outputs should be preserved.

Loops must have an exit condition: success, explicit failure, user input required, insufficient permissions, reaching a step/time/cost budget, or no progress for consecutive iterations.

Tool definitions determine the agent's capability boundaries

Tools should have:

• Clear, mutually exclusive purposes and parameter schema;
• Structured return values and categorizable errors;
• Minimum permissions and caller identity propagation;
• Idempotent keys or operations that query the current state;
• Timeout, rate limiting, and audit logs;
• An approval mechanism for destructive or external side effects.

"The 'Execute SQL' or 'Call Any URL' permissions are too broad. A safer alternative is getOrder(id) and requestRefund(orderId, amount, reason), where application code handles authorization, limits, and invariants."

The model proposes that tool invocation does not equate to authorization. Permissions are re-evaluated on the server side based on user, tenant, resource, and action.

Prompt Injection is a control flow attack

Text in web pages, emails, and retrieved documents might require the Agent to disclose secrets or invoke tools. External content is data, not trustworthy system instructions.

Protection requires a layered approach:

• Clearly separate trusted instructions from untrusted content;
• Apply least privilege and parameter validation to tools;
• High-risk actions require manual confirmation and display actual parameters;
• Don't include unnecessary keys in the model context;
• Tiered authorization for capabilities such as reading, writing, sending, and paying;
• Continuously validate with adversarial testing, rather than relying on a single statement like "ignore malicious instructions alone."

State, memory, and context are not synonyms

• Run Status: Current step, tool results, approval, and budget;
• Session context: information provided to the model for this interaction;
• Long-term memory: information retained and retrievable across conversations;
• Business Truth: Official system records such as orders, permissions, and balances.

Long-term memory must have sources, scope, update mechanisms, deletion protocols, and privacy policies. Summaries generated by a model cannot automatically become business truth.

Use multiple agents only when responsibilities are clearly defined

Adding more Agents increases handoffs, context loss, loops, and evaluation combinations. Signals that warrant splitting are different tool permissions, domain-specific instructions, or distinct result ownership, not "the more roles, the smarter."

Whether using handoff or treating the Agent as a tool call, always clarify: who owns the final answer, who can execute side effects, and where control returns upon failure.

The next lesson establishes evaluation and runtime governance, enabling quantitative comparison of model, prompt, retriever, or tool changes.

References

• Anthropic, Building effective agents
• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models
• OpenAI, Agents SDK