Start with the loop
The useful unit of analysis is the loop, not the model. A basic agent receives a goal and current state, proposes an action, executes that action through a constrained interface, records the outcome, and repeats until a stopping condition fires. Every production question maps onto one of those boundaries: What state can the model see? Which actions are legal? How is a tool result validated? Who decides that the work is done?
This framing prevents a common mistake: attributing system behavior entirely to model intelligence. A strong model inside a weak loop can repeat failures, lose context, or call the wrong tool. A modest model inside a carefully bounded workflow can be dependable on a narrow task because the surrounding software supplies validation, retries, and explicit state.
Mental model: treat the language model as a probabilistic policy inside a deterministic shell. The shell owns permissions, durable state, budgets, validation, and termination.
Tools turn text into consequences
A tool definition is part API contract and part safety boundary. Good tools expose a narrow action with typed inputs, clear errors, and outputs the loop can verify. Broad “run anything” tools may make a demo look capable, but they move nearly every failure mode into prompt interpretation.
Read-only and mutating tools should be separated. A search or inspection step usually has a different risk profile from sending a message, spending money, deleting data, or changing access. Reliable systems resolve exact targets before mutation, request human approval at meaningful boundaries, and make idempotency explicit so a retry cannot create duplicate side effects.
Memory is several different problems
“Give the agent memory” is underspecified. Working memory keeps the current task coherent. Episodic memory recalls previous runs. Semantic memory retrieves durable facts. Procedural memory stores tested ways of doing work. These stores have different write rules and failure modes. Saving every transcript creates noise; letting the model rewrite durable facts creates drift.
A defensible memory design answers three questions: who may write, how a record is validated, and when it expires. Retrieval quality also matters. A memory that cannot surface the right item at decision time is merely storage.
Planning is useful only when it changes control
A visible plan can help decompose long work, but a list of plausible steps is not evidence of execution quality. Ask whether the plan controls tool access, budgets, checkpoints, and recovery. Useful planners revise based on observed failures. Decorative planners produce a polished outline and then ignore it.
Evaluate trajectories, not just answers
Final-task success is necessary but incomplete. Two agents can reach the same answer with very different costs and risks. Evaluation should record tool calls, invalid actions, retries, latency, tokens, human interventions, permission violations, and whether the agent stopped for the right reason.
Benchmark contamination and overly forgiving simulators can make results brittle. Prefer held-out tasks, real tool responses, adversarial inputs, and repeated runs. Report distributions rather than a single best run. For costly or safety-relevant actions, measure false approvals and false refusals separately.
Questions to ask when reading a paper
- What is actually new? A model, a controller, a memory policy, a tool interface, training data, or only a prompt?
- What is the baseline? Does the method beat a simpler fixed workflow with the same model and tool budget?
- Where does state live? Is it reproducible and inspectable, or hidden inside a conversation?
- What can go wrong? Are partial failure, duplicate actions, prompt injection, and stale context tested?
- What does success cost? Include model calls, latency, retries, human review, and infrastructure.
A minimal implementation test
Choose a narrow task with an objective checker: for example, update a structured record from a known source. Give the agent one read tool and one proposed-write tool. Keep the final write behind approval. Log every observation, action, result, and state transition. Compare against a non-agentic script. Only add planning or memory after the logs show a specific failure they can address.
That experiment teaches more than an unconstrained chatbot demo. It reveals whether the loop adds value, which failures are recoverable, and where deterministic software should replace model judgment.
Bottom line
Agentic AI is systems engineering around uncertain decisions. The strongest work makes uncertainty visible, limits consequences, evaluates full trajectories, and proves that autonomy beats a simpler workflow under equal conditions.