Agent Architectures: ReAct, Plan-and-Execute and Reflection, Compared

From Function Calling to the Agent Loop built the basic agent: a model, some tools, and a loop that runs the model’s tool calls until it answers. Within that loop there are choices about when the model thinks and what it thinks about. This post compares the three best-known answers, each from an influential paper, and runs them against each other on the same tasks.

Table of contents

ReAct: think and act, interleaved

ReAct (Yao et al., 2022) has the model alternate between a written thought (“I need X’s founding year first”), an action (a tool call), and an observation (the tool’s result), repeating until it can answer. Reasoning steers the actions, and each observation updates the reasoning. Today’s tool-calling models can do this directly: a reply can contain text and tool calls, and the loop of From Function Calling to the Agent Loop is a ReAct loop.

Its typical failure is myopia: deciding each step from the current state, it can wander, repeat a lookup it has already done, or lose track of a goal several steps back. The ReAct paper’s own error analysis found runs stuck repeating the same thoughts and actions. Long tasks expose it most.

Plan-and-execute: decide the steps first

Plan-and-execute splits the work: one call writes a plan, a numbered list of steps, before any tool runs; then each step is carried out, often by a cheaper model or a short loop of its own; then a final call writes the answer from the step results. The idea has two roots. Plan-and-Solve prompting (Wang et al., 2023) is a single prompt, no tools: devise a plan, then carry it out, which cut the steps that zero-shot chain of thought skips. ReWOO (Xu et al., 2023) moved the plan into tool use: a planner writes every tool call up front, with placeholders for results it hasn’t seen, workers run them, and a solver answers; on HotpotQA it used five times fewer tokens than interleaving, and its planner could be distilled into a 7B model. LLMCompiler (Kim et al., 2023) runs independent planned calls in parallel, reporting up to 3.7 times lower latency than ReAct.

Its advantages: the plan is visible and can be checked (by code or a person) before anything runs; steps can run in parallel or on smaller models; the overall shape doesn’t drift. Its failure is rigidity: a plan written before seeing any data can be wrong, and if a step’s result changes what should happen next, a strict executor won’t notice. Real systems add replanning: after each step, or when a step fails, the planner revises the remaining plan.

Reflection: answer, critique, retry

Reflection adds a second look. After an attempt, the model (or another model) critiques it (was every value looked up? is the arithmetic right?) and, if the critique finds a problem, tries again with the critique in hand. Reflexion (Shinn et al., 2023) keeps such verbal self-critiques as memory across attempts; Self-Refine (Madaan et al., 2023) iterates feedback and revision on a single output.

Reflexion’s gains came with an external signal: the environment’s reward, unit tests for code, an exact-match check for question answering. Without one, Huang et al. (2023) found that models asked to review their own reasoning rarely fixed it, and sometimes made it worse. That’s the general failure: a model’s critique of its own work is only as good as its ability to check it. A critic that can’t detect the error approves the wrong answer; one that is too eager rejects right ones and the retry makes them worse. Reflection works best when the critique can lean on something external: a test, a tool, a schema.

The three, measured

The same tasks for all three: 12 fictional companies with founding years, headcounts and cities (fictional, so nothing can be answered from memory), and 20 questions that each need two or three lookups and some arithmetic. Two tools, lookup and calculator; Qwen2.5-3B-Instruct; three runs per question. The 180 trials sample at temperature 0.7 on the GPU; on an Apple M4 laptop they take about 87 minutes. arch.py builds the companies, the tools and the questions, runs every trial and logs each one to arch_trials_v3.jsonl. The three architectures, as it implements them, on top of tool_loop (model calls, executing tool calls, until a reply without one):

def react(q):
    return tool_loop([{"role": "system", "content": SYS}, {"role": "user", "content": q}])

def plan_execute(q):
    plan = gen([{"role": "user", "content": "Write a short numbered plan (one tool action per line, no answers) "
                 f"for answering this with lookup and calculator tools:\n{q}"}], tools=None, max_new=200)
    steps = [l for l in plan.splitlines() if re.match(r"\s*(\d+[.):]|[-*]|step\s*\d+)", l, re.I)][:6]
    steps = steps or [l for l in plan.splitlines() if l.strip()][:6]
    LOG["plan"] = steps
    notes = []                                # each step's tool calls and results, verbatim
    for s in steps:
        msgs = [{"role": "system", "content": "Do exactly this one step with the tools."},
                {"role": "user", "content": f"Question: {q}\nResults so far: {notes}\nStep: {s}"}]
        tool_loop(msgs, max_calls=3)
        notes += tool_loop.calls
    LOG["notes"] = notes
    return tool_loop([{"role": "system", "content": SYS}, {"role": "user", "content":
                      f"{q}\nTool results from the plan's steps: {notes}"}])

def reflect(q):
    msgs = [{"role": "system", "content": SYS}, {"role": "user", "content": q}]
    first = tool_loop(msgs)
    transcript = "\n".join(f"{m['role']}: {m['content']}" for m in msgs[1:])[-3000:]
    critique = gen([{"role": "user", "content": "Check this attempt. Were all values looked up (not guessed) and is the "
                     f"arithmetic right? Reply OK, or describe the error.\n\n{transcript}"}], tools=None, max_new=150)
    LOG["critique"] = critique
    if re.match(r"\W*OK\b", critique.strip().upper()):
        return first
    return tool_loop([{"role": "system", "content": SYS}, {"role": "user", "content":
                       f"{q}\nA previous attempt was checked and this problem was found: {critique}"}])

Full script: arch.py

20 questions x 3 runs; Qwen2.5-3B-Instruct

architecture        right  tokens processed/question  seconds/question
ReAct                 35%                      1,176              12.1
plan-and-execute      48%                      6,058              51.7
reflection            37%                      2,420              29.6

A few design choices are worth stating, since each one matters below. ReAct gets up to 8 model calls per question; each plan step gets 3. The plan-and-execute executor passes on every tool call it made, with its result, verbatim; the final step can use the calculator. Reflection’s retry is a fresh attempt that sees the critique but not the first transcript, and its answer replaces the first one. Only a reply containing Answer: <number> counts as an answer.

Plan-and-execute came out ahead: 48% against 35% for ReAct and 37% for reflection. The intervals overlap, but the paired counts (the same question and run under two architectures) point the same way: of the trials where ReAct and plan-and-execute disagreed, plan-and-execute was right in 12 and ReAct in 4. That’s suggestive rather than settled (a two-sided sign test gives p ≈ 0.08), and it cost five times ReAct’s tokens and four times its time. The rerun on an NVIDIA RTX 3090 put plan-and-execute ahead again, 58% against 42% for ReAct and 35% for reflection, right in 14 of its disagreements with ReAct, and ReAct in 4 (the same test on 4 of 18 gives p ≈ 0.03). Reflection and ReAct were level within noise: they differed on only 11 trials on the M4, 6 one way and 5 the other, and on 8 in the rerun, 6 for ReAct and 2 for reflection (p ≈ 0.29).

Accuracy doesn’t say why, so every wrong answer was sorted by what it got wrong. failures.py rebuilds the same questions and reads the trials arch.py logged:

for a in archs:
    c = collections.Counter()
    ts = [t for t in trials if t["arch"] == a]
    for t in ts:
        p, y, i = t["pred"], t["truth"], t["i"]
        if p == y: c["right"] += 1
        elif p is None: c["none"] += 1
        elif p == -y: c["sign"] += 1
        elif parts[i] and p in {x + z for x, z in itertools.combinations(parts[i], 2)}: c["missing"] += 1
        else: c["other"] += 1
    print(f"{a:18s} {wilson(c['right'], len(ts)):>18s} {c['sign']:>13d} {c['missing']:>18d} {c['none']:>10d} {c['other']:>6d}")

Full script: failures.py

                       right [95% CI]  sign flipped  total missing one  no answer  other
ReAct                  35% [24%, 48%]            17                  9          1     12
reflection             37% [26%, 49%]            19                  4          8      7
plan-and-execute       48% [36%, 61%]            22                  2          0      7

right, by question type:
  ReAct              years older 1/21, more employees 16/21, total of three 4/18
  reflection         years older 2/21, more employees 13/21, total of three 7/18
  plan-and-execute   years older 0/21, more employees 18/21, total of three 11/18

paired (same question and run): only the first right / only the second right
  ReAct vs reflection: 5 / 6
  ReAct vs plan-and-execute: 4 / 12
  reflection vs plan-and-execute: 5 / 12

reflection: the critic flagged 44 of 60 first attempts (16 of the 21 right ones, 28 of the 39 wrong ones)
  after the retry: wrong->right 6, right->wrong 5, wrong->wrong 22, right->right 11
  first attempts with the sign flipped: 17; flagged 11; right after the retry 1

years older, wrong: 60; the right number with the wrong sign 51
ReAct, wrong totals: 14; the reply says a lookup failed 8
reflection: critic replies other than OK 44; first sentence calls it correct 42
reflection: unanswered 8; after a retry 7

One question type defeated all three. “How many more employees does A have than B?” is A minus B, in the order the question names them, and plan-and-execute got it right 18 times in 21. “How many years older is A than B?” is B’s founding year minus A’s, the reverse of the order named, and no architecture got it right more than 2 times in 21; 51 of their 60 misses were the right number with the wrong sign. Nothing in any of the three loops checks what a subtraction means.

ReAct lost companies to a malformed call. On the three-company totals ReAct was right 4 times in 18 (8 in the rerun), and most of its misses were the sum of two companies. Full transcripts of three of them showed the same thing each time: the model emitted all three lookups at once, the third came out malformed ({"name": "lookup", {"arguments": ...}}}), the tool replied with the exact format to use, and the model ignored it and answered with the two values it had. Eight of its 14 wrong totals say so in the reply (“It seems that there was an issue with fetching the employee count for Istria. Let’s proceed with the available information.”). A tool call missing its tags or its "arguments" key could have been repaired by the parser instead of returned as an error; this harness doesn’t, so the model’s reaction to the error is what’s measured. Plan-and-execute, whose plan lists all three lookups as separate steps, got 11 of 18 (14 in the rerun). Its remaining misses on totals mostly came from the same verbatim hand-off that helped it: step executors sometimes did more than their step, leaving stray sums like 7550 + 190 = 7740 in the notes, and the final step trusted them.

The critic couldn’t tell right from wrong. It was asked to reply OK or describe the error, and it rarely replied OK: 42 of its 44 other replies began by calling the attempt correct (“Your approach and calculations are correct, but…”) and then raised a quibble, often about formatting. The harness counts anything but OK as a problem found, so those triggered a retry. In all, it flagged 16 of the 21 right first attempts (76%) and 28 of the 39 wrong ones (72%), and its retries came out even: 6 fixed, 5 broken. The rerun’s critic did no better: it flagged 19 of 25 right first attempts (76%) and 21 of 35 wrong ones (60%), and its retries fixed 2 and broke 6. It flagged 11 of the 17 sign errors (13 of 21 in the rerun), but the retry re-derived “older” the same way and fixed only one in either run. On the M4, seven of reflection’s 8 unanswered trials were retries that ended without an answer: each emitted a tool call without its tags, which the loop takes as a final reply. The rerun had only 2 unanswered trials.

What the harness got wrong first

This comparison took three runs, and the first two were measuring the harness more than the architectures. The scripts and outputs of those two were not kept, so the numbers in this section are the ones they printed at the time, from notes; they can’t be rerun from the companion folder. Read them as an account of how the harness was fixed, not as a measurement.

The tool’s schema. In the first version, the lookup tool described its fields in prose (“Fields: founded (year), employees (count), city”) with no list of allowed values, and every bad call got the same reply, error: unknown company or field. The model copied “employees (count)” as the field name, couldn’t tell what was wrong, repeated the call and gave up:

assistant | <tool_call> {"name": "lookup", "arguments": {"company": "Dunmore", "field": "employees (count)"}} </tool_call> ...
tool      | error: unknown company or field
assistant | It seems there's an issue with accessing the company details. Let's try another way ...
tool      | error: unknown company or field
assistant | ... Could you please confirm the correct names or identifiers for these companies? Answer: -1

ReAct scored 2%, plan-and-execute 5%, reflection 8%. An enum of the three field names and errors that say what would have worked took ReAct and reflection into the 30s and plan-and-execute to 48%. The enum sits in arch.py’s TOOLS list; the errors are in tool:

def tool(name, args):
    if name == "lookup":                      # errors say exactly what would have worked
        company, field = str(args.get("company", "")).strip(), args.get("field")
        if company not in db:
            return f"error: no company named {company!r}; companies are: {', '.join(names)}"
        if field not in db[company]:
            return f"error: no field {field!r}; fields are: founded, employees, city"
        return str(db[company][field])
    if name == "calculator":
        e = str(args.get("expression", ""))
        return str(round(eval(e), 4)) if re.fullmatch(r"[\d\s+\-*/().]+", e) else "error: arithmetic only"
    return "error: unknown tool"

Full script: arch.py

The hand-off between steps. In the second version, each plan step reported its result as a sentence, and the final call read only those sentences: plan-and-execute scored 30%, with numbers distorted in the retelling (“700 more” for a difference of 696). Passing on the bare tool results instead made it worse, 22%: a list like ['8033', '7337', '696'] doesn’t say which number is which company. Passing each call with its result, lookup({"company": "Dunmore", "field": "employees"}) = 8033, gave the 48% above. For plan-and-execute, what the executor hands the final step is most of the design.

Choosing

• ReAct is the default: simplest, adaptive, and what tool-calling models are trained for. Add a step budget and loop detection.
• Plan-and-execute when the task’s structure is known in advance, when a plan must be approved before acting, or when steps can run in parallel or on cheaper models; add replanning when results can change the plan.
• Reflection when there’s something real to check against: a test, a tool, a schema. Here, a self-critic with nothing external to check was a coin flip with extra cost, as Huang et al. found for self-correction of reasoning more broadly.
• For plan-and-execute, design the hand-off: pass each step’s call and result, labeled, and keep executors to their one step.

Where this leads

These combine: a planner whose steps are ReAct loops, a ReAct agent with a verification step before answering, a team of agents each running one of these. Later posts build those out:

• Agent Memory gives an agent memory beyond its transcript: what to write, what to read back, and what to forget.
• Many Agents runs several agents, each with its own window, and orchestrates them.
• Engineering Around the Model adds the harness this post found decisive, including a definition of done checked before the agent answers.

Takeaways

• ReAct interleaves reasoning and tool calls; it adapts but can wander.
• Plan-and-execute decides the steps first; it’s checkable and parallelizable but rigid without replanning.
• Reflection critiques and retries; it helps only as far as the critique can actually detect errors.
• Measured on 20 tool-use questions with a 3B model: plan-and-execute 48%, reflection 37%, ReAct 35% on the M4; 58%, 35% and 42% on a rerun. Plan-and-execute led in both runs, and ReAct and reflection were level within noise. Plan-and-execute’s lead came from its plan listing every lookup, and it depended on handing each step’s call and result to the final step verbatim.
• The harness can matter more than the architecture. In earlier versions of this experiment, which weren’t kept, an unclear tool schema and the hand-off format moved scores more than the choice of architecture did: treat that as a lesson from development, not a measurement.
• Sort errors before comparing architectures. Here they were a misread subtraction none of the three could catch, a malformed call ReAct gave up on, and a self-critic that flagged right answers at least as often as wrong ones.

References

Start with the ReAct paper, then ReWOO for planning up front, and Huang et al. for why a self-critique without an external check rarely helps; Reflexion shows what changes when there is one.

• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models (2022): the interleaved thought, action and observation loop, and its error analysis.
• Wang et al., Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models (2023): planning first in a single prompt, one root of plan-and-execute.
• Xu et al., ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models (2023): every tool call planned up front, and its token savings on HotpotQA.
• Kim et al., An LLM Compiler for Parallel Function Calling (2023): planned calls run in parallel, and the latency it reported against ReAct.
• Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning (2023): self-critiques kept as memory, with gains that came from an external signal.
• Madaan et al., Self-Refine: Iterative Refinement with Self-Feedback (2023): feedback and revision on a single output.
• Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet (2023): why self-correction without an external check rarely fixes reasoning.