LangChain and LangGraph, from the Inside: What the Frameworks Compile Down To

From Function Calling to the Agent Loop wrote an agent as a loop in plain Python, and Engineering Around the Model argued that the harness around the model decides how well an agent works. Frameworks package that harness. LangChain and LangGraph are the most widely used, and the usual debate about them (“too much abstraction”, “saves time”) is easier to settle by looking at what they actually do. This post opens them up: what a chain is, what a graph is, and what the same agent costs and gains in each form. Versions matter with these libraries; everything here was run with LangGraph 1.2 and langchain-core 1.6.

Table of contents

Runnables: LangChain’s pipe

LangChain’s core abstraction is the Runnable: anything with invoke (one input), batch (many), stream (output as it’s produced), and async versions of each. Prompts, models, output parsers and plain functions are all Runnables, and the | operator composes them, the LangChain Expression Language (LCEL). A prompt, a stand-in model and a parser:

from langchain_core.messages import AIMessage
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableLambda

# LCEL: a prompt, a model and a parser, composed with |. The model is a
# stub that echoes, so the plumbing is all that's measured.
prompt = ChatPromptTemplate.from_messages([("system", "Answer briefly."), ("user", "{question}")])
model = RunnableLambda(lambda pv: AIMessage(content=f"(model saw {len(pv.to_messages())} messages, "
                                                    f"last: {pv.to_messages()[-1].content!r})"))
chain = prompt | model | StrOutputParser()
print(type(chain).__name__, "of", [type(s).__name__ for s in chain.steps])
print(chain.invoke({"question": "What is a KV cache?"}))
print(chain.batch([{"question": "a"}, {"question": "b"}]))

# What | built is ordinary function composition with a shared interface
# (invoke, batch, stream, async versions), equivalent to:
def by_hand(question):
    messages = prompt.invoke({"question": question})
    return StrOutputParser().invoke(model.invoke(messages))
print(by_hand("What is a KV cache?") == chain.invoke({"question": "What is a KV cache?"}))

Full script: lcel.py

RunnableSequence of ['ChatPromptTemplate', 'RunnableLambda', 'StrOutputParser']
(model saw 2 messages, last: 'What is a KV cache?')
["(model saw 2 messages, last: 'a')", "(model saw 2 messages, last: 'b')"]
True

The pipe builds a RunnableSequence, and running it gives exactly what calling the three steps by hand gives. That’s all a chain is: function composition, with the benefit that every composed chain also gets batch, stream and async for free, and that tracing tools can see each step. A prompt template fills variables into messages; an output parser turns the model’s message into something the next step can use (a string, JSON validated against a schema). Neither is magic; both are small and replaceable.

Chains are linear, or at most branch and merge. An agent isn’t: it loops until the model decides it’s done, and that needs state and control flow, which is what LangGraph adds.

LangGraph: a state machine around the model

LangGraph models an agent as a graph:

• State: a typed record that the whole run shares, such as the list of messages. Each field can have a reducer that says how updates combine; for messages, a new message is appended and one with an existing ID replaces it, which is how a run’s history can be edited.
• Nodes: functions that read the state and return an update to it (call the model; run the tools).
• Edges: which node runs next, either fixed or conditional, decided by a function of the state (if the model asked for a tool, go to the tools node; otherwise stop).
• A checkpointer: storage that saves the state after every step, keyed by a thread ID: in memory, or in SQLite or Postgres through separate packages (with community ones for other stores), and with a setting for how eagerly each step is written.

Execution runs in steps, borrowed from Google’s Pregel model for graph processing: in each step the scheduled nodes run, their updates are merged into the state through the reducers, and the edges pick the next nodes. After each step the checkpointer saves a snapshot. LangChain’s own agent constructor, create_agent (LangChain 1.0, October 2025), is built on this runtime.

The same agent, twice

The calculator agent that Evaluating and Observing Agents evaluates, written by hand, as a LangGraph graph built from scratch, and with LangGraph’s prebuilt tool node, with a stub in place of the model so that only the code around it is compared. The stub, stub_model, answers the question with one call to a calculator tool on 8113-423+7*755, then replies “The warehouse holds N boxes.” with the tool’s result as N:

# 1. By hand.
def run_by_hand(question, max_steps=5):
    messages = [HumanMessage(question)]
    for _ in range(max_steps):
        reply = stub_model(messages)
        messages.append(reply)
        if not reply.tool_calls:
            return messages
        for call in reply.tool_calls:
            messages.append(ToolMessage(calculator.invoke(call["args"]), tool_call_id=call["id"]))
    return messages

# 2. LangGraph, by hand: the state is a list of messages; nodes return
# updates to it; a conditional edge decides where to go after the model.
def model_node(state: MessagesState):
    return {"messages": [stub_model(state["messages"])]}

def tools_node(state: MessagesState):
    calls = state["messages"][-1].tool_calls
    return {"messages": [ToolMessage(calculator.invoke(c["args"]), tool_call_id=c["id"]) for c in calls]}

def route(state: MessagesState):
    return "tools" if state["messages"][-1].tool_calls else END

def build(checkpointer=None, tools=tools_node):
    g = StateGraph(MessagesState)
    g.add_node("model", model_node)
    g.add_node("tools", tools)
    g.add_edge(START, "model")
    g.add_conditional_edges("model", route, ["tools", END])
    g.add_edge("tools", "model")
    return g.compile(checkpointer=checkpointer)

Full script: agent_three_ways.py

The third version swaps tools_node and route for LangGraph’s prebuilt ToolNode and tools_condition. The script runs all three on the same question, counts the lines around the model and tool, and times a run of each with an instant model. It then gives the graph a tools node that asks a person for approval before it runs the tool:

# Human approval: the tools node asks for it with interrupt(), which pauses
# the run and saves its state; Command(resume=...) continues it with the
# person's reply, here an edited expression.
def approved_tools(state: MessagesState):
    call = state["messages"][-1].tool_calls[0]
    decision = interrupt({"tool": call["name"], "args": call["args"]})
    args = decision.get("args", call["args"])
    return {"messages": [ToolMessage(calculator.invoke(args), tool_call_id=call["id"])]}
g2 = build(InMemorySaver(), tools=approved_tools)
cfg = {"configurable": {"thread_id": "demo"}}
paused = g2.invoke({"messages": [HumanMessage(q)]}, cfg)
print(f"\npaused for approval: {paused['__interrupt__'][0].value}")
final = g2.invoke(Command(resume={"args": {"expression": "8113-423+7*755+100"}}), cfg)
print("resumed with an edited expression:", final["messages"][-1].content)

The last part of the script replays the run from the snapshot taken just before the tool.

Full script: agent_three_ways.py

same final answer: True -> The warehouse holds 12975 boxes.
lines of code around the model and tool: by hand 10, graph by hand 15, graph with prebuilt tool node 10
time per run with an instant model: by hand 76 us, graph 469 us, graph with a checkpointer 924 us

paused for approval: {'tool': 'calculator', 'args': {'expression': '8113-423+7*755'}}
resumed with an edited expression: The warehouse holds 13075 boxes.
checkpoints saved: [(3, ()), (2, ('model',)), (1, ('tools',)), (0, ('model',)), (-1, ('__start__',))]
replaying from step 1 -> paused again: True
approved as is this time: The warehouse holds 12975 boxes.

Code. The hand-written loop is 10 lines; the graph built from scratch is 15, spread over three node functions and a builder; with LangGraph’s prebuilt ToolNode and tools_condition it’s back to 10, and the prebuilt node also handles unknown tools and tool errors, which the hand loop doesn’t. (LangChain’s create_agent packages the whole thing into one call.) The graph version says more about structure (which steps exist, which can follow which) and less about order of execution, which the runtime decides.

Speed. With an instant model, the loop runs in about 76 microseconds, the graph in about 0.47 ms, and 0.92 ms with checkpointing, on an Apple M4 laptop. Six to twelve times more overhead sounds like a lot until it meets a real model: one call takes about 1.5 seconds in Evaluating and Observing Agents’s runs, so the framework adds well under a tenth of a percent. Framework overhead is almost never what makes an agent slow.

What the graph buys is the second half of the output. The approval version’s tools node calls interrupt() with the pending tool call, which pauses the run and saves its state. A person (or another program) sees the call, and here edits it, adding 100 to the expression; Command(resume=...) continues the run with that reply, and the answer changes accordingly. The checkpointer kept a snapshot after every step, five in all, each recording which node runs next. Invoking from the snapshot taken just before the tool forks the run from that point: it pauses for approval again, and approving unchanged this time gives the original answer. That is replay, the basis for debugging a run after the fact, and the same mechanism lets a run continue days later from a database. LangGraph’s documentation recommends this interrupt() function for human approval; the older interrupt_before option on the compiled graph is now positioned as a debugging breakpoint.

The hand-written loop can do all of this, but each piece (persistence, resumption, approval, replay) is code to write, test and keep consistent. Two more things the runtime provides that this example doesn’t show: streaming each step’s updates or the model’s tokens as they’re produced, and fanning work out to parallel nodes (its Send mechanism), which is how Many Agents’s orchestrator-worker pattern is built in LangGraph.

When each makes sense

The comparison suggests a rule rather than a winner:

• Write the loop by hand when the agent is a single loop with a few tools, runs in one process, and finishes in one go. It’s shorter, easier to read, and nothing is hidden. Most of this series’ examples are in this class.
• Use a graph framework when the agent needs what the checkpoint gives: long-running tasks that must survive restarts, human approval in the middle of a run, several agents or branches with shared state, or replaying a run to debug it. Those features are where hand-written code accumulates bugs.
• Either way, know what it compiles to. A chain is composition; a graph is a loop over a state machine with saved snapshots. When a framework behaves unexpectedly, that’s the level to debug at.

Where this leads

• How Coding Agents Work shows the same needs at production scale: checkpoints that rewind edits, permission prompts that pause a run, and hooks that run at fixed points in the loop.
• Evaluating and Observing Agents traces every step of an agent, which is where per-step visibility like the graph’s pays off.

Takeaways

• LCEL’s | builds a RunnableSequence: function composition with a shared interface (invoke, batch, stream, async).
• LangGraph runs an agent as a state graph: typed state with reducers, nodes that return updates, conditional edges, and a checkpoint after every step.
• The same agent was 10 lines by hand, 15 as a graph built from scratch and 10 with LangGraph’s prebuilt tool node; the graph added about 0.4 ms of framework overhead per run (0.85 ms with a checkpointer): negligible next to a model call.
• What the graph adds is persistence: pause for approval with interrupt(), edit and resume, and replay from any saved step. A framework earns its place when an agent needs that; otherwise a hand-written loop is simpler.

References

Start with the LangGraph documentation, especially its pages on persistence and interrupts, which this post’s second half exercises; the Pregel paper is where the step model comes from.

• LangChain, LangChain v1 release notes and LangGraph documentation: create_agent, state, reducers, checkpointers, interrupt() and replay.
• Malewicz et al., Pregel: A System for Large-Scale Graph Processing (SIGMOD 2010): the step model LangGraph’s execution borrows.