How do you design idempotent agent workflow steps?

An idempotent agent workflow step produces the same intended external outcome when the same logical operation is executed more than once. This matters because workers can retry tasks, networks can time out, and checkpoint recovery can replay a node.

Beginner-friendly example: Saving generated API tests

An AI agent generates a test suite for the GET /orders/{id} endpoint. A worker saves the suite to a database, but its acknowledgment is lost. Celery retries the task.

• Without idempotency: The second attempt inserts another identical suite.
• With idempotency: Both attempts refer to the same logical save operation, so only one test-suite record is created.

For example, assign the save operation a stable key such as run-42:orders-get:save-suite:v1. If the same request arrives again with the same key and payload, return the existing result rather than creating another record.

How do you design an idempotent step?

1. Identify side effects: Look for database writes, emails, payments, deployments, and external API calls. Pure calculations generally do not create duplicate external effects.
2. Define a logical operation ID: Build a stable idempotency key from the workflow run, target resource, step, and operation version. A retry must reuse the original key—not generate a new random key.
3. Enforce deduplication atomically: Use a database unique constraint, transactional insert/upsert, or the external service's supported idempotency mechanism. A separate “check then insert” is vulnerable to concurrent retries.
4. Validate repeated requests: If a key is reused with materially different parameters, reject the conflict rather than silently treating a new action as an old one.
5. Record outcomes: Persist enough result information to recognize completed operations and return or reconcile their status.
6. Handle uncertain results: After a timeout, query the external system when possible before retrying. A local success flag cannot prove that a remote side effect did—or did not—occur.

Small database example

Assume a test_suites table has a unique constraint on operation_id. The save step can use an atomic insert with conflict handling:

SQL
INSERT INTO test_suites (operation_id, endpoint, content)
VALUES ('run-42:orders-get:save-suite:v1',
        'GET /orders/{id}', 'validated suite')
ON CONFLICT (operation_id) DO NOTHING;

A later read can retrieve the existing row and compare its stored content or request hash. The unique constraint prevents duplicate rows even if two Celery workers attempt the same operation concurrently.

Important: This protects only the specified database insert. If the step also sends an email or triggers another external service, that side effect needs its own deduplication strategy.

LangGraph and Celery considerations

• LangGraph: A node may run again after checkpoint replay or resume. Persist operation IDs and protect side effects independently from graph state.
• Celery: Delivery and retries can result in repeated task execution. Reusing a stable operation ID across attempts prevents duplicate effects.
• Parallel branches: Namespace operation IDs by branch and target, while ensuring retries of that branch reuse the same ID.
• Multi-tenancy: Scope identifiers and uniqueness appropriately so one tenant's operation never collides with another's.

Common mistakes

• Generating a fresh idempotency key on every retry.
• Relying only on an in-memory “already done” flag.
• Using a non-atomic read-before-write check.
• Assuming a timeout means the external operation failed.
• Treating checkpointing as automatic rollback of external actions.

In short

Design idempotent steps around stable operation identities and atomic deduplication at the side-effect boundary. This lets an agent safely retry or resume work without accidentally repeating the same action.