← All writing
FIELD NOTES / Backend · Queues

A queue is a boundary, not a magic fix

Moving work into the background changes when failures happen. It does not remove the need to handle them.

Decide what the user must wait for

A request often mixes essential work with work that can happen later. Creating the authoritative record may need to finish before responding. Sending a notification or generating a report may not.

A queue lets these activities have different lifecycles. That separation is useful only when the product makes the pending state understandable. Accepted is not the same as completed.

Pass identity, not a snapshot of everything

A job usually needs an identifier and enough context to establish its purpose. Loading the authoritative record when processing avoids carrying an unnecessarily large payload, but it introduces a question: should the job use the current data or the data as it existed when requested?

Choose deliberately. A report requested for a specific period may require immutable inputs, while a notification may need the latest delivery preferences.

Assume the job can run again

Workers can stop after completing a side effect but before acknowledging the job. A retry can therefore repeat work. Design handlers with idempotency, durable state transitions, and explicit duplicate handling.

A retry policy also needs limits. Repeating a permanently invalid operation creates noise, while retrying a temporary dependency failure may be useful. Backoff helps avoid hammering an unhealthy service.

Make failure visible

Queue depth alone is incomplete. Track the age of waiting work, execution time, retries, and failed jobs. A small queue containing old work can be more concerning than a larger queue that drains quickly.

Give operators a way to inspect failures and replay work deliberately. Include useful identifiers in logs so a failed job can be connected to the original user action.

Own the deployment lifecycle

Long-running workers need a release strategy. They may still be running old code after application files change. Shutdown, restart, and timeout behavior are part of the application design.

The practical test is simple: if a worker disappears at any point, can the system explain what happened and recover without silently losing or duplicating important work?

KEEP EXPLORING

There's more
behind the code.

Explore all writing ↗
✳ Ask about Majid