Delayed Message Processing: Mermaid Flowchart

Delayed Message Processing flowchart diagram

About Source

Delayed message processing is a pattern in which a message is published now but deliberately withheld from consumers until a specified time in the future, enabling scheduled execution of business logic without a separate cron or job scheduling system.

Many business workflows require time-delayed execution: send a reminder email 24 hours after a trial signup, cancel an order if payment is not confirmed within 15 minutes, retry a webhook delivery after 5 minutes. Rather than polling a database on a schedule, delayed messaging encodes the execution time into the message itself and lets the broker handle the scheduling.

Implementations vary by platform. AWS SQS supports a delivery delay of up to 15 minutes at the message level, after which the message becomes visible to consumers. For longer delays, the common pattern is a delay queue backed by a persistent store (DynamoDB, Redis Sorted Set with score = fire-at timestamp) from which a scheduler polls and re-enqueues messages to the main queue when their time arrives. RabbitMQ supports per-message TTL combined with dead-letter routing to achieve similar results.

The visibility timeout mechanism used by SQS-style queues is a related concept: when a consumer fetches a message, it becomes invisible to other consumers for a configurable period. If the consumer fails to acknowledge within that window, the message becomes visible again — similar to a short automatic delay for retry purposes. This mechanism powers Message Queue Retry.

For precision scheduling (sub-second granularity) or very long delays (days to weeks), a dedicated scheduler like Temporal, Quartz, or a database-backed job queue is more appropriate than a message broker's native delay features.

Frequently asked questions

Delayed message processing is a pattern where a message is published immediately but deliberately withheld from consumers until a specified future time. The broker or a delay layer holds the message and makes it visible only when the scheduled time arrives, allowing time-based business logic without a separate cron or job scheduling system.

AWS SQS supports per-message delivery delays up to 15 minutes natively. For longer delays, a common approach uses a persistent store such as a Redis sorted set with scores equal to the fire-at timestamp. A lightweight scheduler process polls the store and re-enqueues messages to the main queue when their time arrives. RabbitMQ achieves similar behaviour by combining per-message TTL with a dead-letter exchange that routes expired messages to the target queue.

Use it for reminder notifications (send 24 hours after signup), order timeout logic (cancel if not confirmed within 15 minutes), scheduled report generation, or any recurring business logic that needs to fire at a future time. It is preferable to polling a database on a cron schedule because the execution time is encoded in the message itself, reducing database load and avoiding missed firings.

mermaid

flowchart TD
    P[Producer] -->|publish with delay_until=T+24h| DS[Delay Store\nRedis Sorted Set\nscore = fire_at timestamp]

    SCHED[Scheduler\ncron every 1 min] -->|query: score <= now| DS
    DS -->|return due messages| SCHED
    SCHED -->|re-enqueue| MQ[Main Queue]
    DS -->|delete delivered message| DS

    MQ -->|message now visible| C[Consumer]
    C -->|process| BL[Execute Business Logic\nSend email, cancel order, etc.]
    C -->|ack| MQ

    subgraph ShortDelay[Short Delay - SQS Native]
        P2[Producer] -->|DelaySeconds=300| SQ[SQS Queue\nmessage invisible for 5m]
        SQ -->|visible after 5 minutes| C2[Consumer]
    end

    style BL fill:#4a4,color:#fff