Cron Job Scheduler: Mermaid Flowchart Diagram

About Source

A cron job scheduler is a time-based task execution system that evaluates job schedules on a tick interval, acquires distributed locks to prevent duplicate execution across multiple instances, runs the scheduled task, and records the outcome.

What the diagram shows

This flowchart models a production-grade cron scheduler running in a horizontally-scaled environment where multiple instances of the scheduler process may be running simultaneously:

1. Tick: the scheduler wakes on a configurable interval (e.g., every minute). 2. Load due jobs: it queries the job store for jobs whose next_run_at timestamp is in the past. 3. Acquire distributed lock: for each due job, the scheduler attempts to acquire a lock in a shared store (Redis SET NX, Postgres advisory lock). This ensures only one scheduler instance executes the job even when multiple replicas are running. 4. Lock contention: if the lock is already held by another instance, this instance skips the job — the other instance is executing it. 5. Execute job: the winning instance runs the job logic, which may enqueue a background job, call an API, run a database cleanup, or generate a report. 6. Record outcome: success or failure is recorded in the job history table with timestamps and duration. 7. Update next run: on success, the scheduler computes and stores next_run_at based on the cron expression. 8. Release lock: the lock is released, allowing the next scheduled run to be claimed.

Why this matters

The distributed lock step is critical. Without it, every replica executes every job — leading to duplicate emails sent, duplicate billing runs, or duplicate data imports. Tools like Sidekiq-Cron, Celery Beat, and AWS EventBridge handle this coordination, but understanding the underlying pattern helps when rolling your own or debugging duplicate executions.

For one-off async jobs (as opposed to scheduled), see Background Job Processing. For the workers that execute the enqueued jobs, see Worker Queue Processing.

Frequently asked questions

A cron job scheduler is a time-based task execution system that evaluates job schedules on a tick interval and runs tasks at their configured times. In distributed environments, it uses distributed locks to ensure only one instance executes each job even when multiple scheduler replicas are running simultaneously.

On each tick, the scheduler loads all jobs whose next run time is in the past. For each due job, it attempts to acquire a distributed lock — typically a Redis SET NX or a Postgres advisory lock. The instance that wins the lock executes the job, records the outcome, computes the next run timestamp, and releases the lock. Instances that lose the lock skip the job entirely, knowing another replica is handling it.

Use a cron scheduler for any recurring time-based work: nightly database cleanups, daily billing runs, hourly report generation, or periodic data synchronization. If the job must run at a specific time regardless of system load — as opposed to being triggered by a user action — a cron scheduler is the right tool.

The most dangerous mistake is running multiple scheduler replicas without distributed locking, leading to duplicate executions — duplicate emails sent, duplicate charges processed, duplicate data imports. Another common issue is not recording job history, making it impossible to audit whether a job ran, when it last succeeded, and how long it took. Teams also frequently set overly tight schedules without accounting for job duration, causing overlapping runs.

mermaid

flowchart TD
    A([Scheduler tick]) --> B[Load jobs due for execution]
    B --> C{Any due jobs?}
    C -- No due jobs --> D([Sleep until next tick])
    C -- Jobs found --> E[For each due job]
    E --> F[Attempt distributed lock acquisition]
    F --> G{Lock acquired?}
    G -- Lock held by another instance --> H[Skip job this tick]
    H --> D
    G -- Lock acquired --> I[Mark job as running]
    I --> J[Execute job logic]
    J --> K{Execution result}
    K -- Success --> L[Record success in job history]
    L --> M[Compute next_run_at from cron expression]
    M --> N[Store updated schedule]
    N --> O[Release distributed lock]
    O --> D
    K -- Failure --> P[Record error and stack trace]
    P --> Q{Retry allowed?}
    Q -- Yes --> R[Schedule retry with backoff]
    R --> O
    Q -- No --> S[Emit alert]
    S --> O