Laravel Jobs & Queues: Retry, Backoff, Failed Jobs & Idempotency

Reading Time: 7 minutes

Series: Every Laravel Project Should Have These Building Blocks 
Part: 15 of 35 Level: Intermediate Prerequisites: Action Classes

---

What You’ll Learn

• When to use a Job vs. a queued Event Listener
• The anatomy of a well-written Job class
• Retry configuration, backoff strategies, and failure handling
• The HeartBeat pattern — checking queue health on a schedule
• Unique jobs and idempotency
• Named queues and job prioritization

---

Short answer: set $tries, $backoff and $timeout on every job; keep $timeout below the connection’s retry_after; use retryUntil() when a deadline matters more than an attempt count; and make any job with side effects idempotent, because Laravel queues deliver at least once.

$tries vs $backoff vs retryUntil() vs retry_after

These four settings get mixed up constantly. They control different things:

Setting	Where	What it controls
$tries	Job class (or --trieson the worker)	Maximum attempts before the job goes to failed_jobs. The job property wins over the worker flag.
$backoff	Job class (or --backoff)	Seconds to wait before retrying after an exception. An array sets a delay per attempt, and the last value repeats for any remaining attempts.
retryUntil()	Job class	A deadline. When defined, $tries is ignored and the job retries until this time passes.
$maxExceptions	Job class	How many unhandled exceptions to allow. Useful with retryUntil() or release(), where attempts aren’t the real limit.
$timeout	Job class (or --timeout)	Seconds before the worker kills a running job.
retry_after	config/queue.php, per connection	Seconds before a reserved, unfinished job is assumed dead and released again.

The rule that prevents duplicate runs: $timeout must be several seconds shorter than retry_after. If retry_after is 90 and a job runs for 120 seconds, a second worker picks it up while the first is still running.

// config/queue.php
'redis' => [
    'driver' => 'redis',
    'retry_after' => 180, // longest job timeout + margin
],

// The job
public int $tries = 3;
public int $timeout = 120;      // < retry_after
public array $backoff = [10, 60, 300];

If a timeout means the job will never succeed, set public bool $failOnTimeout = true; so it fails instead of burning attempts.

Retrying failed Laravel jobs

php artisan queue:failed                 # list failed jobs with their UUIDs
php artisan queue:retry <uuid>           # retry one job
php artisan queue:retry all              # retry everything
php artisan queue:retry --queue=emails   # retry one queue's failures
php artisan queue:forget <uuid>          # delete one failed job
php artisan queue:prune-failed --hours=48

Before you run queue:retry all, fix the cause. Mass-retrying a bug just refills failed_jobs.

Inside a job you have two manual controls:

$this->release(30);  // put back on the queue in 30s (counts as an attempt)
$this->fail($e);     // fail now, no more retries

Delaying jobs and dispatching to a specific queue

SendReminder::dispatch($userId)
    ->onConnection('redis')
    ->onQueue('emails')
    ->delay(now()->addMinutes(10));

SQS caps delays at 15 minutes. For longer delays, store a timestamp and let the scheduler dispatch the job.

Dispatching jobs after the database transaction commits

If you dispatch inside a transaction, a fast worker can run the job before the commit. It then can’t find the row, or sees stale data.

DB::transaction(function () use ($order) {
    $order->save();
    ProcessOrder::dispatch($order->id)->afterCommit();
});

Other ways to get the same behaviour: set 'after_commit' => true on the connection in config/queue.php, or implement ShouldQueueAfterCommit on the job, listener or notification. If the transaction rolls back, the job is never dispatched.

Job chains: run steps in order, stop on failure

Bus::chain([
    new CompressVideo($mediaId),
    new GenerateThumbnail($mediaId),
    new NotifyOwner($mediaId),
])->catch(function (Throwable $e) use ($mediaId) {
    logger()->error('Media pipeline failed', ['media_id' => $mediaId]);
})->onQueue('media')->dispatch();

If one job fails, the rest of the chain doesn’t run. Each step retries on its own $tries and $backoff.

Idempotency: making retries safe

Laravel queues are at-least-once. A job can run twice after a timeout, a crash or a deploy, so any side effect (charging a card, sending an email, creating a record) must be safe to repeat.

• Guard with state: check if ($invoice->paid_at) return; before acting.
• Let the database enforce it: add a unique index on an idempotency key and catch the duplicate-key exception.
• Prevent duplicates in the queue: implement ShouldBeUnique with a uniqueId() and a bounded $uniqueFor.
• Prevent concurrent runs: use the WithoutOverlapping middleware keyed by record ID.

public function middleware(): array
{
    return [(new WithoutOverlapping($this->orderId))->releaseAfter(60)];
}

If a model may be deleted before the job runs, set public bool $deleteWhenMissingModels = true; so the job is quietly discarded instead of failing.

How do I retry a single failed job by ID?

Run php artisan queue:failed to find its UUID, then php artisan queue:retry <uuid>.

The Problem Jobs Solve

Some operations take too long for an HTTP request:

• Sending emails (30–500ms each, potentially many)
• Resizing uploaded images
• Calling external APIs (webhook deliveries, payment captures)
• Generating PDF reports
• Syncing data with a CRM or ERP

If you do these inline, the user waits. If the external service is slow, the user waits longer. If the service is down, the request fails. Queues solve all three: the work happens in the background, retries on failure, and the user sees a response immediately.

---

Job Anatomy

<?php

declare(strict_types=1);

namespace App\Jobs;

use App\Models\Order;
use App\Services\InventoryService;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Throwable;

class ProcessOrderInventory implements ShouldQueue
{
    use Dispatchable;
    use InteractsWithQueue;
    use Queueable;
    use SerializesModels;

    // How many times to attempt before marking as failed
    public int $tries = 3;

    // Wait this many seconds before first retry
    public int $backoff = 10;

    // Timeout for a single attempt (seconds)
    public int $timeout = 60;

    public function __construct(
        public readonly Order $order
    ) {}

    public function handle(InventoryService $inventory): void
    {
        $inventory->reserveForOrder($this->order);
    }

    /**
     * Called when all retries are exhausted.
     */
    public function failed(Throwable $e): void
    {
        logger()->error('ProcessOrderInventory failed permanently', [
            'order_id' => $this->order->id,
            'error'    => $e->getMessage(),
        ]);

        // Notify the team or create an incident record
        $this->order->update(['fulfillment_status' => 'manual_review']);
    }
}

Then dispatch it:

// In an Action, after the order is created:
ProcessOrderInventory::dispatch($order);

// Or with a delay:
ProcessOrderInventory::dispatch($order)->delay(now()->addMinutes(2));

---

Retry and Backoff Configuration

Not all jobs should retry the same way. An API rate limit error needs exponential backoff. A one-off timeout might just need one retry.

// Fixed backoff: wait 30s, 60s, 90s between retries
public int $tries = 3;
public array $backoff = [30, 60, 90];

// Exponential backoff via method:
public function backoff(): array
{
    return [1, 5, 15]; // seconds between each retry
}

// Max time to attempt at all (regardless of $tries)
public int $retryUntil = 3600; // 1 hour

For API calls that might hit rate limits:

public function retryUntil(): \DateTime
{
    return now()->addHours(6); // keep trying for 6 hours
}

---

Named Queues

Different jobs need different resources. Email sends don’t need the same queue slot as report generation:

// Assign queue at dispatch time
ProcessOrderInventory::dispatch($order)->onQueue('inventory');
SyncCrmContact::dispatch($user)->onQueue('crm-sync');
SendWeeklyNewsletter::dispatch()->onQueue('bulk-email');

// Or set a default queue on the job class:
class SyncCrmContact implements ShouldQueue
{
    public string $queue = 'crm-sync';
}

Run separate worker processes for separate queues, with different concurrency settings. Critical queues get more workers:

php artisan queue:work --queue=high,default,low

The worker processes jobs from high first, then default, then low.

---

Idempotent Jobs

A job may execute more than once if it times out partway through and gets retried. If your job writes to the database, it must be idempotent – safe to run multiple times with the same result.

class SyncProductToShopify implements ShouldQueue
{
    public function handle(): void
    {
        // ❌ Non-idempotent — creates duplicates on retry
        ShopifyProduct::create([...]);

        // ✅ Idempotent — upsert by external ID
        ShopifyProduct::updateOrCreate(
            ['shopify_id' => $this->product->shopify_id],
            [...],
        );
    }
}

For jobs that must only run once (e.g., charge a payment):

use Illuminate\Contracts\Queue\ShouldBeUnique;

class ChargeOrderPayment implements ShouldQueue, ShouldBeUnique
{
    public function __construct(public readonly Order $order) {}

    // The unique lock key — no two jobs with the same order ID can run simultaneously
    public function uniqueId(): int
    {
        return $this->order->id;
    }
}

---

The HeartBeat Pattern

Queue workers can silently die. A worker process might crash, run out of memory, or get killed by the OS. If no one checks, jobs pile up unprocessed.

The HeartBeat job solves this: schedule it every minute, have it write a timestamp to the database, and alert if it doesn’t run.

// app/Jobs/HeartBeatJob.php
final class HeartBeatJob implements ShouldQueue
{
    use Dispatchable;
    use InteractsWithQueue;
    use Queueable;
    use SerializesModels;

    public int $tries = 1;

    public function handle(): void
    {
        HeartBeat::updateOrCreate(
            ['id' => 1],
            ['last_beat_at' => now()]
        );

        Log::channel('heartbeat')->info('HeartBeat job ran', [
            'timestamp' => now()->toISOString(),
            'queue'     => $this->queue ?? 'default',
        ]);
    }

    public function failed(Throwable $e): void
    {
        Log::channel('heartbeat')->error('HeartBeat job failed', [
            'error' => $e->getMessage(),
        ]);
    }
}

Schedule it in routes/console.php (Laravel 11+):

Schedule::job(new HeartBeatJob)->everyMinute()->onQueue('default');

Then check queue health from any status page:

final class QueueCheck implements StatusCheckInterface
{
    public function run(): ServiceCheckResult
    {
        $heartbeat = HeartBeat::first();

        if ($heartbeat === null || $heartbeat->last_beat_at < now()->subMinutes(3)) {
            return new ServiceCheckResult(
                name: 'Queue',
                icon: 'fa-list-check',
                status: ServiceStatus::Outage,
                message: 'No heartbeat in the last 3 minutes',
            );
        }

        $lagSeconds = $heartbeat->last_beat_at->diffInSeconds(now());

        return new ServiceCheckResult(
            name: 'Queue',
            icon: 'fa-list-check',
            status: ServiceStatus::Operational,
            message: "Healthy — {$lagSeconds}s since last beat",
        );
    }
}

---

Job Chaining and Batching

When one job must run after another:

// Sequential — second job only runs if the first succeeds
Bus::chain([
    new ProcessPayment($order),
    new FulfillOrder($order),
    new SendOrderConfirmation($order),
])->dispatch();

When jobs can run in parallel and you want to know when they’re all done:

$batch = Bus::batch([
    new ProcessProductImage($product, 'thumbnail'),
    new ProcessProductImage($product, 'medium'),
    new ProcessProductImage($product, 'full'),
])->then(function (Batch $batch) use ($product): void {
    $product->update(['images_processed' => true]);
})->catch(function (Batch $batch, Throwable $e): void {
    logger()->error("Image processing batch failed: {$e->getMessage()}");
})->dispatch();

---

Key Takeaways

• Jobs for background work: emails, API calls, report generation, image processing, data sync.
• Set $tries, $backoff, and $timeout on every job – don’t rely on defaults.
• Implement failed() to handle exhausted retries explicitly: log it, alert, set a status flag.
• Use ShouldBeUnique for jobs that must only have one in-flight instance at a time.
• Named queues (high, default, low) let you prioritize and scale independently.
• The HeartBeat job is your queue health monitor: schedule it every minute, alert when it stops running.

---

Frequently Asked Questions

What’s the difference between a Job and a queued Event Listener? 
Functionally, almost nothing – both run on a queue worker via the same mechanism. Semantically: Jobs are for work you explicitly push (dispatch(new GenerateWeeklyReport())). Queued Listeners are for reacting to domain events asynchronously (class SendOrderEmailListener implements ShouldQueue). Use Jobs for scheduled, batch, or triggered work; use queued Listeners for event-driven async work.

Should every slow operation be a Job? 
Not necessarily. Jobs add infrastructure overhead: you need a queue driver (Redis or database), a queue worker process, and failure handling. If an operation takes under 500ms and is user-facing, keep it synchronous. Dispatch a Job when the operation takes seconds, blocks the response, has external dependencies that might be slow, or needs retry logic on failure.

What’s the right number of retry attempts? 
It depends on the operation. Email: 3 attempts with exponential backoff. Third-party API calls: 5 attempts. Operations that are idempotent (can run twice with the same result): be generous. Operations that have side effects that can’t be undone (charging a card): 1 attempt, then alert and manual review.

---

Tips and Gotchas

⚠️ Warning: Never dispatch a job inside a database transaction if the job reads data that the transaction writes. The job may run before the transaction commits, reading stale or missing data. Use DB::afterCommit() to dispatch after the transaction succeeds.

💡 Tip: Set $tries = 1 on jobs that are not idempotent (cannot be safely re-run). For idempotent jobs, use $tries = 3 with exponential backoff. Retrying non-idempotent jobs silently doubles or triples side effects.

🔥 Expert Note: Use separate queues for different workloads: default for general tasks, emails for mail, heavyfor exports and reports. This prevents a backlog of low-priority exports from delaying high-priority transactional emails.

FAQ

What happens when $tries and retryUntil() are both set?

retryUntil() wins and $tries is ignored. Use $maxExceptions to cap real failures within the deadline.

Why does my Laravel job run twice?

Usually $timeout is longer than the connection’s retry_after, so a second worker picks the job up while the first is still running. Less often, a worker crashed mid-job. Fix the timing, and make the job idempotent.

What’s the default backoff in Laravel?

Zero. Without $backoff, a failed job is retried immediately, which burns through all its attempts in seconds during an outage.

Further Reading

• Laravel Docs: Queues
• Laravel Docs: Job Middleware
• Laravel Docs: Rate Limiting on Jobs
• Laravel Horizon – Redis queue monitoring dashboard

---

← Support Classes | Next: Artisan Commands →