Jobs and Queues — Background Processing That Doesn’t Fail Silently

Reading Time: 5 minutes

Series: Every Laravel Project Should Have These Building Blocks 
Part: 15 of 35 Level: Intermediate Prerequisites: Action Classes

---

What You’ll Learn

• When to use a Job vs. a queued Event Listener
• The anatomy of a well-written Job class
• Retry configuration, backoff strategies, and failure handling
• The HeartBeat pattern — checking queue health on a schedule
• Unique jobs and idempotency
• Named queues and job prioritization

---

The Problem Jobs Solve

Some operations take too long for an HTTP request:

• Sending emails (30–500ms each, potentially many)
• Resizing uploaded images
• Calling external APIs (webhook deliveries, payment captures)
• Generating PDF reports
• Syncing data with a CRM or ERP

If you do these inline, the user waits. If the external service is slow, the user waits longer. If the service is down, the request fails. Queues solve all three: the work happens in the background, retries on failure, and the user sees a response immediately.

---

Job Anatomy

<?php

declare(strict_types=1);

namespace App\Jobs;

use App\Models\Order;
use App\Services\InventoryService;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Throwable;

class ProcessOrderInventory implements ShouldQueue
{
    use Dispatchable;
    use InteractsWithQueue;
    use Queueable;
    use SerializesModels;

    // How many times to attempt before marking as failed
    public int $tries = 3;

    // Wait this many seconds before first retry
    public int $backoff = 10;

    // Timeout for a single attempt (seconds)
    public int $timeout = 60;

    public function __construct(
        public readonly Order $order
    ) {}

    public function handle(InventoryService $inventory): void
    {
        $inventory->reserveForOrder($this->order);
    }

    /**
     * Called when all retries are exhausted.
     */
    public function failed(Throwable $e): void
    {
        logger()->error('ProcessOrderInventory failed permanently', [
            'order_id' => $this->order->id,
            'error'    => $e->getMessage(),
        ]);

        // Notify the team or create an incident record
        $this->order->update(['fulfillment_status' => 'manual_review']);
    }
}

Then dispatch it:

// In an Action, after the order is created:
ProcessOrderInventory::dispatch($order);

// Or with a delay:
ProcessOrderInventory::dispatch($order)->delay(now()->addMinutes(2));

---

Retry and Backoff Configuration

Not all jobs should retry the same way. An API rate limit error needs exponential backoff. A one-off timeout might just need one retry.

// Fixed backoff: wait 30s, 60s, 90s between retries
public int $tries = 3;
public array $backoff = [30, 60, 90];

// Exponential backoff via method:
public function backoff(): array
{
    return [1, 5, 15]; // seconds between each retry
}

// Max time to attempt at all (regardless of $tries)
public int $retryUntil = 3600; // 1 hour

For API calls that might hit rate limits:

public function retryUntil(): \DateTime
{
    return now()->addHours(6); // keep trying for 6 hours
}

---

Named Queues

Different jobs need different resources. Email sends don’t need the same queue slot as report generation:

// Assign queue at dispatch time
ProcessOrderInventory::dispatch($order)->onQueue('inventory');
SyncCrmContact::dispatch($user)->onQueue('crm-sync');
SendWeeklyNewsletter::dispatch()->onQueue('bulk-email');

// Or set a default queue on the job class:
class SyncCrmContact implements ShouldQueue
{
    public string $queue = 'crm-sync';
}

Run separate worker processes for separate queues, with different concurrency settings. Critical queues get more workers:

php artisan queue:work --queue=high,default,low

The worker processes jobs from high first, then default, then low.

---

Idempotent Jobs

A job may execute more than once if it times out partway through and gets retried. If your job writes to the database, it must be idempotent — safe to run multiple times with the same result.

class SyncProductToShopify implements ShouldQueue
{
    public function handle(): void
    {
        // ❌ Non-idempotent — creates duplicates on retry
        ShopifyProduct::create([...]);

        // ✅ Idempotent — upsert by external ID
        ShopifyProduct::updateOrCreate(
            ['shopify_id' => $this->product->shopify_id],
            [...],
        );
    }
}

For jobs that must only run once (e.g., charge a payment):

use Illuminate\Contracts\Queue\ShouldBeUnique;

class ChargeOrderPayment implements ShouldQueue, ShouldBeUnique
{
    public function __construct(public readonly Order $order) {}

    // The unique lock key — no two jobs with the same order ID can run simultaneously
    public function uniqueId(): int
    {
        return $this->order->id;
    }
}

---

The HeartBeat Pattern

Queue workers can silently die. A worker process might crash, run out of memory, or get killed by the OS. If no one checks, jobs pile up unprocessed.

The HeartBeat job solves this: schedule it every minute, have it write a timestamp to the database, and alert if it doesn’t run.

// app/Jobs/HeartBeatJob.php
final class HeartBeatJob implements ShouldQueue
{
    use Dispatchable;
    use InteractsWithQueue;
    use Queueable;
    use SerializesModels;

    public int $tries = 1;

    public function handle(): void
    {
        HeartBeat::updateOrCreate(
            ['id' => 1],
            ['last_beat_at' => now()]
        );

        Log::channel('heartbeat')->info('HeartBeat job ran', [
            'timestamp' => now()->toISOString(),
            'queue'     => $this->queue ?? 'default',
        ]);
    }

    public function failed(Throwable $e): void
    {
        Log::channel('heartbeat')->error('HeartBeat job failed', [
            'error' => $e->getMessage(),
        ]);
    }
}

Schedule it in routes/console.php (Laravel 11+):

Schedule::job(new HeartBeatJob)->everyMinute()->onQueue('default');

Then check queue health from any status page:

final class QueueCheck implements StatusCheckInterface
{
    public function run(): ServiceCheckResult
    {
        $heartbeat = HeartBeat::first();

        if ($heartbeat === null || $heartbeat->last_beat_at < now()->subMinutes(3)) {
            return new ServiceCheckResult(
                name: 'Queue',
                icon: 'fa-list-check',
                status: ServiceStatus::Outage,
                message: 'No heartbeat in the last 3 minutes',
            );
        }

        $lagSeconds = $heartbeat->last_beat_at->diffInSeconds(now());

        return new ServiceCheckResult(
            name: 'Queue',
            icon: 'fa-list-check',
            status: ServiceStatus::Operational,
            message: "Healthy — {$lagSeconds}s since last beat",
        );
    }
}

---

Job Chaining and Batching

When one job must run after another:

// Sequential — second job only runs if the first succeeds
Bus::chain([
    new ProcessPayment($order),
    new FulfillOrder($order),
    new SendOrderConfirmation($order),
])->dispatch();

When jobs can run in parallel and you want to know when they’re all done:

$batch = Bus::batch([
    new ProcessProductImage($product, 'thumbnail'),
    new ProcessProductImage($product, 'medium'),
    new ProcessProductImage($product, 'full'),
])->then(function (Batch $batch) use ($product): void {
    $product->update(['images_processed' => true]);
})->catch(function (Batch $batch, Throwable $e): void {
    logger()->error("Image processing batch failed: {$e->getMessage()}");
})->dispatch();

---

Key Takeaways

• Jobs for background work: emails, API calls, report generation, image processing, data sync.
• Set $tries, $backoff, and $timeout on every job — don’t rely on defaults.
• Implement failed() to handle exhausted retries explicitly: log it, alert, set a status flag.
• Use ShouldBeUnique for jobs that must only have one in-flight instance at a time.
• Named queues (high, default, low) let you prioritize and scale independently.
• The HeartBeat job is your queue health monitor: schedule it every minute, alert when it stops running.

---

---

Frequently Asked Questions

What’s the difference between a Job and a queued Event Listener? Functionally, almost nothing — both run on a queue worker via the same mechanism. Semantically: Jobs are for work you explicitly push (dispatch(new GenerateWeeklyReport())). Queued Listeners are for reacting to domain events asynchronously (class SendOrderEmailListener implements ShouldQueue). Use Jobs for scheduled, batch, or triggered work; use queued Listeners for event-driven async work.

Should every slow operation be a Job? Not necessarily. Jobs add infrastructure overhead: you need a queue driver (Redis or database), a queue worker process, and failure handling. If an operation takes under 500ms and is user-facing, keep it synchronous. Dispatch a Job when the operation takes seconds, blocks the response, has external dependencies that might be slow, or needs retry logic on failure.

What’s the right number of retry attempts? It depends on the operation. Email: 3 attempts with exponential backoff. Third-party API calls: 5 attempts. Operations that are idempotent (can run twice with the same result): be generous. Operations that have side effects that can’t be undone (charging a card): 1 attempt, then alert and manual review.

---

Tips and Gotchas

⚠️ Warning: Never dispatch a job inside a database transaction if the job reads data that the transaction writes. The job may run before the transaction commits, reading stale or missing data. Use DB::afterCommit() to dispatch after the transaction succeeds.

💡 Tip: Set $tries = 1 on jobs that are not idempotent (cannot be safely re-run). For idempotent jobs, use $tries = 3 with exponential backoff. Retrying non-idempotent jobs silently doubles or triples side effects.

🔥 Expert Note: Use separate queues for different workloads: default for general tasks, emails for mail, heavyfor exports and reports. This prevents a backlog of low-priority exports from delaying high-priority transactional emails.

Further Reading

• Laravel Docs: Queues
• Laravel Docs: Job Middleware
• Laravel Docs: Rate Limiting on Jobs
• Laravel Horizon — Redis queue monitoring dashboard

---

← Support Classes | Next: Artisan Commands →