timebank-cc-public/references/QUEUE_WORKERS_SETUP.md

Queue Workers Setup

Overview

The application uses separate queue workers for different types of jobs:

• High Priority Worker: Handles critical emails (password resets, email verification)
• Main Worker: Handles standard emails and jobs (notifications, messages, etc.)
• Mailing Workers: Dedicated workers for bulk newsletter processing (3 workers for parallel processing)

Queue Priority

High Priority Worker

php artisan queue:work --queue=high --tries=3 --timeout=90

Dedicated worker for critical operations:

• high - Password resets, email verification
• Runs independently to ensure these emails are never delayed
• Fast timeout (90 seconds) for quick processing

Main Worker

php artisan queue:work --queue=messages,default,emails,low --tries=3 --timeout=90

Processes queues in this order:

1. messages - Real-time messaging
2. default - General jobs
3. emails - Standard emails (notifications, contact forms, etc.)
4. low - Background tasks

Note: The high queue has been removed from this worker and now runs independently.

Mailing Workers

php artisan queue:work --queue=mailing --tries=3 --timeout=600

Dedicated queue for bulk newsletters:

• mailing - Bulk newsletter batches
• Runs with 3 parallel workers for faster processing
• 10-minute timeout for large batches
• 3 retry attempts per batch

Systemd Service Configuration

High Priority Worker Service

File: /etc/systemd/system/timebank-high-priority-worker.service

[Unit]
Description=Timebank High Priority Queue Worker (Password Resets & Email Verification)
After=network.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/path/to/your/timebank/installation
ExecStart=/usr/bin/php artisan queue:work --queue=high --tries=3 --timeout=90
Restart=always
RestartSec=10

# Environment
Environment=PATH=/usr/bin:/bin
Environment=TIMEBANK_CONFIG=timebank_cc

# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=timebank-high-priority

[Install]
WantedBy=multi-user.target

Main Worker Service

File: /etc/systemd/system/timebank-queue-worker.service

[Unit]
Description=Timebank Main Queue Worker
After=network.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/path/to/your/timebank/installation
ExecStart=/usr/bin/php artisan queue:work --queue=messages,default,emails,low --tries=3 --timeout=90
Restart=always
RestartSec=10

# Environment
Environment=PATH=/usr/bin:/bin
Environment=TIMEBANK_CONFIG=timebank_cc

# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=timebank-queue
    
[Install]
WantedBy=multi-user.target

Mailing Worker Services

Create 3 separate service files for parallel processing:

File 1: /etc/systemd/system/timebank-mailing-worker-1.service File 2: /etc/systemd/system/timebank-mailing-worker-2.service File 3: /etc/systemd/system/timebank-mailing-worker-3.service

[Unit]
Description=Timebank Bulk Mailing Queue Worker 1
After=network.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/path/to/your/timebank/installation
ExecStart=/usr/bin/php artisan queue:work --queue=mailing --tries=3 --timeout=600
Restart=always
RestartSec=10

# Environment
Environment=PATH=/usr/bin:/bin
Environment=TIMEBANK_CONFIG=timebank_cc

# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=timebank-mailing-1

[Install]
WantedBy=multi-user.target

Note: Change timebank-mailing-1 to timebank-mailing-2 and timebank-mailing-3 in the other service files.

Installation Steps

1. Create Service Files

# Create all 5 service files
sudo nano /etc/systemd/system/timebank-high-priority-worker.service
sudo nano /etc/systemd/system/timebank-queue-worker.service
sudo nano /etc/systemd/system/timebank-mailing-worker-1.service
sudo nano /etc/systemd/system/timebank-mailing-worker-2.service
sudo nano /etc/systemd/system/timebank-mailing-worker-3.service

2. Update Paths

In each service file, replace:

• /path/to/your/timebank/installation with your actual installation path
• www-data with your web server user if different
• TIMEBANK_CONFIG=timebank_cc with your config name if different, i.e. timebank-default, uuro, etc

3. Reload Systemd

sudo systemctl daemon-reload

4. Enable Services (start on boot)

sign

# Enable high priority worker
sudo systemctl enable timebank-high-priority-worker

# Enable main worker
sudo systemctl enable timebank-queue-worker

# Enable mailing workers
sudo systemctl enable timebank-mailing-worker-1
sudo systemctl enable timebank-mailing-worker-2
sudo systemctl enable timebank-mailing-worker-3

5. Start Services

# Start high priority worker
sudo systemctl start timebank-high-priority-worker

# Start main worker
sudo systemctl start timebank-queue-worker

# Start mailing workers
sudo systemctl start timebank-mailing-worker-1
sudo systemctl start timebank-mailing-worker-2
sudo systemctl start timebank-mailing-worker-3

Management Commands

Check Status

# Check all workers
sudo systemctl status timebank-high-priority-worker
sudo systemctl status timebank-queue-worker
sudo systemctl status timebank-mailing-worker-1
sudo systemctl status timebank-mailing-worker-2
sudo systemctl status timebank-mailing-worker-3

# Or check all at once
sudo systemctl status 'timebank-*'

Stop Workers

# Stop specific worker
sudo systemctl stop timebank-mailing-worker-1

# Stop all mailing workers
sudo systemctl stop timebank-mailing-worker-{1,2,3}

# Stop all workers
sudo systemctl stop 'timebank-*'

Restart Workers

# Restart after code deployment
sudo systemctl restart timebank-high-priority-worker
sudo systemctl restart timebank-queue-worker
sudo systemctl restart timebank-mailing-worker-{1,2,3}

View Logs

# View high priority worker logs
sudo journalctl -u timebank-high-priority-worker -f

# View main worker logs
sudo journalctl -u timebank-queue-worker -f

# View mailing worker logs
sudo journalctl -u timebank-mailing-worker-1 -f

# View all worker logs
sudo journalctl -u 'timebank-*' -f

# View last 100 lines
sudo journalctl -u timebank-queue-worker -n 100

Performance

With This Setup

• Critical Emails: Processed immediately by dedicated high priority worker

  • Password resets: Sent within seconds
  • Email verification: Sent within seconds
  • Never blocked by other jobs

• Standard Emails: Processed by main worker

  • Notifications, messages, contact forms
  • Fast processing without bulk mailing delays

• Bulk Newsletters: Processed by 3 parallel workers

  • ~36 emails/minute (with 5 sec delay)
  • ~180 emails/minute (with 1 sec delay)
  • 2,182 emails complete in ~60 minutes (5 sec delay) or ~12 minutes (1 sec delay)
  • Completely isolated from critical and standard emails

Monitoring

# Watch queue in real-time
php artisan queue:monitor

# Check failed jobs
php artisan queue:failed

# Retry failed jobs
php artisan queue:retry all

Deployment Integration

Add to your deployment script after code updates:

# Restart all queue workers after deployment
sudo systemctl restart timebank-high-priority-worker
sudo systemctl restart timebank-queue-worker
sudo systemctl restart timebank-mailing-worker-{1,2,3}

Troubleshooting

Workers Not Processing Jobs

1. Check if services are running:

   sudo systemctl status 'timebank-*'
   

2. Check Laravel logs:

   tail -f storage/logs/laravel.log
   

3. Check systemd logs:

   sudo journalctl -u timebank-mailing-worker-1 -n 50
   

High Memory Usage

If workers consume too much memory, add memory limits to service files:

[Service]
# Restart worker after processing 1000 jobs
ExecStart=/usr/bin/php artisan queue:work --queue=mailing --tries=3 --timeout=600 --max-jobs=1000

Stuck Jobs

# Clear stuck jobs
php artisan queue:flush

# Restart workers
sudo systemctl restart 'timebank-*'

Configuration

Adjust these settings in config/timebank_cc.php:

'mailing' => [
    'batch_size' => 10,              // Recipients per job
    'send_delay_seconds' => 5,       // Delay between emails
    'max_retries' => 3,              // Job retry attempts
    'retry_delay_minutes' => 15,     // Initial retry delay
],