355 lines
8.3 KiB
Markdown
355 lines
8.3 KiB
Markdown
# Queue Workers Setup
|
|
|
|
## Overview
|
|
|
|
The application uses separate queue workers for different types of jobs:
|
|
|
|
- **High Priority Worker**: Handles critical emails (password resets, email verification)
|
|
- **Main Worker**: Handles standard emails and jobs (notifications, messages, etc.)
|
|
- **Mailing Workers**: Dedicated workers for bulk newsletter processing (3 workers for parallel processing)
|
|
|
|
## Queue Priority
|
|
|
|
### High Priority Worker
|
|
```bash
|
|
php artisan queue:work --queue=high --tries=3 --timeout=90
|
|
```
|
|
|
|
Dedicated worker for critical operations:
|
|
- `high` - Password resets, email verification
|
|
- Runs independently to ensure these emails are never delayed
|
|
- Fast timeout (90 seconds) for quick processing
|
|
|
|
### Main Worker
|
|
```bash
|
|
php artisan queue:work --queue=messages,default,emails,low --tries=3 --timeout=90
|
|
```
|
|
|
|
Processes queues in this order:
|
|
1. `messages` - Real-time messaging
|
|
2. `default` - General jobs
|
|
3. `emails` - Standard emails (notifications, contact forms, etc.)
|
|
4. `low` - Background tasks
|
|
|
|
**Note:** The `high` queue has been removed from this worker and now runs independently.
|
|
|
|
### Mailing Workers
|
|
```bash
|
|
php artisan queue:work --queue=mailing --tries=3 --timeout=600
|
|
```
|
|
|
|
Dedicated queue for bulk newsletters:
|
|
- `mailing` - Bulk newsletter batches
|
|
- Runs with 3 parallel workers for faster processing
|
|
- 10-minute timeout for large batches
|
|
- 3 retry attempts per batch
|
|
|
|
## Systemd Service Configuration
|
|
|
|
### High Priority Worker Service
|
|
|
|
**File:** `/etc/systemd/system/timebank-high-priority-worker.service`
|
|
|
|
```ini
|
|
[Unit]
|
|
Description=Timebank High Priority Queue Worker (Password Resets & Email Verification)
|
|
After=network.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=www-data
|
|
Group=www-data
|
|
WorkingDirectory=/path/to/your/timebank/installation
|
|
ExecStart=/usr/bin/php artisan queue:work --queue=high --tries=3 --timeout=90
|
|
Restart=always
|
|
RestartSec=10
|
|
|
|
# Environment
|
|
Environment=PATH=/usr/bin:/bin
|
|
Environment=TIMEBANK_CONFIG=timebank_cc
|
|
|
|
# Logging
|
|
StandardOutput=journal
|
|
StandardError=journal
|
|
SyslogIdentifier=timebank-high-priority
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
```
|
|
|
|
### Main Worker Service
|
|
|
|
**File:** `/etc/systemd/system/timebank-queue-worker.service`
|
|
|
|
```ini
|
|
[Unit]
|
|
Description=Timebank Main Queue Worker
|
|
After=network.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=www-data
|
|
Group=www-data
|
|
WorkingDirectory=/path/to/your/timebank/installation
|
|
ExecStart=/usr/bin/php artisan queue:work --queue=messages,default,emails,low --tries=3 --timeout=90
|
|
Restart=always
|
|
RestartSec=10
|
|
|
|
# Environment
|
|
Environment=PATH=/usr/bin:/bin
|
|
Environment=TIMEBANK_CONFIG=timebank_cc
|
|
|
|
# Logging
|
|
StandardOutput=journal
|
|
StandardError=journal
|
|
SyslogIdentifier=timebank-queue
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
```
|
|
|
|
### Mailing Worker Services
|
|
|
|
Create 3 separate service files for parallel processing:
|
|
|
|
**File 1:** `/etc/systemd/system/timebank-mailing-worker-1.service`
|
|
**File 2:** `/etc/systemd/system/timebank-mailing-worker-2.service`
|
|
**File 3:** `/etc/systemd/system/timebank-mailing-worker-3.service`
|
|
|
|
```ini
|
|
[Unit]
|
|
Description=Timebank Bulk Mailing Queue Worker 1
|
|
After=network.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=www-data
|
|
Group=www-data
|
|
WorkingDirectory=/path/to/your/timebank/installation
|
|
ExecStart=/usr/bin/php artisan queue:work --queue=mailing --tries=3 --timeout=600
|
|
Restart=always
|
|
RestartSec=10
|
|
|
|
# Environment
|
|
Environment=PATH=/usr/bin:/bin
|
|
Environment=TIMEBANK_CONFIG=timebank_cc
|
|
|
|
# Logging
|
|
StandardOutput=journal
|
|
StandardError=journal
|
|
SyslogIdentifier=timebank-mailing-1
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
```
|
|
|
|
> **Note:** Change `timebank-mailing-1` to `timebank-mailing-2` and `timebank-mailing-3` in the other service files.
|
|
|
|
## Installation Steps
|
|
|
|
### 1. Create Service Files
|
|
|
|
```bash
|
|
# Create all 5 service files
|
|
sudo nano /etc/systemd/system/timebank-high-priority-worker.service
|
|
sudo nano /etc/systemd/system/timebank-queue-worker.service
|
|
sudo nano /etc/systemd/system/timebank-mailing-worker-1.service
|
|
sudo nano /etc/systemd/system/timebank-mailing-worker-2.service
|
|
sudo nano /etc/systemd/system/timebank-mailing-worker-3.service
|
|
```
|
|
|
|
### 2. Update Paths
|
|
|
|
In each service file, replace:
|
|
- `/path/to/your/timebank/installation` with your actual installation path
|
|
- `www-data` with your web server user if different
|
|
- `TIMEBANK_CONFIG=timebank_cc` with your config name if different, i.e. timebank-default, uuro, etc
|
|
|
|
### 3. Reload Systemd
|
|
|
|
```bash
|
|
sudo systemctl daemon-reload
|
|
```
|
|
|
|
### 4. Enable Services (start on boot)
|
|
sign
|
|
```bash
|
|
# Enable high priority worker
|
|
sudo systemctl enable timebank-high-priority-worker
|
|
|
|
# Enable main worker
|
|
sudo systemctl enable timebank-queue-worker
|
|
|
|
# Enable mailing workers
|
|
sudo systemctl enable timebank-mailing-worker-1
|
|
sudo systemctl enable timebank-mailing-worker-2
|
|
sudo systemctl enable timebank-mailing-worker-3
|
|
```
|
|
|
|
### 5. Start Services
|
|
|
|
```bash
|
|
# Start high priority worker
|
|
sudo systemctl start timebank-high-priority-worker
|
|
|
|
# Start main worker
|
|
sudo systemctl start timebank-queue-worker
|
|
|
|
# Start mailing workers
|
|
sudo systemctl start timebank-mailing-worker-1
|
|
sudo systemctl start timebank-mailing-worker-2
|
|
sudo systemctl start timebank-mailing-worker-3
|
|
```
|
|
|
|
## Management Commands
|
|
|
|
### Check Status
|
|
|
|
```bash
|
|
# Check all workers
|
|
sudo systemctl status timebank-high-priority-worker
|
|
sudo systemctl status timebank-queue-worker
|
|
sudo systemctl status timebank-mailing-worker-1
|
|
sudo systemctl status timebank-mailing-worker-2
|
|
sudo systemctl status timebank-mailing-worker-3
|
|
|
|
# Or check all at once
|
|
sudo systemctl status 'timebank-*'
|
|
```
|
|
|
|
### Stop Workers
|
|
|
|
```bash
|
|
# Stop specific worker
|
|
sudo systemctl stop timebank-mailing-worker-1
|
|
|
|
# Stop all mailing workers
|
|
sudo systemctl stop timebank-mailing-worker-{1,2,3}
|
|
|
|
# Stop all workers
|
|
sudo systemctl stop 'timebank-*'
|
|
```
|
|
|
|
### Restart Workers
|
|
|
|
```bash
|
|
# Restart after code deployment
|
|
sudo systemctl restart timebank-high-priority-worker
|
|
sudo systemctl restart timebank-queue-worker
|
|
sudo systemctl restart timebank-mailing-worker-{1,2,3}
|
|
```
|
|
|
|
### View Logs
|
|
|
|
```bash
|
|
# View high priority worker logs
|
|
sudo journalctl -u timebank-high-priority-worker -f
|
|
|
|
# View main worker logs
|
|
sudo journalctl -u timebank-queue-worker -f
|
|
|
|
# View mailing worker logs
|
|
sudo journalctl -u timebank-mailing-worker-1 -f
|
|
|
|
# View all worker logs
|
|
sudo journalctl -u 'timebank-*' -f
|
|
|
|
# View last 100 lines
|
|
sudo journalctl -u timebank-queue-worker -n 100
|
|
```
|
|
|
|
## Performance
|
|
|
|
### With This Setup
|
|
|
|
- **Critical Emails**: Processed immediately by dedicated high priority worker
|
|
- Password resets: Sent within seconds
|
|
- Email verification: Sent within seconds
|
|
- Never blocked by other jobs
|
|
|
|
- **Standard Emails**: Processed by main worker
|
|
- Notifications, messages, contact forms
|
|
- Fast processing without bulk mailing delays
|
|
|
|
- **Bulk Newsletters**: Processed by 3 parallel workers
|
|
- ~36 emails/minute (with 5 sec delay)
|
|
- ~180 emails/minute (with 1 sec delay)
|
|
- 2,182 emails complete in ~60 minutes (5 sec delay) or ~12 minutes (1 sec delay)
|
|
- Completely isolated from critical and standard emails
|
|
|
|
### Monitoring
|
|
|
|
```bash
|
|
# Watch queue in real-time
|
|
php artisan queue:monitor
|
|
|
|
# Check failed jobs
|
|
php artisan queue:failed
|
|
|
|
# Retry failed jobs
|
|
php artisan queue:retry all
|
|
```
|
|
|
|
## Deployment Integration
|
|
|
|
Add to your deployment script after code updates:
|
|
|
|
```bash
|
|
# Restart all queue workers after deployment
|
|
sudo systemctl restart timebank-high-priority-worker
|
|
sudo systemctl restart timebank-queue-worker
|
|
sudo systemctl restart timebank-mailing-worker-{1,2,3}
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Workers Not Processing Jobs
|
|
|
|
1. Check if services are running:
|
|
```bash
|
|
sudo systemctl status 'timebank-*'
|
|
```
|
|
|
|
2. Check Laravel logs:
|
|
```bash
|
|
tail -f storage/logs/laravel.log
|
|
```
|
|
|
|
3. Check systemd logs:
|
|
```bash
|
|
sudo journalctl -u timebank-mailing-worker-1 -n 50
|
|
```
|
|
|
|
### High Memory Usage
|
|
|
|
If workers consume too much memory, add memory limits to service files:
|
|
|
|
```ini
|
|
[Service]
|
|
# Restart worker after processing 1000 jobs
|
|
ExecStart=/usr/bin/php artisan queue:work --queue=mailing --tries=3 --timeout=600 --max-jobs=1000
|
|
```
|
|
|
|
### Stuck Jobs
|
|
|
|
```bash
|
|
# Clear stuck jobs
|
|
php artisan queue:flush
|
|
|
|
# Restart workers
|
|
sudo systemctl restart 'timebank-*'
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Adjust these settings in `config/timebank_cc.php`:
|
|
|
|
```php
|
|
'mailing' => [
|
|
'batch_size' => 10, // Recipients per job
|
|
'send_delay_seconds' => 5, // Delay between emails
|
|
'max_retries' => 3, // Job retry attempts
|
|
'retry_delay_minutes' => 15, // Initial retry delay
|
|
],
|
|
```
|