Initial commit

2026-03-23 21:37:59 +01:00
commit 2547717edb
2193 changed files with 972171 additions and 0 deletions
--- a/references/QUEUE_WORKERS_SETUP.md
+++ b/references/QUEUE_WORKERS_SETUP.md
@@ -0,0 +1,354 @@
+# Queue Workers Setup
+
+## Overview
+
+The application uses separate queue workers for different types of jobs:
+
+- **High Priority Worker**: Handles critical emails (password resets, email verification)
+- **Main Worker**: Handles standard emails and jobs (notifications, messages, etc.)
+- **Mailing Workers**: Dedicated workers for bulk newsletter processing (3 workers for parallel processing)
+
+## Queue Priority
+
+### High Priority Worker
+```bash
+php artisan queue:work --queue=high --tries=3 --timeout=90
+```
+
+Dedicated worker for critical operations:
+- `high` - Password resets, email verification
+- Runs independently to ensure these emails are never delayed
+- Fast timeout (90 seconds) for quick processing
+
+### Main Worker
+```bash
+php artisan queue:work --queue=messages,default,emails,low --tries=3 --timeout=90
+```
+
+Processes queues in this order:
+1. `messages` - Real-time messaging
+2. `default` - General jobs
+3. `emails` - Standard emails (notifications, contact forms, etc.)
+4. `low` - Background tasks
+
+**Note:** The `high` queue has been removed from this worker and now runs independently.
+
+### Mailing Workers
+```bash
+php artisan queue:work --queue=mailing --tries=3 --timeout=600
+```
+
+Dedicated queue for bulk newsletters:
+- `mailing` - Bulk newsletter batches
+- Runs with 3 parallel workers for faster processing
+- 10-minute timeout for large batches
+- 3 retry attempts per batch
+
+## Systemd Service Configuration
+
+### High Priority Worker Service
+
+**File:** `/etc/systemd/system/timebank-high-priority-worker.service`
+
+```ini
+[Unit]
+Description=Timebank High Priority Queue Worker (Password Resets & Email Verification)
+After=network.target
+
+[Service]
+Type=simple
+User=www-data
+Group=www-data
+WorkingDirectory=/path/to/your/timebank/installation
+ExecStart=/usr/bin/php artisan queue:work --queue=high --tries=3 --timeout=90
+Restart=always
+RestartSec=10
+
+# Environment
+Environment=PATH=/usr/bin:/bin
+Environment=TIMEBANK_CONFIG=timebank_cc
+
+# Logging
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=timebank-high-priority
+
+[Install]
+WantedBy=multi-user.target
+```
+
+### Main Worker Service
+
+**File:** `/etc/systemd/system/timebank-queue-worker.service`
+
+```ini
+[Unit]
+Description=Timebank Main Queue Worker
+After=network.target
+
+[Service]
+Type=simple
+User=www-data
+Group=www-data
+WorkingDirectory=/path/to/your/timebank/installation
+ExecStart=/usr/bin/php artisan queue:work --queue=messages,default,emails,low --tries=3 --timeout=90
+Restart=always
+RestartSec=10
+
+# Environment
+Environment=PATH=/usr/bin:/bin
+Environment=TIMEBANK_CONFIG=timebank_cc
+
+# Logging
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=timebank-queue
+    
+[Install]
+WantedBy=multi-user.target
+```
+
+### Mailing Worker Services
+
+Create 3 separate service files for parallel processing:
+
+**File 1:** `/etc/systemd/system/timebank-mailing-worker-1.service`
+**File 2:** `/etc/systemd/system/timebank-mailing-worker-2.service`
+**File 3:** `/etc/systemd/system/timebank-mailing-worker-3.service`
+
+```ini
+[Unit]
+Description=Timebank Bulk Mailing Queue Worker 1
+After=network.target
+
+[Service]
+Type=simple
+User=www-data
+Group=www-data
+WorkingDirectory=/path/to/your/timebank/installation
+ExecStart=/usr/bin/php artisan queue:work --queue=mailing --tries=3 --timeout=600
+Restart=always
+RestartSec=10
+
+# Environment
+Environment=PATH=/usr/bin:/bin
+Environment=TIMEBANK_CONFIG=timebank_cc
+
+# Logging
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=timebank-mailing-1
+
+[Install]
+WantedBy=multi-user.target
+```
+
+> **Note:** Change `timebank-mailing-1` to `timebank-mailing-2` and `timebank-mailing-3` in the other service files.
+
+## Installation Steps
+
+### 1. Create Service Files
+
+```bash
+# Create all 5 service files
+sudo nano /etc/systemd/system/timebank-high-priority-worker.service
+sudo nano /etc/systemd/system/timebank-queue-worker.service
+sudo nano /etc/systemd/system/timebank-mailing-worker-1.service
+sudo nano /etc/systemd/system/timebank-mailing-worker-2.service
+sudo nano /etc/systemd/system/timebank-mailing-worker-3.service
+```
+
+### 2. Update Paths
+
+In each service file, replace:
+- `/path/to/your/timebank/installation` with your actual installation path
+- `www-data` with your web server user if different
+- `TIMEBANK_CONFIG=timebank_cc` with your config name if different, i.e. timebank-default, uuro, etc
+
+### 3. Reload Systemd
+
+```bash
+sudo systemctl daemon-reload
+```
+
+### 4. Enable Services (start on boot)
+sign
+```bash
+# Enable high priority worker
+sudo systemctl enable timebank-high-priority-worker
+
+# Enable main worker
+sudo systemctl enable timebank-queue-worker
+
+# Enable mailing workers
+sudo systemctl enable timebank-mailing-worker-1
+sudo systemctl enable timebank-mailing-worker-2
+sudo systemctl enable timebank-mailing-worker-3
+```
+
+### 5. Start Services
+
+```bash
+# Start high priority worker
+sudo systemctl start timebank-high-priority-worker
+
+# Start main worker
+sudo systemctl start timebank-queue-worker
+
+# Start mailing workers
+sudo systemctl start timebank-mailing-worker-1
+sudo systemctl start timebank-mailing-worker-2
+sudo systemctl start timebank-mailing-worker-3
+```
+
+## Management Commands
+
+### Check Status
+
+```bash
+# Check all workers
+sudo systemctl status timebank-high-priority-worker
+sudo systemctl status timebank-queue-worker
+sudo systemctl status timebank-mailing-worker-1
+sudo systemctl status timebank-mailing-worker-2
+sudo systemctl status timebank-mailing-worker-3
+
+# Or check all at once
+sudo systemctl status 'timebank-*'
+```
+
+### Stop Workers
+
+```bash
+# Stop specific worker
+sudo systemctl stop timebank-mailing-worker-1
+
+# Stop all mailing workers
+sudo systemctl stop timebank-mailing-worker-{1,2,3}
+
+# Stop all workers
+sudo systemctl stop 'timebank-*'
+```
+
+### Restart Workers
+
+```bash
+# Restart after code deployment
+sudo systemctl restart timebank-high-priority-worker
+sudo systemctl restart timebank-queue-worker
+sudo systemctl restart timebank-mailing-worker-{1,2,3}
+```
+
+### View Logs
+
+```bash
+# View high priority worker logs
+sudo journalctl -u timebank-high-priority-worker -f
+
+# View main worker logs
+sudo journalctl -u timebank-queue-worker -f
+
+# View mailing worker logs
+sudo journalctl -u timebank-mailing-worker-1 -f
+
+# View all worker logs
+sudo journalctl -u 'timebank-*' -f
+
+# View last 100 lines
+sudo journalctl -u timebank-queue-worker -n 100
+```
+
+## Performance
+
+### With This Setup
+
+- **Critical Emails**: Processed immediately by dedicated high priority worker
+  - Password resets: Sent within seconds
+  - Email verification: Sent within seconds
+  - Never blocked by other jobs
+
+- **Standard Emails**: Processed by main worker
+  - Notifications, messages, contact forms
+  - Fast processing without bulk mailing delays
+
+- **Bulk Newsletters**: Processed by 3 parallel workers
+  - ~36 emails/minute (with 5 sec delay)
+  - ~180 emails/minute (with 1 sec delay)
+  - 2,182 emails complete in ~60 minutes (5 sec delay) or ~12 minutes (1 sec delay)
+  - Completely isolated from critical and standard emails
+
+### Monitoring
+
+```bash
+# Watch queue in real-time
+php artisan queue:monitor
+
+# Check failed jobs
+php artisan queue:failed
+
+# Retry failed jobs
+php artisan queue:retry all
+```
+
+## Deployment Integration
+
+Add to your deployment script after code updates:
+
+```bash
+# Restart all queue workers after deployment
+sudo systemctl restart timebank-high-priority-worker
+sudo systemctl restart timebank-queue-worker
+sudo systemctl restart timebank-mailing-worker-{1,2,3}
+```
+
+## Troubleshooting
+
+### Workers Not Processing Jobs
+
+1. Check if services are running:
+   ```bash
+   sudo systemctl status 'timebank-*'
+   ```
+
+2. Check Laravel logs:
+   ```bash
+   tail -f storage/logs/laravel.log
+   ```
+
+3. Check systemd logs:
+   ```bash
+   sudo journalctl -u timebank-mailing-worker-1 -n 50
+   ```
+
+### High Memory Usage
+
+If workers consume too much memory, add memory limits to service files:
+
+```ini
+[Service]
+# Restart worker after processing 1000 jobs
+ExecStart=/usr/bin/php artisan queue:work --queue=mailing --tries=3 --timeout=600 --max-jobs=1000
+```
+
+### Stuck Jobs
+
+```bash
+# Clear stuck jobs
+php artisan queue:flush
+
+# Restart workers
+sudo systemctl restart 'timebank-*'
+```
+
+## Configuration
+
+Adjust these settings in `config/timebank_cc.php`:
+
+```php
+'mailing' => [
+    'batch_size' => 10,              // Recipients per job
+    'send_delay_seconds' => 5,       // Delay between emails
+    'max_retries' => 3,              // Job retry attempts
+    'retry_delay_minutes' => 15,     // Initial retry delay
+],
+```