ronald/timebank-cc-public

0 Fork 0

Files

T

Ronald Huynen 6ccf1792d7 Snapshot of main at 9f39a6e1e (2026-11-02)

2026-11-02 10:41:47 +01:00

9.7 KiB

Raw Blame History

Universal Email Bounce Handling System

Overview

This system provides universal bounce handling for ALL mailables that works with any SMTP server by:

Automatically intercepting all outgoing emails via Laravel's MessageSending event
Checking for suppressed recipients before emails are sent
Adding bounce tracking headers to all outgoing emails
Processing bounce emails from a dedicated mailbox
Using configurable thresholds to suppress emails and reset verification status
Providing conservative bounce counting to prevent false positives

🎯 Key Feature: This system works automatically with ALL existing and future mailables without requiring code changes!

Setup Steps

1. Configure Bounce Email Address

Add these to your .env file:

# Bounce handling configuration
BOUNCE_PROCESSING_ENABLED=true  # Set to false on local/staging environments without IMAP
MAIL_BOUNCE_ADDRESS=bounces@yourdomain.org
BOUNCE_MAILBOX=bounces@yourdomain.org
BOUNCE_HOST=imap.yourdomain.org
BOUNCE_PORT=993
BOUNCE_PROTOCOL=imap
BOUNCE_USERNAME=bounces@yourdomain.org
BOUNCE_PASSWORD=your-bounce-mailbox-password
BOUNCE_SSL=true

Important: Set BOUNCE_PROCESSING_ENABLED=false on local development and staging environments that don't have access to the bounce mailbox to prevent IMAP connection errors.

2. Create Bounce Email Address

Create a dedicated email address (e.g., bounces@yourdomain.org) that will receive bounce notifications:

Set up the email account on your email server
Configure IMAP access for programmatic reading

3. Configure Your SMTP Server

Most SMTP servers will respect the Return-Path header and send bounces to that address automatically.

4. Process Bounces

Run the bounce processing command periodically:

# Process bounces (dry run first to test)
php artisan mailings:process-bounces --dry-run

# Process bounces for real
php artisan mailings:process-bounces --delete

# Or use command options instead of config file
php artisan mailings:process-bounces \
    --mailbox=bounces@yourdomain.org \
    --host=imap.yourdomain.org \
    --username=bounces@yourdomain.org \
    --password=your-password \
    --ssl \
    --delete

5. Schedule Automatic Processing

Add to your app/Console/Kernel.php:

protected function schedule(Schedule $schedule)
{
    // Process bounces every hour
    $schedule->command('mailings:process-bounces --delete')
             ->hourly()
             ->withoutOverlapping();
}

Threshold Configuration

Configure bounce thresholds in config/timebank-cc.php:

'bounce_thresholds' => [
    // Number of hard bounces before email is suppressed from future mailings
    'suppression_threshold' => 3,

    // Number of hard bounces before email_verified_at is set to null
    'verification_reset_threshold' => 2,

    // Time window in days to count bounces (prevents old bounces from accumulating)
    'counting_window_days' => 30,

    // Only count these bounce types toward thresholds
    'counted_bounce_types' => ['hard'],

    // Specific bounce reasons that count as definitive hard bounces
    'definitive_hard_bounce_patterns' => [
        'user unknown', 'no such user', 'mailbox unavailable',
        'does not exist', 'invalid recipient', 'address rejected',
        '5.1.1', '5.1.2', '5.1.3', '550', '551',
    ],
]

Universal System Architecture

Automatic Email Interception

The system uses Laravel's MessageSending event to automatically:

Intercept ALL outgoing emails from any mailable class
Check recipients against the suppression list
Block emails to suppressed addresses (logs the action)
Add bounce tracking headers to all outgoing emails
Remove suppressed recipients from multi-recipient emails

No Code Changes Required

Works with existing mailables: ContactFormMailable, TransferReceived, NewMessageMail, etc.
Works with future mailables automatically
Works with Mail::to(), Mail::queue(), and Mail::later() methods
Works with both sync and queued emails

Enhanced Mailables (Optional)

For enhanced bounce tracking, mailables can optionally:

Extend BounceTrackingMailable base class
Use TracksBounces trait for additional tracking features

Commands Available

Test Universal System

# Test with normal (non-suppressed) email
php artisan test:universal-bounce --scenario=normal --email=test@example.com

# Test with suppressed email (should be blocked)
php artisan test:universal-bounce --scenario=suppressed --email=test@example.com

# Test with mixed recipients
php artisan test:universal-bounce --scenario=mixed

Process Bounce Emails

php artisan mailings:process-bounces [options]

Manage Bounced Emails

# Show comprehensive bounce statistics with threshold info
php artisan mailings:manage-bounces stats

# List bounced emails
php artisan mailings:manage-bounces list

# Check bounce counts for a specific email
php artisan mailings:manage-bounces check-thresholds --email=user@example.com

# Check all emails against current thresholds
php artisan mailings:manage-bounces check-thresholds

# Suppress a specific email
php artisan mailings:manage-bounces suppress --email=problem@example.com

# Unsuppress an email
php artisan mailings:manage-bounces unsuppress --email=fixed@example.com

# Cleanup old soft bounces (older than 90 days)
php artisan mailings:manage-bounces cleanup --days=90

How It Works

Outgoing Emails: Each newsletter email gets:
- Return-Path header set to your bounce address
- X-Mailing-ID header for tracking
- X-Recipient-Email header for identification
Bounce Detection: When an email bounces:
- SMTP server sends bounce notification to Return-Path address
- Bounce processor reads the dedicated mailbox
- Extracts recipient email and bounce type from bounce message
- Records bounce in database
Threshold-Based Actions:
- System counts definitive hard bounces within a time window (default: 30 days)
- After verification_reset_threshold hard bounces: email_verified_at is set to null for all profiles
- After suppression_threshold hard bounces: Email is suppressed from future mailings
- Only specific bounce patterns count toward thresholds (prevents false positives)
Conservative Approach:
- Only "definitive" hard bounces count (user unknown, domain invalid, etc.)
- Time window prevents old bounces from accumulating indefinitely
- Configurable thresholds allow fine-tuning for your use case
Integration:
- SendBulkMailJob checks for suppressed emails before sending
- Mailing.getRecipientsQuery() excludes suppressed emails
- Bounce detection works with any SMTP server
- Multi-profile support: affects Users, Organizations, Banks, and Admins

Bounce Types

Hard Bounce: Permanent delivery failure (user doesn't exist, domain invalid)
- Counts toward suppression and verification reset thresholds
- Only "definitive" patterns count (configured in bounce_thresholds)
Soft Bounce: Temporary failure (mailbox full, server temporarily unavailable)
- Recorded but doesn't count toward thresholds
- Not automatically suppressed
Unknown: Could not determine bounce type from message content
- Recorded but doesn't count toward thresholds

Threshold Benefits

Prevents False Positives: Temporary server issues won't immediately suppress emails
Gradual Response: Reset verification before full suppression
Time-Based: Old bounces don't accumulate indefinitely
Conservative: Only definitive bounce patterns count
Configurable: Adjust thresholds for your sending patterns

Testing

Testing with `bounce:simulate`

bounce:simulate is the single testing command for all environments. It auto-detects the environment and works in two modes:

Inject mode — local dev, no mailbox needed. Injects a fake bounce directly into the pipeline:

php artisan bounce:simulate --email=user@example.com --status   # check baseline
php artisan bounce:simulate --email=user@example.com            # record a hard bounce
php artisan bounce:simulate --email=user@example.com --type=soft  # record a soft bounce
php artisan bounce:simulate --email=user@example.com --clear    # reset records
php artisan mailings:manage-bounces stats                       # inspect results

IMAP mode — production. Reads real DSN messages from the bounce mailbox in .env. Skips gracefully if credentials are not configured:

php artisan bounce:simulate --imap --dry-run    # verify connection and parsing, record nothing
php artisan bounce:simulate --imap --delete     # process, record, and clear mailbox
php artisan mailings:manage-bounces stats       # inspect results

See references/BOUNCE_SIMULATE_COMMAND.md for full usage, options, and example output.

Production Testing

Set up a test bounce mailbox
Send a test mailing to a non-existent address
Check that bounce appears in the bounce mailbox
Run php artisan mailings:process-bounces --dry-run to test parsing
Verify the bounce is correctly detected and categorized

What to Expect

Exact behaviour depends on mailing.bounce_thresholds in config/timebank_cc.php. Check suppression_threshold and verification_reset_threshold for current values. Run bounce:simulate --email=X --status to see the live thresholds alongside the address state.

Email Sending: Suppressed emails are automatically skipped during bulk sends

Troubleshooting

Check IMAP/POP3 credentials and server settings
Verify Return-Path is being set correctly on outgoing emails
Test bounce mailbox connection manually
Check Laravel logs for bounce processing errors
Use --dry-run flag to test without making changes

9.7 KiB Raw Blame History