timebank-cc-public/references/WEBSOCKET_SEPARATE_DOMAIN.md

WebSocket Separate Domain Setup Guide

This guide explains how to configure Laravel Reverb WebSocket server on a separate subdomain (e.g., ws.yourdomain.org) instead of using the main application domain.

Why Use a Separate Domain?

Using a separate subdomain for WebSocket connections provides several benefits:

• Better organization: Clear separation between HTTP and WebSocket traffic
• Easier monitoring: Dedicated logs and metrics for WebSocket connections
• Flexible scaling: Can be hosted on different servers or load balancers
• Security isolation: Separate security policies and firewall rules
• SSL/TLS management: Independent certificate management

Prerequisites

Before starting, ensure you have:

• A working Laravel Reverb installation (see WEBSOCKET_SETUP.md)
• DNS access to create subdomains
• Apache2 with proxy modules enabled
• SSL certificate for the WebSocket subdomain (recommended for production)

Setup Steps

1. DNS Configuration

Create an A record for your WebSocket subdomain pointing to your server's IP address.

Example DNS Record:

Type: A
Name: ws
Value: 203.0.113.10 (your server IP)
TTL: 3600

This creates ws.yourdomain.org pointing to your server.

Verification:

# Test DNS resolution
dig ws.yourdomain.org

# Or use nslookup
nslookup ws.yourdomain.org

Wait for DNS propagation (usually 5-15 minutes, but can take up to 48 hours).

2. Apache Virtual Host Configuration

Create a dedicated Apache VirtualHost for the WebSocket subdomain.

Option A: HTTP Only (Development/Testing)

Create /etc/apache2/sites-available/ws-yourdomain.conf:

<VirtualHost *:80>
    ServerName ws.yourdomain.org
    ServerAdmin webmaster@yourdomain.org

    # WebSocket proxy for Reverb
    # Proxy WebSocket connections to /app/* to Reverb server
    ProxyPass /app/ ws://127.0.0.1:8080/app/
    ProxyPassReverse /app/ ws://127.0.0.1:8080/app/

    # Regular HTTP proxy for Reverb API endpoints
    ProxyPass /apps/ http://127.0.0.1:8080/apps/
    ProxyPassReverse /apps/ http://127.0.0.1:8080/apps/

    # Logging
    ErrorLog ${APACHE_LOG_DIR}/ws-error.log
    CustomLog ${APACHE_LOG_DIR}/ws-access.log combined
</VirtualHost>

Option B: HTTPS (Production - Recommended)

Create /etc/apache2/sites-available/ws-yourdomain-ssl.conf:

<VirtualHost *:80>
    ServerName ws.yourdomain.org

    # Redirect all HTTP to HTTPS
    Redirect permanent / https://ws.yourdomain.org/
</VirtualHost>

<VirtualHost *:443>
    ServerName ws.yourdomain.org
    ServerAdmin webmaster@yourdomain.org

    # SSL Configuration
    SSLEngine on
    SSLCertificateFile /etc/letsencrypt/live/ws.yourdomain.org/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/ws.yourdomain.org/privkey.pem

    # Modern SSL configuration
    SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
    SSLCipherSuite HIGH:!aNULL:!MD5
    SSLHonorCipherOrder on

    # WebSocket proxy for Reverb (wss:// for secure connections)
    ProxyPass /app/ ws://127.0.0.1:8080/app/
    ProxyPassReverse /app/ ws://127.0.0.1:8080/app/

    # Regular HTTPS proxy for Reverb API endpoints
    ProxyPass /apps/ http://127.0.0.1:8080/apps/
    ProxyPassReverse /apps/ http://127.0.0.1:8080/apps/

    # Logging
    ErrorLog ${APACHE_LOG_DIR}/ws-ssl-error.log
    CustomLog ${APACHE_LOG_DIR}/ws-ssl-access.log combined
</VirtualHost>

3. Enable Apache Modules and Site

# Enable required Apache modules (if not already enabled)
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod proxy_wstunnel
sudo a2enmod ssl  # For HTTPS

# Enable the WebSocket site
sudo a2ensite ws-yourdomain.conf
# Or for SSL:
sudo a2ensite ws-yourdomain-ssl.conf

# Test Apache configuration
sudo apache2ctl configtest

# If test is successful, reload Apache
sudo systemctl reload apache2

4. SSL Certificate Setup (Production)

For production environments, obtain an SSL certificate using Let's Encrypt:

# Install Certbot
sudo apt-get update
sudo apt-get install certbot python3-certbot-apache

# Obtain SSL certificate for WebSocket subdomain
sudo certbot --apache -d ws.yourdomain.org

# Certbot will automatically configure Apache for HTTPS
# Follow the prompts to complete setup

# Test certificate renewal
sudo certbot renew --dry-run

5. Update Laravel Environment Variables

Update your .env file to use the separate WebSocket domain:

For HTTP (Development/Testing):

PUSHER_HOST=ws.yourdomain.org
PUSHER_PORT=80
PUSHER_SCHEME=http

# Mirror for Reverb
REVERB_HOST="${PUSHER_HOST}"
REVERB_PORT="${PUSHER_PORT}"
REVERB_SCHEME="${PUSHER_SCHEME}"

# For Vite build-time variables
VITE_REVERB_APP_KEY="${PUSHER_APP_KEY}"
VITE_REVERB_HOST="${PUSHER_HOST}"
VITE_REVERB_PORT="${PUSHER_PORT}"
VITE_REVERB_SCHEME="${PUSHER_SCHEME}"

For HTTPS (Production - Recommended):

PUSHER_HOST=ws.yourdomain.org
PUSHER_PORT=443
PUSHER_SCHEME=https

# Mirror for Reverb
REVERB_HOST="${PUSHER_HOST}"
REVERB_PORT="${PUSHER_PORT}"
REVERB_SCHEME="${PUSHER_SCHEME}"

# For Vite build-time variables
VITE_REVERB_APP_KEY="${PUSHER_APP_KEY}"
VITE_REVERB_HOST="${PUSHER_HOST}"
VITE_REVERB_PORT="${PUSHER_PORT}"
VITE_REVERB_SCHEME="${PUSHER_SCHEME}"

Important Notes:

• Do NOT use variable references (e.g., "${PUSHER_HOST}") for PUSHER_HOST, PUSHER_APP_KEY, PUSHER_APP_SECRET in production
• Hard-code these values as they may be used by external services that don't expand variables
• Keep PUSHER_APP_KEY and PUSHER_APP_SECRET unchanged from your original setup

6. Rebuild Frontend Assets

After updating environment variables, you must rebuild the frontend assets to include the new WebSocket configuration:

# Clear Laravel caches
php artisan config:clear
php artisan cache:clear

# Rebuild config cache
php artisan config:cache

# Rebuild frontend assets
npm run build

7. Restart Services

Restart all relevant services to apply the changes:

# Restart Laravel Reverb
sudo systemctl restart timebank-reverb.service

# Restart queue workers (if using queued broadcasts)
php artisan queue:restart

# Verify Reverb is running
sudo systemctl status timebank-reverb.service

# Check that Reverb is listening
ss -tlnp | grep 8080

Testing the Configuration

1. Test DNS Resolution

# Verify DNS is resolving correctly
ping ws.yourdomain.org

# Check DNS record
dig ws.yourdomain.org

2. Test Apache Proxy

# Test HTTP connection
curl -I http://ws.yourdomain.org/apps/

# Test HTTPS connection (if configured)
curl -I https://ws.yourdomain.org/apps/

Expected response: HTTP headers from Reverb server

3. Test WebSocket Connection

Using wscat:

# Install wscat (if not already installed)
npm install -g wscat

# Test WebSocket connection (HTTP)
wscat -c "ws://ws.yourdomain.org/app/your-app-key?protocol=7&client=js&version=7.0.0"

# Test secure WebSocket connection (HTTPS)
wscat -c "wss://ws.yourdomain.org/app/your-app-key?protocol=7&client=js&version=7.0.0"

Successful connection shows:

Connected (press CTRL+C to quit)
< {"event":"pusher:connection_established","data":"{...}"}

4. Test from Browser Console

Open your application in a browser and check the console:

// Check Echo configuration
console.log(window.Echo);
console.log(window.Echo.connector.pusher.config);

// Test connection
Echo.connector.pusher.connection.bind('connected', () => {
    console.log('WebSocket connected to:', Echo.connector.pusher.config.wsHost);
});

Echo.connector.pusher.connection.bind('error', (err) => {
    console.error('WebSocket error:', err);
});

5. Check Apache Logs

Monitor the WebSocket logs for connection attempts:

# Follow WebSocket access logs
sudo tail -f /var/log/apache2/ws-access.log

# Follow WebSocket error logs
sudo tail -f /var/log/apache2/ws-error.log

# Or for SSL:
sudo tail -f /var/log/apache2/ws-ssl-access.log
sudo tail -f /var/log/apache2/ws-ssl-error.log

Troubleshooting

Issue: DNS Not Resolving

Symptoms: ping ws.yourdomain.org fails or returns wrong IP

Solutions:

1. Wait for DNS propagation (can take up to 48 hours)
2. Check DNS configuration at your DNS provider
3. Flush local DNS cache: sudo systemd-resolve --flush-caches
4. Try different DNS server: dig @8.8.8.8 ws.yourdomain.org

Issue: Connection Refused / 502 Bad Gateway

Symptoms: Browser shows connection error, Apache logs show "Connection refused"

Solutions:

1. Verify Reverb is running: sudo systemctl status timebank-reverb.service
2. Check Reverb is listening on port 8080: ss -tlnp | grep 8080
3. Check Apache proxy configuration: sudo apache2ctl -t -D DUMP_VHOSTS
4. Verify proxy modules are enabled: apache2ctl -M | grep proxy
5. Check firewall rules allow localhost connections: sudo ufw status

Issue: SSL Certificate Errors

Symptoms: Browser shows SSL certificate warning

Solutions:

1. Verify certificate is valid: sudo certbot certificates
2. Check certificate paths in Apache config match actual certificate location
3. Ensure certificate covers the WebSocket subdomain: openssl x509 -in /etc/letsencrypt/live/ws.yourdomain.org/cert.pem -text -noout | grep DNS
4. Reload Apache after certificate renewal: sudo systemctl reload apache2

Issue: WebSocket Connects but Closes Immediately

Symptoms: Connection establishes but closes within seconds

Solutions:

1. Check Reverb logs: sudo journalctl -u timebank-reverb.service -f
2. Verify PUSHER_APP_KEY matches in .env and frontend
3. Check Redis is running: redis-cli ping
4. Verify all environment variables are correct and frontend was rebuilt
5. Clear browser cache and hard refresh (Ctrl+Shift+R)

Issue: Mixed Content Errors (HTTP/HTTPS)

Symptoms: Console shows "Mixed Content" errors when using HTTPS site with HTTP WebSocket

Solutions:

1. Ensure WebSocket uses same scheme as main site (both HTTP or both HTTPS)
2. Update .env to use PUSHER_SCHEME=https and PUSHER_PORT=443
3. Configure SSL for WebSocket subdomain (see SSL Certificate Setup above)
4. Rebuild frontend assets: npm run build

Issue: Assets Not Updated After .env Changes

Symptoms: WebSocket still trying to connect to old domain

Solutions:

# Clear all caches
php artisan config:clear
php artisan cache:clear
php artisan view:clear

# Rebuild config cache
php artisan config:cache

# Rebuild frontend assets
npm run build

# Hard refresh browser (Ctrl+Shift+R)

Security Considerations

Firewall Configuration

Ensure your firewall allows traffic to the WebSocket subdomain:

# Allow HTTP (port 80)
sudo ufw allow 80/tcp

# Allow HTTPS (port 443)
sudo ufw allow 443/tcp

# Block direct access to Reverb from external networks
# Only allow localhost connections
sudo ufw deny 8080/tcp
sudo ufw allow from 127.0.0.1 to any port 8080

Additional Security Measures

1. Use HTTPS in production: Always use wss:// (WebSocket over SSL) for production
2. Strong credentials: Use secure random keys for PUSHER_APP_KEY and PUSHER_APP_SECRET
3. Rate limiting: Configure rate limiting in Apache or at application level
4. CORS configuration: Update allowed_origins in config/reverb.php if needed
5. Monitor logs: Regularly check WebSocket logs for suspicious activity

Content Security Policy (CSP)

Update your Content Security Policy headers to allow WebSocket connections to the subdomain:

# Add to your main application VirtualHost
Header set Content-Security-Policy "connect-src 'self' ws://ws.yourdomain.org wss://ws.yourdomain.org;"

Maintenance

Monitoring WebSocket Connections

# Count active WebSocket connections through Apache
sudo tail -f /var/log/apache2/ws-ssl-access.log | grep "GET /app/"

# Monitor Reverb process
watch -n 1 'ps aux | grep reverb'

# Monitor port usage
watch -n 1 'ss -t | grep :8080 | wc -l'

Log Rotation

Ensure log rotation is configured for WebSocket logs:

Create /etc/logrotate.d/websocket-apache:

/var/log/apache2/ws-*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        if invoke-rc.d apache2 status > /dev/null 2>&1; then \
            invoke-rc.d apache2 reload > /dev/null 2>&1; \
        fi;
    endscript
}

Quick Reference

Essential Commands

# DNS testing
dig ws.yourdomain.org
nslookup ws.yourdomain.org

# Apache configuration
sudo apache2ctl configtest
sudo systemctl reload apache2
sudo a2ensite ws-yourdomain-ssl.conf

# SSL certificate
sudo certbot --apache -d ws.yourdomain.org
sudo certbot renew --dry-run

# Service management
sudo systemctl restart timebank-reverb.service
sudo systemctl status timebank-reverb.service

# Testing WebSocket
wscat -c "wss://ws.yourdomain.org/app/your-app-key?protocol=7&client=js&version=7.0.0"

# Logs
sudo tail -f /var/log/apache2/ws-ssl-access.log
sudo tail -f /var/log/apache2/ws-ssl-error.log
sudo journalctl -u timebank-reverb.service -f

Configuration Files

• DNS: Your DNS provider's control panel
• Apache VirtualHost: /etc/apache2/sites-available/ws-yourdomain-ssl.conf
• SSL Certificate: /etc/letsencrypt/live/ws.yourdomain.org/
• Laravel .env: /path/to/your/app/.env
• Frontend config: Embedded in built assets via Vite environment variables

Additional Resources

• Main WebSocket Setup: WEBSOCKET_SETUP.md
• Laravel Reverb Documentation: https://laravel.com/docs/11.x/reverb
• Apache Proxy Guide: https://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html
• Let's Encrypt: https://letsencrypt.org/getting-started/