timebank-cc-public/scripts/BACKUP_GUIDE.md

Laravel Timebank Backup & Restore Guide

This guide provides comprehensive instructions for backing up and restoring your Laravel Timebank application data, including database and storage files.

Table of Contents

1. Overview
2. Quick Start
3. Installation & Setup
4. Backup Scripts
5. Restore Scripts
6. Automation Setup
7. Monitoring & Maintenance
8. Troubleshooting
9. Best Practices

Overview

The backup system consists of several components:

• Database Backup: Compressed MySQL dumps with rotation
• Storage Backup: Incremental rsync-based file backups
• Automated Scheduling: Cron-based periodic backups
• Retention Management: Automatic cleanup of old backups
• Verification: Backup integrity checks
• Email Notifications: Success/failure notifications for all operations
• Restore Tools: Simple restoration procedures

Backup Types

• Daily: 7 days retention, incremental storage backups
• Weekly: 4 weeks retention, full storage backups
• Monthly: 12 months retention, complete archives
• Full: On-demand complete backups

Quick Start

Perform a Manual Backup

# Complete backup (database + storage)
./scripts/backup-all.sh daily

# Database only
./scripts/backup-database.sh daily

# Storage only
./scripts/backup-storage.sh daily

# With verification and notifications
./scripts/backup-all.sh daily --verify --notify

List Available Backups

# List database backups
./scripts/restore-database.sh --list-backups

# List storage backups
./scripts/restore-storage.sh --list-backups

Restore from Latest Backup

# Restore database from latest backup
./scripts/restore-database.sh --latest

# Restore storage from latest backup
./scripts/restore-storage.sh --latest

# Merge storage (don't replace existing files)
./scripts/restore-storage.sh --latest --merge

Configure Retention (Optional)

Customize how long backups are kept:

# View current settings
./scripts/cleanup-backups.sh --help

# Edit retention policy  
nano scripts/backup-retention.conf

Quick settings: Change DAILY_RETENTION, MONTHLY_RETENTION, or disk thresholds to suit your storage needs.

Installation & Setup

1. Verify Prerequisites

Ensure the following are installed on your system:

# Required tools
which mysqldump mysql rsync tar gzip

2. Configure Environment

Your .env file must contain proper database credentials:

DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=timebank_cc
DB_USERNAME=your_username
DB_PASSWORD=your_password

3. Configure Retention Policy (Optional)

The backup system uses default retention settings, but you can customize them:

# View current retention settings
./scripts/cleanup-backups.sh --help

# Edit retention configuration (optional)
nano scripts/backup-retention.conf

# Common adjustments:
# - DAILY_RETENTION=14     # Keep daily backups for 2 weeks instead of 1
# - MONTHLY_RETENTION=730  # Keep monthly backups for 2 years instead of 1
# - DISK_WARNING_THRESHOLD=75  # Earlier disk space warnings

Quick retention examples:

• More storage space: Increase DAILY_RETENTION=14, MONTHLY_RETENTION=730
• Less storage space: Decrease DAILY_COUNT_LIMIT=3, MONTHLY_COUNT_LIMIT=6
• Earlier disk warnings: Set DISK_WARNING_THRESHOLD=75

4. Set Permissions

Ensure backup scripts are executable:

chmod +x scripts/*.sh

4. Test Backup

Run a test backup to verify everything works:

./scripts/backup-all.sh daily --verify

5. Check Backup Directory

Verify backups are created:

ls -la backups/

Backup Scripts

backup-database.sh

Creates compressed MySQL database backups.

Usage:

./scripts/backup-database.sh [daily|weekly|monthly]

Features:

• Compressed gzip output
• Single-transaction consistency
• Includes routines, triggers, events
• Automatic retention cleanup
• Backup verification
• Email notifications on success

Example:

# Daily database backup
./scripts/backup-database.sh daily

# Monthly database backup
./scripts/backup-database.sh monthly

backup-storage.sh

Creates compressed archives of the storage directory.

Usage:

./scripts/backup-storage.sh [daily|weekly|monthly|full]

Features:

• Incremental backups using rsync
• Hard links for space efficiency
• Excludes cache and temporary files
• Full backup option available
• Compressed tar.gz output
• Email notifications on success

Example:

# Incremental storage backup
./scripts/backup-storage.sh daily

# Full storage backup
./scripts/backup-storage.sh full

backup-all.sh

Master orchestration script for complete backups.

Usage:

./scripts/backup-all.sh [daily|weekly|monthly] [options]

Options:

• --storage-only: Backup only storage files
• --database-only: Backup only database
• --verify: Verify backups after creation
• --notify: Send notifications on completion

Example:

# Complete backup with verification
./scripts/backup-all.sh weekly --verify --notify

# Storage only backup
./scripts/backup-all.sh daily --storage-only

Restore Scripts

restore-database.sh

Restores database from backup files.

Usage:

./scripts/restore-database.sh [backup_file] [options]

Options:

• --confirm: Skip confirmation prompt
• --list-backups: List available backups
• --latest: Restore from latest backup

Examples:

# List available backups
./scripts/restore-database.sh --list-backups

# Restore latest backup
./scripts/restore-database.sh --latest

# Restore specific backup (full path)
./scripts/restore-database.sh backups/database/daily/timebank_daily_20240101.sql.gz

# Restore specific backup (filename only - auto-resolves path)
./scripts/restore-database.sh timebank_daily_20240101.sql.gz

Safety Features:

• Creates pre-restore backup
• Confirmation prompts
• Backup verification
• Detailed logging
• Email notifications on success
• Automatic path resolution

restore-storage.sh

Restores storage files from backup archives.

Usage:

./scripts/restore-storage.sh [backup_file] [options]

Options:

• --confirm: Skip confirmation prompt
• --list-backups: List available backups
• --latest: Restore from latest backup
• --merge: Merge with existing files (don't replace)

Examples:

# Restore latest storage backup
./scripts/restore-storage.sh --latest

# Merge latest backup with existing files
./scripts/restore-storage.sh --latest --merge

# Restore specific backup (full path)
./scripts/restore-storage.sh backups/storage/weekly/weekly_20240101.tar.gz

# Restore specific backup (filename only - auto-resolves path)
./scripts/restore-storage.sh weekly_20240101.tar.gz

restore-all.sh

Complete system restoration script for both database and storage.

Usage:

./scripts/restore-all.sh [options]

Options:

• --latest: Restore from latest backups (both database and storage)
• --database-file FILE: Specify database backup file
• --storage-file FILE: Specify storage backup file
• --database-latest: Use latest database backup only
• --storage-latest: Use latest storage backup only
• --confirm: Skip confirmation prompts
• --list-backups: List available backups

Examples:

# Restore both database and storage from latest backups
./scripts/restore-all.sh --latest

# Restore specific files
./scripts/restore-all.sh --database-file timebank_daily_20240101.sql.gz --storage-file daily_20240101.tar.gz

# Database only restore
./scripts/restore-all.sh --database-latest

# List all available backups
./scripts/restore-all.sh --list-backups

Features:

• Orchestrates complete system restoration
• Handles both database and storage restoration
• Automatic path resolution for backup files
• Pre-restore backups for safety
• Email notifications on success
• Post-restore Laravel optimization

Automation Setup

Cron Configuration

Copy and customize the cron configuration:

# Copy template
sudo cp scripts/cron-backup.conf /etc/cron.d/timebank-backup

# Edit paths and email addresses
sudo nano /etc/cron.d/timebank-backup

# Verify cron syntax
sudo cron -T

Alternative: User Crontab

For non-root installations:

# Edit user crontab
crontab -e

# Add backup schedules (example)
0 2 * * * cd /path/to/timebank && ./scripts/backup-all.sh daily --verify
0 3 * * 0 cd /path/to/timebank && ./scripts/backup-all.sh weekly --verify
0 4 1 * * cd /path/to/timebank && ./scripts/backup-all.sh monthly --verify

Email Notification Setup

All backup and restore scripts automatically send email notifications on successful completion.

Prerequisites

Install mail utilities:

# Ubuntu/Debian
sudo apt install mailutils postfix

# Configure postfix for "Local only" delivery during setup

Email Configuration

By default, notifications are sent to $USER@localhost. To customize:

# Set custom email address (optional)
export BACKUP_NOTIFY_EMAIL="admin@yourdomain.org"

Email Subjects and Content

Backup Notifications:

• Subject: "Timebank DB Backup Success"
• Subject: "Timebank Storage Backup Success"
• Content: Includes timestamp and backup details

Restore Notifications:

• Subject: "Timebank DB Restore Success"
• Subject: "Timebank Storage Restore Success"
• Subject: "Timebank Complete Restore Success"
• Content: Includes timestamp and restored file information

Checking Local Mail

View received notifications:

# Open mail client
mail

# List messages
mail -H

# Quick check for new messages
ls -la /var/mail/$USER

Webhook Notifications (Optional)

For additional notification methods:

# Webhook notifications
export BACKUP_WEBHOOK_URL="https://hooks.slack.com/your-webhook"

Monitoring & Maintenance

cleanup-backups.sh

Manages backup retention and disk space.

Usage:

./scripts/cleanup-backups.sh [options]

Options:

• --dry-run: Show what would be deleted
• --force: Force cleanup regardless of disk space
• --verbose: Show detailed information

Features:

• Automatic retention policy enforcement
• Disk space monitoring
• Empty directory cleanup
• Detailed reporting

Example:

# Dry run to see what would be cleaned up
./scripts/cleanup-backups.sh --dry-run --verbose

# Force cleanup
./scripts/cleanup-backups.sh --force

Retention Policy Configuration

The cleanup script uses a configurable retention policy system that allows you to customize backup retention settings.

Configuration File

Location: scripts/backup-retention.conf

The configuration file controls:

• Time-based retention: How long to keep backups (in days)
• Count-based retention: How many recent backups to keep
• Disk space thresholds: When to trigger cleanup
• Email notifications: Enable/disable cleanup emails
• Advanced settings: Cleanup modes and verification options

Configuration Options

Time-based Retention (days):

DAILY_RETENTION=7          # Keep daily backups for 7 days
WEEKLY_RETENTION=28        # Keep weekly backups for 28 days  
MONTHLY_RETENTION=365      # Keep monthly backups for 365 days
PRE_RESTORE_RETENTION=30   # Keep pre-restore backups for 30 days
LOG_RETENTION=30           # Keep log files for 30 days

Count-based Retention (number of files):

DAILY_COUNT_LIMIT=7        # Keep 7 most recent daily backups
WEEKLY_COUNT_LIMIT=4       # Keep 4 most recent weekly backups
MONTHLY_COUNT_LIMIT=12     # Keep 12 most recent monthly backups
PRE_RESTORE_COUNT_LIMIT=5  # Keep 5 most recent pre-restore backups
SNAPSHOT_COUNT_LIMIT=3     # Keep 3 most recent storage snapshots

Disk Space Management:

DISK_WARNING_THRESHOLD=85  # Send warning at 85% disk usage
DISK_CRITICAL_THRESHOLD=95 # Force cleanup at 95% disk usage

Notification Settings:

EMAIL_NOTIFICATIONS_ENABLED=true  # Enable/disable email notifications
BACKUP_NOTIFY_EMAIL=admin@domain.org  # Custom email address (optional)

Customizing Retention Policies

View Current Configuration:

./scripts/cleanup-backups.sh --help

Edit Configuration:

# Edit the config file
nano scripts/backup-retention.conf

# Test changes with dry run
./scripts/cleanup-backups.sh --dry-run --verbose

Configuration Examples

Conservative (longer retention):

DAILY_RETENTION=14         # 2 weeks
WEEKLY_RETENTION=56        # 8 weeks  
MONTHLY_RETENTION=730      # 2 years
DAILY_COUNT_LIMIT=14       # More daily backups
MONTHLY_COUNT_LIMIT=24     # 2 years of monthly backups

Aggressive (shorter retention):

DAILY_RETENTION=3          # 3 days only
WEEKLY_RETENTION=14        # 2 weeks
MONTHLY_RETENTION=180      # 6 months
DAILY_COUNT_LIMIT=3        # Fewer daily backups
DISK_WARNING_THRESHOLD=75  # Earlier warning

Space-constrained environment:

DAILY_COUNT_LIMIT=3        # Keep only 3 daily backups
WEEKLY_COUNT_LIMIT=2       # Keep only 2 weekly backups
MONTHLY_COUNT_LIMIT=6      # Keep only 6 monthly backups
DISK_WARNING_THRESHOLD=70  # Early disk space warnings
DISK_CRITICAL_THRESHOLD=85 # More aggressive cleanup threshold

Validation and Safety

The configuration system includes:

• Input validation: Invalid values fall back to defaults
• Range checking: Values must be within reasonable ranges
• Relationship validation: Critical threshold must be higher than warning
• Fallback system: Uses defaults if config file is missing or corrupted

Advanced Settings

# Cleanup behavior
CLEANUP_MODE=both              # Options: age_only, count_only, both
CLEANUP_EMPTY_DIRS=true        # Remove empty directories after cleanup
VERIFY_BEFORE_DELETE=false     # Verify backup integrity before deletion

# Email control
EMAIL_NOTIFICATIONS_ENABLED=true  # Master switch for email notifications

Health Monitoring

The backup system includes health checks:

# View health check results
cat backups/health_check.json

# Check backup logs
tail -f backups/backup.log

# View backup summary
./scripts/backup-all.sh daily --verify

Log Management

Backup logs are stored in:

• backups/backup.log - Main backup log
• backups/restore.log - Restore operations log
• /var/log/timebank-backup.log - System cron log

Rotate logs using logrotate:

# Create logrotate configuration
sudo tee /etc/logrotate.d/timebank-backup <<EOF
/var/log/timebank-backup.log {
    daily
    missingok
    rotate 30
    compress
    delaycompress
    notifempty
    create 644 root root
}
EOF

Troubleshooting

Common Issues

Permission Denied

# Fix script permissions
chmod +x scripts/*.sh

# Fix backup directory permissions
sudo chown -R $(whoami):$(whoami) backups/

MySQL Connection Issues

# Test database connection
mysql -h$DB_HOST -P$DB_PORT -u$DB_USERNAME -p$DB_PASSWORD -e "SHOW DATABASES;"

# Check .env file
grep "^DB_" .env

Disk Space Issues

# Check available space
df -h backups/

# Run cleanup
./scripts/cleanup-backups.sh --force

# Check large files
find backups/ -type f -size +100M -exec ls -lh {} \;

Backup Verification Failures

# Test gzip integrity
gzip -t backups/database/daily/*.sql.gz

# Test tar integrity
tar -tzf backups/storage/daily/*.tar.gz >/dev/null

Recovery Procedures

Complete System Recovery

1. Prepare clean environment:

   # Install Laravel dependencies
   composer install --no-dev --optimize-autoloader
   npm install && npm run build
   

2. Restore database:

   ./scripts/restore-database.sh --latest --confirm
   

3. Restore storage:

   ./scripts/restore-storage.sh --latest --confirm
   

4. Post-restore steps:

   php artisan storage:link
   php artisan config:clear
   php artisan cache:clear
   php artisan migrate:status
   

Partial Recovery

Database only:

./scripts/restore-database.sh --latest
php artisan migrate:status

Storage only:

./scripts/restore-storage.sh --latest --merge
php artisan storage:link

Best Practices

Security

1. Protect backup files:

   chmod 600 backups/database/**/*.sql.gz
   chmod 600 backups/storage/**/*.tar.gz
   

2. Use secure file transfer:

   # Example: Upload to secure cloud storage
   rclone sync backups/ remote:timebank-backups/
   

3. Encrypt sensitive backups:

   # Example: GPG encryption
   gpg --symmetric --cipher-algo AES256 backup.sql.gz
   

Performance

1. Schedule during low-traffic periods
2. Monitor backup duration and adjust frequency
3. Use fast storage for backup directory
4. Consider incremental-only backups for large storage

Reliability

1. Test restores regularly:

   # Monthly restore test
   ./scripts/restore-database.sh --latest --confirm
   

2. Monitor backup success:

   # Check recent backup status
   find backups/ -name "*.gz" -mtime -1 -ls
   

3. Verify backup integrity:

   # Run verification on all recent backups
   ./scripts/backup-all.sh daily --verify
   

4. Keep multiple backup locations:

   • Local backups for quick recovery
   • Remote backups for disaster recovery
   • Cloud storage for long-term retention

Documentation

1. Keep this guide updated
2. Document any customizations
3. Maintain recovery contact information
4. Document environment-specific configurations

---

Support

For issues or questions regarding the backup system:

1. Check the troubleshooting section above
2. Review backup logs: tail -f backups/backup.log
3. Test with --dry-run options first
4. Ensure all prerequisites are installed

Remember: Always test your restore procedures in a non-production environment before relying on them in production!