ronald/timebank-cc-public
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Snapshot of main at 9f39a6e1e (2026-11-02)

Browse Source
This commit is contained in:
Ronald Huynen committed 2026-11-02 10:41:47 +01:00
commit 6ccf1792d7
2580 files changed
+1040202
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

scripts/BACKUP_GUIDE.md
+790
View File
@@ -0,0 +1,790 @@
# Laravel Timebank Backup & Restore Guide
This guide provides comprehensive instructions for backing up and restoring your Laravel Timebank application data, including database and storage files.
## Table of Contents
1. [Overview](#overview)
2. [Quick Start](#quick-start)
3. [Installation & Setup](#installation--setup)
4. [Backup Scripts](#backup-scripts)
5. [Restore Scripts](#restore-scripts)
6. [Automation Setup](#automation-setup)
7. [Monitoring & Maintenance](#monitoring--maintenance)
8. [Troubleshooting](#troubleshooting)
9. [Best Practices](#best-practices)
## Overview
The backup system consists of several components:
- **Database Backup**: Compressed MySQL dumps with rotation
- **Storage Backup**: Incremental rsync-based file backups
- **Automated Scheduling**: Cron-based periodic backups
- **Retention Management**: Automatic cleanup of old backups
- **Verification**: Backup integrity checks
- **Email Notifications**: Success/failure notifications for all operations
- **Restore Tools**: Simple restoration procedures
### Backup Types
- **Daily**: 7 days retention, incremental storage backups
- **Weekly**: 4 weeks retention, full storage backups
- **Monthly**: 12 months retention, complete archives
- **Full**: On-demand complete backups
## Quick Start
### Perform a Manual Backup
```bash
# Complete backup (database + storage)
./scripts/backup-all.sh daily
# Database only
./scripts/backup-database.sh daily
# Storage only
./scripts/backup-storage.sh daily
# With verification and notifications
./scripts/backup-all.sh daily --verify --notify
```
### List Available Backups
```bash
# List database backups
./scripts/restore-database.sh --list-backups
# List storage backups
./scripts/restore-storage.sh --list-backups
```
### Restore from Latest Backup
```bash
# Restore database from latest backup
./scripts/restore-database.sh --latest
# Restore storage from latest backup
./scripts/restore-storage.sh --latest
# Merge storage (don't replace existing files)
./scripts/restore-storage.sh --latest --merge
```
### Configure Retention (Optional)
Customize how long backups are kept:
```bash
# View current settings
./scripts/cleanup-backups.sh --help
# Edit retention policy
nano scripts/backup-retention.conf
```
**Quick settings:** Change DAILY_RETENTION, MONTHLY_RETENTION, or disk thresholds to suit your storage needs.
## Installation & Setup
### 1. Verify Prerequisites
Ensure the following are installed on your system:
```bash
# Required tools
which mysqldump mysql rsync tar gzip
```
### 2. Configure Environment
Your `.env` file must contain proper database credentials:
```env
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=timebank_cc
DB_USERNAME=your_username
DB_PASSWORD=your_password
```
### 3. Configure Retention Policy (Optional)
The backup system uses default retention settings, but you can customize them:
```bash
# View current retention settings
./scripts/cleanup-backups.sh --help
# Edit retention configuration (optional)
nano scripts/backup-retention.conf
# Common adjustments:
# - DAILY_RETENTION=14 # Keep daily backups for 2 weeks instead of 1
# - MONTHLY_RETENTION=730 # Keep monthly backups for 2 years instead of 1
# - DISK_WARNING_THRESHOLD=75 # Earlier disk space warnings
```
**Quick retention examples:**
- **More storage space**: Increase DAILY_RETENTION=14, MONTHLY_RETENTION=730
- **Less storage space**: Decrease DAILY_COUNT_LIMIT=3, MONTHLY_COUNT_LIMIT=6
- **Earlier disk warnings**: Set DISK_WARNING_THRESHOLD=75
### 4. Set Permissions
Ensure backup scripts are executable:
```bash
chmod +x scripts/*.sh
```
### 4. Test Backup
Run a test backup to verify everything works:
```bash
./scripts/backup-all.sh daily --verify
```
### 5. Check Backup Directory
Verify backups are created:
```bash
ls -la backups/
```
## Backup Scripts
### backup-database.sh
Creates compressed MySQL database backups.
**Usage:**
```bash
./scripts/backup-database.sh [daily|weekly|monthly]
```
**Features:**
- Compressed gzip output
- Single-transaction consistency
- Includes routines, triggers, events
- Automatic retention cleanup
- Backup verification
- Email notifications on success
**Example:**
```bash
# Daily database backup
./scripts/backup-database.sh daily
# Monthly database backup
./scripts/backup-database.sh monthly
```
### backup-storage.sh
Creates compressed archives of the storage directory.
**Usage:**
```bash
./scripts/backup-storage.sh [daily|weekly|monthly|full]
```
**Features:**
- Incremental backups using rsync
- Hard links for space efficiency
- Excludes cache and temporary files
- Full backup option available
- Compressed tar.gz output
- Email notifications on success
- Also archives the platform folder (`platform/`, or `PLATFORM_PATH` in `.env`) when it exists, as
`backups/platform/<type>/platform_<type>_<timestamp>.tar.gz`, with the same retention
**Restoring the platform folder** (it holds the platform's config, theme, text and template overrides and
public files; see `references/MULTI_PLATFORM_INSTANCES.md`, section 3):
```bash
mv platform platform.before-restore # keep the current one aside
tar -xzf backups/platform/daily/platform_daily_<timestamp>.tar.gz -C .
php artisan platform:link && php artisan view:clear # then restart the queue workers and Reverb
php artisan platform:check
```
These backups stay on the server. Keep the folder in a private git repository of its own as well, for an
off-site copy.
**Example:**
```bash
# Incremental storage backup
./scripts/backup-storage.sh daily
# Full storage backup
./scripts/backup-storage.sh full
```
### backup-all.sh
Master orchestration script for complete backups.
**Usage:**
```bash
./scripts/backup-all.sh [daily|weekly|monthly] [options]
```
**Options:**
- `--storage-only`: Backup only storage files
- `--database-only`: Backup only database
- `--verify`: Verify backups after creation
- `--notify`: Send notifications on completion
**Example:**
```bash
# Complete backup with verification
./scripts/backup-all.sh weekly --verify --notify
# Storage only backup
./scripts/backup-all.sh daily --storage-only
```
## Restore Scripts
### restore-database.sh
Restores database from backup files.
**Usage:**
```bash
./scripts/restore-database.sh [backup_file] [options]
```
**Options:**
- `--confirm`: Skip confirmation prompt
- `--list-backups`: List available backups
- `--latest`: Restore from latest backup
**Examples:**
```bash
# List available backups
./scripts/restore-database.sh --list-backups
# Restore latest backup
./scripts/restore-database.sh --latest
# Restore specific backup (full path)
./scripts/restore-database.sh backups/database/daily/timebank_daily_20240101.sql.gz
# Restore specific backup (filename only - auto-resolves path)
./scripts/restore-database.sh timebank_daily_20240101.sql.gz
```
**Safety Features:**
- Creates pre-restore backup
- Confirmation prompts
- Backup verification
- Detailed logging
- Email notifications on success
- Automatic path resolution
### restore-storage.sh
Restores storage files from backup archives.
**Usage:**
```bash
./scripts/restore-storage.sh [backup_file] [options]
```
**Options:**
- `--confirm`: Skip confirmation prompt
- `--list-backups`: List available backups
- `--latest`: Restore from latest backup
- `--merge`: Merge with existing files (don't replace)
**Examples:**
```bash
# Restore latest storage backup
./scripts/restore-storage.sh --latest
# Merge latest backup with existing files
./scripts/restore-storage.sh --latest --merge
# Restore specific backup (full path)
./scripts/restore-storage.sh backups/storage/weekly/weekly_20240101.tar.gz
# Restore specific backup (filename only - auto-resolves path)
./scripts/restore-storage.sh weekly_20240101.tar.gz
```
### restore-all.sh
Complete system restoration script for both database and storage.
**Usage:**
```bash
./scripts/restore-all.sh [options]
```
**Options:**
- `--latest`: Restore from latest backups (both database and storage)
- `--database-file FILE`: Specify database backup file
- `--storage-file FILE`: Specify storage backup file
- `--database-latest`: Use latest database backup only
- `--storage-latest`: Use latest storage backup only
- `--confirm`: Skip confirmation prompts
- `--list-backups`: List available backups
**Examples:**
```bash
# Restore both database and storage from latest backups
./scripts/restore-all.sh --latest
# Restore specific files
./scripts/restore-all.sh --database-file timebank_daily_20240101.sql.gz --storage-file daily_20240101.tar.gz
# Database only restore
./scripts/restore-all.sh --database-latest
# List all available backups
./scripts/restore-all.sh --list-backups
```
**Features:**
- Orchestrates complete system restoration
- Handles both database and storage restoration
- Automatic path resolution for backup files
- Pre-restore backups for safety
- Email notifications on success
- Post-restore Laravel optimization
## Automation Setup
### Cron Configuration
Copy and customize the cron configuration:
```bash
# Copy template
sudo cp scripts/cron-backup.conf /etc/cron.d/timebank-backup
# Edit paths and email addresses
sudo nano /etc/cron.d/timebank-backup
# Verify cron syntax
sudo cron -T
```
### Alternative: User Crontab
For non-root installations:
```bash
# Edit user crontab
crontab -e
# Add backup schedules (example)
0 2 * * * cd /path/to/timebank && ./scripts/backup-all.sh daily --verify
0 3 * * 0 cd /path/to/timebank && ./scripts/backup-all.sh weekly --verify
0 4 1 * * cd /path/to/timebank && ./scripts/backup-all.sh monthly --verify
```
### Email Notification Setup
All backup and restore scripts automatically send email notifications on successful completion.
#### Prerequisites
Install mail utilities:
```bash
# Ubuntu/Debian
sudo apt install mailutils postfix
# Configure postfix for "Local only" delivery during setup
```
#### Email Configuration
By default, notifications are sent to `$USER@localhost`. To customize:
```bash
# Set custom email address (optional)
export BACKUP_NOTIFY_EMAIL="admin@yourdomain.org"
```
#### Email Subjects and Content
**Backup Notifications:**
- Subject: "Timebank DB Backup Success"
- Subject: "Timebank Storage Backup Success"
- Content: Includes timestamp and backup details
**Restore Notifications:**
- Subject: "Timebank DB Restore Success"
- Subject: "Timebank Storage Restore Success"
- Subject: "Timebank Complete Restore Success"
- Content: Includes timestamp and restored file information
#### Checking Local Mail
View received notifications:
```bash
# Open mail client
mail
# List messages
mail -H
# Quick check for new messages
ls -la /var/mail/$USER
```
#### Webhook Notifications (Optional)
For additional notification methods:
```bash
# Webhook notifications
export BACKUP_WEBHOOK_URL="https://hooks.slack.com/your-webhook"
```
## Monitoring & Maintenance
### cleanup-backups.sh
Manages backup retention and disk space.
**Usage:**
```bash
./scripts/cleanup-backups.sh [options]
```
**Options:**
- `--dry-run`: Show what would be deleted
- `--force`: Force cleanup regardless of disk space
- `--verbose`: Show detailed information
**Features:**
- Automatic retention policy enforcement
- Disk space monitoring
- Empty directory cleanup
- Detailed reporting
**Example:**
```bash
# Dry run to see what would be cleaned up
./scripts/cleanup-backups.sh --dry-run --verbose
# Force cleanup
./scripts/cleanup-backups.sh --force
```
### Retention Policy Configuration
The cleanup script uses a configurable retention policy system that allows you to customize backup retention settings.
#### Configuration File
**Location**: `scripts/backup-retention.conf`
The configuration file controls:
- **Time-based retention**: How long to keep backups (in days)
- **Count-based retention**: How many recent backups to keep
- **Disk space thresholds**: When to trigger cleanup
- **Email notifications**: Enable/disable cleanup emails
- **Advanced settings**: Cleanup modes and verification options
#### Configuration Options
**Time-based Retention (days):**
```bash
DAILY_RETENTION=7 # Keep daily backups for 7 days
WEEKLY_RETENTION=28 # Keep weekly backups for 28 days
MONTHLY_RETENTION=365 # Keep monthly backups for 365 days
PRE_RESTORE_RETENTION=30 # Keep pre-restore backups for 30 days
LOG_RETENTION=30 # Keep log files for 30 days
```
**Count-based Retention (number of files):**
```bash
DAILY_COUNT_LIMIT=7 # Keep 7 most recent daily backups
WEEKLY_COUNT_LIMIT=4 # Keep 4 most recent weekly backups
MONTHLY_COUNT_LIMIT=12 # Keep 12 most recent monthly backups
PRE_RESTORE_COUNT_LIMIT=5 # Keep 5 most recent pre-restore backups
SNAPSHOT_COUNT_LIMIT=3 # Keep 3 most recent storage snapshots
```
**Disk Space Management:**
```bash
DISK_WARNING_THRESHOLD=85 # Send warning at 85% disk usage
DISK_CRITICAL_THRESHOLD=95 # Force cleanup at 95% disk usage
```
**Notification Settings:**
```bash
EMAIL_NOTIFICATIONS_ENABLED=true # Enable/disable email notifications
BACKUP_NOTIFY_EMAIL=admin@domain.org # Custom email address (optional)
```
#### Customizing Retention Policies
**View Current Configuration:**
```bash
./scripts/cleanup-backups.sh --help
```
**Edit Configuration:**
```bash
# Edit the config file
nano scripts/backup-retention.conf
# Test changes with dry run
./scripts/cleanup-backups.sh --dry-run --verbose
```
#### Configuration Examples
**Conservative (longer retention):**
```bash
DAILY_RETENTION=14 # 2 weeks
WEEKLY_RETENTION=56 # 8 weeks
MONTHLY_RETENTION=730 # 2 years
DAILY_COUNT_LIMIT=14 # More daily backups
MONTHLY_COUNT_LIMIT=24 # 2 years of monthly backups
```
**Aggressive (shorter retention):**
```bash
DAILY_RETENTION=3 # 3 days only
WEEKLY_RETENTION=14 # 2 weeks
MONTHLY_RETENTION=180 # 6 months
DAILY_COUNT_LIMIT=3 # Fewer daily backups
DISK_WARNING_THRESHOLD=75 # Earlier warning
```
**Space-constrained environment:**
```bash
DAILY_COUNT_LIMIT=3 # Keep only 3 daily backups
WEEKLY_COUNT_LIMIT=2 # Keep only 2 weekly backups
MONTHLY_COUNT_LIMIT=6 # Keep only 6 monthly backups
DISK_WARNING_THRESHOLD=70 # Early disk space warnings
DISK_CRITICAL_THRESHOLD=85 # More aggressive cleanup threshold
```
#### Validation and Safety
The configuration system includes:
- **Input validation**: Invalid values fall back to defaults
- **Range checking**: Values must be within reasonable ranges
- **Relationship validation**: Critical threshold must be higher than warning
- **Fallback system**: Uses defaults if config file is missing or corrupted
#### Advanced Settings
```bash
# Cleanup behavior
CLEANUP_MODE=both # Options: age_only, count_only, both
CLEANUP_EMPTY_DIRS=true # Remove empty directories after cleanup
VERIFY_BEFORE_DELETE=false # Verify backup integrity before deletion
# Email control
EMAIL_NOTIFICATIONS_ENABLED=true # Master switch for email notifications
```
### Health Monitoring
The backup system includes health checks:
```bash
# View health check results
cat backups/health_check.json
# Check backup logs
tail -f backups/backup.log
# View backup summary
./scripts/backup-all.sh daily --verify
```
### Log Management
Backup logs are stored in:
- `backups/backup.log` - Main backup log
- `backups/restore.log` - Restore operations log
- `/var/log/timebank-backup.log` - System cron log
Rotate logs using logrotate:
```bash
# Create logrotate configuration
sudo tee /etc/logrotate.d/timebank-backup <<EOF
/var/log/timebank-backup.log {
daily
missingok
rotate 30
compress
delaycompress
notifempty
create 644 root root
}
EOF
```
## Troubleshooting
### Common Issues
#### Permission Denied
```bash
# Fix script permissions
chmod +x scripts/*.sh
# Fix backup directory permissions
sudo chown -R $(whoami):$(whoami) backups/
```
#### MySQL Connection Issues
```bash
# Test database connection
mysql -h$DB_HOST -P$DB_PORT -u$DB_USERNAME -p$DB_PASSWORD -e "SHOW DATABASES;"
# Check .env file
grep "^DB_" .env
```
#### Disk Space Issues
```bash
# Check available space
df -h backups/
# Run cleanup
./scripts/cleanup-backups.sh --force
# Check large files
find backups/ -type f -size +100M -exec ls -lh {} \;
```
#### Backup Verification Failures
```bash
# Test gzip integrity
gzip -t backups/database/daily/*.sql.gz
# Test tar integrity
tar -tzf backups/storage/daily/*.tar.gz >/dev/null
```
### Recovery Procedures
#### Complete System Recovery
1. **Prepare clean environment:**
```bash
# Install Laravel dependencies
composer install --no-dev --optimize-autoloader
npm install && npm run build
```
2. **Restore database:**
```bash
./scripts/restore-database.sh --latest --confirm
```
3. **Restore storage:**
```bash
./scripts/restore-storage.sh --latest --confirm
```
4. **Post-restore steps:**
```bash
php artisan storage:link
php artisan config:clear
php artisan cache:clear
php artisan migrate:status
```
#### Partial Recovery
**Database only:**
```bash
./scripts/restore-database.sh --latest
php artisan migrate:status
```
**Storage only:**
```bash
./scripts/restore-storage.sh --latest --merge
php artisan storage:link
```
## Best Practices
### Security
1. **Protect backup files:**
```bash
chmod 600 backups/database/**/*.sql.gz
chmod 600 backups/storage/**/*.tar.gz
```
2. **Use secure file transfer:**
```bash
# Example: Upload to secure cloud storage
rclone sync backups/ remote:timebank-backups/
```
3. **Encrypt sensitive backups:**
```bash
# Example: GPG encryption
gpg --symmetric --cipher-algo AES256 backup.sql.gz
```
### Performance
1. **Schedule during low-traffic periods**
2. **Monitor backup duration and adjust frequency**
3. **Use fast storage for backup directory**
4. **Consider incremental-only backups for large storage**
### Reliability
1. **Test restores regularly:**
```bash
# Monthly restore test
./scripts/restore-database.sh --latest --confirm
```
2. **Monitor backup success:**
```bash
# Check recent backup status
find backups/ -name "*.gz" -mtime -1 -ls
```
3. **Verify backup integrity:**
```bash
# Run verification on all recent backups
./scripts/backup-all.sh daily --verify
```
4. **Keep multiple backup locations:**
- Local backups for quick recovery
- Remote backups for disaster recovery
- Cloud storage for long-term retention
### Documentation
1. **Keep this guide updated**
2. **Document any customizations**
3. **Maintain recovery contact information**
4. **Document environment-specific configurations**
---
## Support
For issues or questions regarding the backup system:
1. Check the troubleshooting section above
2. Review backup logs: `tail -f backups/backup.log`
3. Test with `--dry-run` options first
4. Ensure all prerequisites are installed
**Remember**: Always test your restore procedures in a non-production environment before relying on them in production!
scripts/backup-all.sh
Executable
+377
View File
@@ -0,0 +1,377 @@
#!/bin/bash
# Laravel Timebank Master Backup Script
# Orchestrates database and storage backups with health checks
# Usage: ./backup-all.sh [backup_type] [options]
# backup_type: daily (default), weekly, monthly
# options: --storage-only, --database-only, --verify, --notify
set -e # Exit on any error
umask 027 # Ensure backup files are created as 640 (not world-readable)
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
BACKUP_ROOT_DIR="$PROJECT_ROOT/backups"
LOG_FILE="$BACKUP_ROOT_DIR/backup.log"
# --- Dead-man's-switch -------------------------------------------------------
#
# A failure notification is not enough on its own: between at least 2026-06 and
# 2026-09-22 every database backup failed, and `set -e` killed this script
# before it could reach send_notification, so nothing was ever sent. The only
# reliable signal is one that fires when a run does NOT complete.
#
# BACKUP_HEALTHCHECK_URL is read from .env (healthchecks.io or any service that
# alerts on a missed ping). Empty = disabled, so this is a no-op until set.
#
# The trap fires on EVERY exit path, `set -e` included, so a killed script
# reports failure instead of silence.
healthcheck_ping() {
local suffix="$1"
# Read straight from .env rather than relying on the environment: this
# script does not source .env (only its children do), and the trap can
# fire before any of them have run.
local url
url=$(sed -n 's/^BACKUP_HEALTHCHECK_URL=//p' "$PROJECT_ROOT/.env" 2>/dev/null | head -1)
url=${url%%#*}
url=$(printf '%s' "$url" | tr -d '"' | tr -d "'" | xargs 2>/dev/null)
[ -n "$url" ] || return 0
curl -fsS -m 10 --retry 3 -o /dev/null "${url}${suffix}" 2>/dev/null || true
}
on_exit() {
local code=$?
if [ $code -eq 0 ]; then
healthcheck_ping ""
else
healthcheck_ping "/fail"
fi
return $code
}
trap on_exit EXIT
# -----------------------------------------------------------------------------
HEALTH_CHECK_FILE="$BACKUP_ROOT_DIR/health_check.json"
# Create backup directories
mkdir -p "$BACKUP_ROOT_DIR/logs"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Logging function with colors
log() {
local level="$1"
local message="$2"
local timestamp="[$(date '+%Y-%m-%d %H:%M:%S')]"
case "$level" in
"INFO")
echo -e "${timestamp} ${BLUE}[INFO]${NC} $message" | tee -a "$LOG_FILE"
;;
"SUCCESS")
echo -e "${timestamp} ${GREEN}[SUCCESS]${NC} $message" | tee -a "$LOG_FILE"
;;
"WARNING")
echo -e "${timestamp} ${YELLOW}[WARNING]${NC} $message" | tee -a "$LOG_FILE"
;;
"ERROR")
echo -e "${timestamp} ${RED}[ERROR]${NC} $message" | tee -a "$LOG_FILE"
;;
*)
echo -e "${timestamp} $level $message" | tee -a "$LOG_FILE"
;;
esac
}
# Function to show usage
show_usage() {
echo "Usage: $0 [backup_type] [options]"
echo ""
echo "Backup Types:"
echo " daily - Daily backup (default)"
echo " weekly - Weekly backup"
echo " monthly - Monthly backup"
echo ""
echo "Options:"
echo " --storage-only - Backup only storage files"
echo " --database-only - Backup only database"
echo " --verify - Verify backups after creation"
echo " --notify - Send notification on completion"
echo " --help - Show this help message"
echo ""
exit 0
}
# Health check function
health_check() {
log "INFO" "Starting backup health check"
local status="healthy"
local issues=()
# Check disk space (warn if less than 1GB free)
local available_space=$(df "$BACKUP_ROOT_DIR" | awk 'NR==2 {print $4}')
local available_mb=$((available_space / 1024))
if [ "$available_mb" -lt 1024 ]; then
status="warning"
issues+=("Low disk space: ${available_mb}MB available")
fi
# Check if backup scripts exist and are executable
for script in "backup-database.sh" "backup-storage.sh"; do
if [ ! -x "$SCRIPT_DIR/$script" ]; then
status="error"
issues+=("Script not found or not executable: $script")
fi
done
# Check environment file
if [ ! -f "$PROJECT_ROOT/.env" ]; then
status="error"
issues+=("Environment file (.env) not found")
fi
# Generate health check report
cat > "$HEALTH_CHECK_FILE" <<EOF
{
"timestamp": "$(date -Iseconds)",
"status": "$status",
"available_space_mb": $available_mb,
"issues": [$(printf '"%s",' "${issues[@]}" | sed 's/,$//')],
"backup_directory": "$BACKUP_ROOT_DIR",
"last_check": "$(date)"
}
EOF
if [ "$status" = "error" ]; then
log "ERROR" "Health check failed. Issues found:"
for issue in "${issues[@]}"; do
log "ERROR" " - $issue"
done
return 1
elif [ "$status" = "warning" ]; then
log "WARNING" "Health check completed with warnings:"
for issue in "${issues[@]}"; do
log "WARNING" " - $issue"
done
else
log "SUCCESS" "Health check passed"
fi
return 0
}
# Backup verification function
verify_backups() {
log "INFO" "Verifying recent backups"
local verification_passed=true
# Verify database backups
local latest_db_backup=$(find "$BACKUP_ROOT_DIR/database" -name "*.sql.gz" -type f -mtime -1 | head -n 1)
if [ -n "$latest_db_backup" ]; then
if gzip -t "$latest_db_backup" 2>/dev/null; then
log "SUCCESS" "Database backup verified: $(basename "$latest_db_backup")"
else
log "ERROR" "Database backup verification failed: $(basename "$latest_db_backup")"
verification_passed=false
fi
else
log "WARNING" "No recent database backup found"
fi
# Verify storage backups
local latest_storage_backup=$(find "$BACKUP_ROOT_DIR/storage" -name "*.tar.gz" -type f -mtime -1 | head -n 1)
if [ -n "$latest_storage_backup" ]; then
if tar -tzf "$latest_storage_backup" >/dev/null 2>&1; then
log "SUCCESS" "Storage backup verified: $(basename "$latest_storage_backup")"
else
log "ERROR" "Storage backup verification failed: $(basename "$latest_storage_backup")"
verification_passed=false
fi
else
log "WARNING" "No recent storage backup found"
fi
# `return` takes an integer, not the string "true"/"false": returning the
# variable directly raised "numeric argument required" and yielded a
# non-zero status, so verification reported "issues" even when every check
# had just passed - and a genuine failure looked exactly the same.
if [ "$verification_passed" = true ]; then
return 0
fi
return 1
}
# Notification function
send_notification() {
local subject="$1"
local message="$2"
local status="$3"
# Try multiple notification methods
# Method 1: Email (if mail is available and configured)
if command -v mail >/dev/null 2>&1; then
echo "$message" | mail -s "$subject" "${BACKUP_NOTIFY_EMAIL:-$USER@localhost}" 2>/dev/null || true
fi
# Method 2: System notification (if notify-send is available)
if command -v notify-send >/dev/null 2>&1 && [ -n "$DISPLAY" ]; then
notify-send "$subject" "$message" 2>/dev/null || true
fi
# Method 3: Webhook (if BACKUP_WEBHOOK_URL is set)
if [ -n "$BACKUP_WEBHOOK_URL" ] && command -v curl >/dev/null 2>&1; then
curl -X POST "$BACKUP_WEBHOOK_URL" \
-H "Content-Type: application/json" \
-d "{\"subject\":\"$subject\",\"message\":\"$message\",\"status\":\"$status\"}" \
2>/dev/null || true
fi
# Method 4: Log file notification marker
log "INFO" "NOTIFICATION: $subject - $message"
}
# Parse command line arguments
BACKUP_TYPE="daily"
STORAGE_ONLY=false
DATABASE_ONLY=false
VERIFY=false
NOTIFY=false
while [[ $# -gt 0 ]]; do
case $1 in
daily|weekly|monthly)
BACKUP_TYPE="$1"
shift
;;
--storage-only)
STORAGE_ONLY=true
shift
;;
--database-only)
DATABASE_ONLY=true
shift
;;
--verify)
VERIFY=true
shift
;;
--notify)
NOTIFY=true
shift
;;
--help)
show_usage
;;
*)
log "ERROR" "Unknown option: $1"
show_usage
;;
esac
done
# Main execution
main() {
log "INFO" "============================================"
log "INFO" "Starting Timebank $BACKUP_TYPE backup process"
log "INFO" "Time: $(date)"
log "INFO" "============================================"
local start_time=$(date +%s)
local backup_success=true
local status_message=""
# Pre-backup health check
if ! health_check; then
log "ERROR" "Health check failed, aborting backup"
exit 1
fi
# Database backup
if [ "$STORAGE_ONLY" = false ]; then
log "INFO" "Starting database backup..."
if "$SCRIPT_DIR/backup-database.sh" "$BACKUP_TYPE"; then
log "SUCCESS" "Database backup completed"
else
log "ERROR" "Database backup failed"
backup_success=false
fi
fi
# Storage backup
if [ "$DATABASE_ONLY" = false ]; then
log "INFO" "Starting storage backup..."
if "$SCRIPT_DIR/backup-storage.sh" "$BACKUP_TYPE"; then
log "SUCCESS" "Storage backup completed"
else
log "ERROR" "Storage backup failed"
backup_success=false
fi
fi
# Backup verification
if [ "$VERIFY" = true ]; then
if verify_backups; then
log "SUCCESS" "Backup verification passed"
else
log "WARNING" "Backup verification had issues"
fi
fi
# Calculate execution time
local end_time=$(date +%s)
local execution_time=$((end_time - start_time))
local execution_time_formatted=$(date -d@$execution_time -u +%H:%M:%S)
# Generate final status
if [ "$backup_success" = true ]; then
status_message="Backup completed successfully in $execution_time_formatted"
log "SUCCESS" "$status_message"
# Generate backup summary
log "INFO" "Backup Summary:"
log "INFO" " - Type: $BACKUP_TYPE"
log "INFO" " - Duration: $execution_time_formatted"
log "INFO" " - Location: $BACKUP_ROOT_DIR"
if [ "$NOTIFY" = true ]; then
send_notification "Timebank Backup Success" "$status_message" "success"
fi
else
status_message="Backup completed with errors in $execution_time_formatted"
log "ERROR" "$status_message"
if [ "$NOTIFY" = true ]; then
send_notification "Timebank Backup Failed" "$status_message" "error"
fi
exit 1
fi
log "INFO" "============================================"
log "INFO" "Backup process finished"
log "INFO" "============================================"
}
# Trap to handle interrupts
trap 'log "ERROR" "Backup interrupted by user"; exit 1' INT TERM
# Run main function
main "$@"
scripts/backup-database.sh
Executable
+170
View File
@@ -0,0 +1,170 @@
#!/bin/bash
# Laravel Timebank Database Backup Script
# Backs up MySQL database with compression and rotation
# Usage: ./backup-database.sh [backup_type]
# backup_type: daily (default), weekly, monthly
set -e # Exit on any error
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
BACKUP_ROOT_DIR="$PROJECT_ROOT/backups"
LOG_FILE="$BACKUP_ROOT_DIR/backup.log"
# Create backup directories
mkdir -p "$BACKUP_ROOT_DIR"/{database/{daily,weekly,monthly},logs}
# Logging function
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}
# Load environment loader
source "$SCRIPT_DIR/load-env.sh"
# Load environment variables
if ! load_env "$PROJECT_ROOT/.env"; then
log "ERROR: .env file not found in $PROJECT_ROOT"
exit 1
fi
# Validate required environment variables
if [ -z "$DB_DATABASE" ] || [ -z "$DB_USERNAME" ] || [ -z "$DB_HOST" ]; then
log "ERROR: Required database environment variables not found (DB_DATABASE, DB_USERNAME, DB_HOST)"
exit 1
fi
# Set backup type (daily, weekly, monthly)
BACKUP_TYPE="${1:-daily}"
TIMESTAMP=$(date '+%Y%m%d_%H%M%S')
BACKUP_DIR="$BACKUP_ROOT_DIR/database/$BACKUP_TYPE"
BACKUP_FILE="$BACKUP_DIR/${DB_DATABASE}_${BACKUP_TYPE}_${TIMESTAMP}.sql"
COMPRESSED_FILE="${BACKUP_FILE}.gz"
# Create backup type directory
mkdir -p "$BACKUP_DIR"
log "Starting $BACKUP_TYPE database backup for $DB_DATABASE"
# Perform database backup
log "Creating database dump..."
# Create MySQL configuration file for secure password handling
MYSQL_CNF_FILE="/tmp/mysql_backup_$$.cnf"
cat > "$MYSQL_CNF_FILE" <<EOF
[mysqldump]
host=${DB_HOST:-127.0.0.1}
port=${DB_PORT:-3306}
user=$DB_USERNAME
password=$DB_PASSWORD
EOF
# Set secure permissions on the config file
chmod 600 "$MYSQL_CNF_FILE"
# Perform the backup using the config file.
#
# Failure is handled explicitly rather than left to `set -e`: when mysqldump
# exits non-zero the shell would otherwise kill this script on the spot,
# leaving a TRUNCATED .sql on disk, skipping the gzip below, and never
# reaching the error branch. That is exactly what happened between at least
# 2026-09-18 and 2026-09-22 - the dump died at its final step and left a
# ~112 MB partial file that looked like a real backup and was rotated like
# one. A backup that fails must not leave anything behind that can be
# mistaken for a good one.
#
# --events is deliberately NOT passed. It requires the EVENT privilege, which
# the application user does not have and should not be given: the schema
# defines no MySQL events (Laravel schedules through cron instead), so the
# flag only widened the app's privileges to satisfy the backup. If events are
# ever introduced, add --events AND grant EVENT on that database - not the
# other way round.
set +e
mysqldump \
--defaults-extra-file="$MYSQL_CNF_FILE" \
--single-transaction \
--routines \
--triggers \
--add-drop-database \
--databases "$DB_DATABASE" \
> "$BACKUP_FILE" 2> "${BACKUP_FILE}.err"
mysqldump_status=$?
set -e
# Clean up the temporary config file
rm -f "$MYSQL_CNF_FILE"
if [ $mysqldump_status -ne 0 ]; then
log "ERROR: mysqldump failed (exit $mysqldump_status): $(tr '\n' ' ' < "${BACKUP_FILE}.err")"
# Remove the partial dump: keeping it would let retention rotate a broken
# backup as though it were valid.
rm -f "$BACKUP_FILE" "${BACKUP_FILE}.err"
exit 1
fi
rm -f "${BACKUP_FILE}.err"
# A dump that does not end with mysqldump's completion marker is truncated,
# whatever the exit status said.
if ! tail -c 200 "$BACKUP_FILE" | grep -q "Dump completed"; then
log "ERROR: dump is truncated (no completion marker) - discarding $BACKUP_FILE"
rm -f "$BACKUP_FILE"
exit 1
fi
# Compress the backup
log "Compressing backup..."
gzip "$BACKUP_FILE"
# Verify the compressed backup exists and has content
if [ -f "$COMPRESSED_FILE" ] && [ -s "$COMPRESSED_FILE" ]; then
BACKUP_SIZE=$(du -h "$COMPRESSED_FILE" | cut -f1)
log "SUCCESS: Database backup completed - $COMPRESSED_FILE ($BACKUP_SIZE)"
else
log "ERROR: Backup verification failed - $COMPRESSED_FILE"
exit 1
fi
# Retention cleanup based on backup type
cleanup_old_backups() {
local backup_dir="$1"
local keep_count="$2"
local pattern="$3"
log "Cleaning up old $BACKUP_TYPE backups (keeping $keep_count most recent)"
# Remove old backups, keeping only the specified number
ls -t "$backup_dir"/$pattern 2>/dev/null | tail -n +$((keep_count + 1)) | while read -r old_backup; do
if [ -f "$backup_dir/$old_backup" ]; then
rm "$backup_dir/$old_backup"
log "Removed old backup: $old_backup"
fi
done
}
# Apply retention policy
case "$BACKUP_TYPE" in
daily)
cleanup_old_backups "$BACKUP_DIR" 7 "${DB_DATABASE}_daily_*.sql.gz"
;;
weekly)
cleanup_old_backups "$BACKUP_DIR" 4 "${DB_DATABASE}_weekly_*.sql.gz"
;;
monthly)
cleanup_old_backups "$BACKUP_DIR" 12 "${DB_DATABASE}_monthly_*.sql.gz"
;;
esac
# Generate backup report
TOTAL_BACKUPS=$(find "$BACKUP_ROOT_DIR/database" -name "*.sql.gz" | wc -l)
TOTAL_SIZE=$(du -sh "$BACKUP_ROOT_DIR/database" | cut -f1)
log "Backup summary: $TOTAL_BACKUPS total backups, $TOTAL_SIZE total size"
log "$BACKUP_TYPE database backup completed successfully"
# Send notification
if command -v mail >/dev/null 2>&1; then
echo "Database backup completed successfully at $(date)" | mail -s "Timebank DB Backup Success" "${BACKUP_NOTIFY_EMAIL:-$USER@localhost}" 2>/dev/null || true
fi
scripts/backup-retention.conf
+145
View File
@@ -0,0 +1,145 @@
# Timebank Backup Retention Policy Configuration
#
# This file controls how long different types of backups are kept before being automatically cleaned up.
#
# IMPORTANT NOTES:
# - Values are in days unless otherwise specified
# - Changes take effect immediately on next cleanup run
# - Lower values = less storage used, shorter recovery window
# - Higher values = more storage used, longer recovery window
# - Test changes with --dry-run before applying
#
# Last updated: $(date)
# =============================================================================
# TIME-BASED RETENTION (in days)
# These policies delete backups older than the specified number of days
# =============================================================================
# Daily backups retention (default: 7 days)
# How long to keep daily backups before deletion
DAILY_RETENTION=3
# Weekly backups retention (default: 28 days = 4 weeks)
# How long to keep weekly backups before deletion
WEEKLY_RETENTION=28
# Monthly backups retention (default: 365 days = 12 months)
# How long to keep monthly backups before deletion
MONTHLY_RETENTION=365
# Pre-restore backups retention (default: 30 days)
# How long to keep automatic backups created before restore operations
PRE_RESTORE_RETENTION=14
# Log files retention (default: 30 days)
# How long to keep backup and cleanup log files
LOG_RETENTION=30
# =============================================================================
# COUNT-BASED RETENTION
# These policies keep only the N most recent backups of each type
# =============================================================================
# Daily backup count limits (default: 7)
# Maximum number of daily backups to keep (newest ones)
DAILY_COUNT_LIMIT=3
# Weekly backup count limits (default: 4)
# Maximum number of weekly backups to keep (newest ones)
WEEKLY_COUNT_LIMIT=2
# Monthly backup count limits (default: 12)
# Maximum number of monthly backups to keep (newest ones). Applies to the
# database dumps, and to storage archives unless STORAGE_MONTHLY_COUNT_LIMIT
# is set.
MONTHLY_COUNT_LIMIT=3
# Monthly STORAGE archives only (default: MONTHLY_COUNT_LIMIT). Each is a
# ~490MB copy of storage/ that barely differs from the next, while a monthly
# database dump is ~5MB: keep three months of database history but only the
# latest storage archive. Cut from 3 on 2026-10-07 to free disk (~1GB).
STORAGE_MONTHLY_COUNT_LIMIT=1
# Pre-restore backup count limits (default: 3)
# Maximum number of pre-restore backups to keep (newest ones)
# Kept low: multiple restores in one day can quickly create many files
PRE_RESTORE_COUNT_LIMIT=3
# Storage snapshot count limits (default: 3)
# Maximum number of storage snapshots to keep (newest ones)
SNAPSHOT_COUNT_LIMIT=3
# Whole-storage archives (backups/storage/full). These are the largest single
# files in the tree, and had no cleanup rule at all before 2026-09-22.
# Cut from 2 to 1 on 2026-10-07 (~490MB): the daily, weekly and monthly
# archives already hold several older copies of the same storage.
FULL_COUNT_LIMIT=1
FULL_RETENTION=14
# =============================================================================
# DISK SPACE MANAGEMENT
# These settings control when cleanup is automatically triggered
# =============================================================================
# Disk usage warning threshold (default: 80%)
# Send warning emails and trigger cleanup when disk usage exceeds this percentage
DISK_WARNING_THRESHOLD=75
# Disk usage critical threshold (default: 95%)
# Force aggressive cleanup when disk usage exceeds this percentage
DISK_CRITICAL_THRESHOLD=85
# =============================================================================
# EMAIL NOTIFICATION SETTINGS
# =============================================================================
# Email address for backup notifications (optional)
# If not set, defaults to $USER@localhost
# BACKUP_NOTIFY_EMAIL=admin@yourdomain.org
# Enable/disable email notifications (default: true)
# Set to false to disable all email notifications from cleanup script
EMAIL_NOTIFICATIONS_ENABLED=true
# =============================================================================
# ADVANCED SETTINGS
# =============================================================================
# Cleanup mode (default: both)
# Options: age_only, count_only, both
# - age_only: Only use time-based retention
# - count_only: Only use count-based retention
# - both: Use both methods (recommended for safety)
CLEANUP_MODE=both
# Empty directory cleanup (default: true)
# Whether to remove empty backup directories after file cleanup
CLEANUP_EMPTY_DIRS=false
# Backup verification before deletion (default: false)
# Set to true to verify backup integrity before deletion (slower but safer)
VERIFY_BEFORE_DELETE=false
# =============================================================================
# CONFIGURATION EXAMPLES
# =============================================================================
#
# Conservative (longer retention):
# DAILY_RETENTION=14
# WEEKLY_RETENTION=56 # 8 weeks
# MONTHLY_RETENTION=730 # 2 years
#
# Aggressive (shorter retention, less storage):
# DAILY_RETENTION=3
# WEEKLY_RETENTION=14 # 2 weeks
# MONTHLY_RETENTION=180 # 6 months
#
# Space-constrained environment:
# DAILY_COUNT_LIMIT=3
# WEEKLY_COUNT_LIMIT=2
# MONTHLY_COUNT_LIMIT=6
# DISK_WARNING_THRESHOLD=75
# DISK_CRITICAL_THRESHOLD=85
#
# =============================================================================
scripts/backup-storage.sh
Executable
+235
View File
@@ -0,0 +1,235 @@
#!/bin/bash
# Laravel Timebank Storage Backup Script
# Backs up storage directory with incremental rsync and compression
# Usage: ./backup-storage.sh [backup_type]
# backup_type: daily (default), weekly, monthly, full
set -e # Exit on any error
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
STORAGE_DIR="$PROJECT_ROOT/storage"
BACKUP_ROOT_DIR="$PROJECT_ROOT/backups"
LOG_FILE="$BACKUP_ROOT_DIR/backup.log"
# Create backup directories
mkdir -p "$BACKUP_ROOT_DIR"/{storage/{daily,weekly,monthly,snapshots},logs}
# Logging function
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}
# Check if storage directory exists
if [ ! -d "$STORAGE_DIR" ]; then
log "ERROR: Storage directory not found: $STORAGE_DIR"
exit 1
fi
# Set backup type
BACKUP_TYPE="${1:-daily}"
TIMESTAMP=$(date '+%Y%m%d_%H%M%S')
BACKUP_DIR="$BACKUP_ROOT_DIR/storage/$BACKUP_TYPE"
SNAPSHOT_DIR="$BACKUP_ROOT_DIR/storage/snapshots"
# Create directories
mkdir -p "$BACKUP_DIR" "$SNAPSHOT_DIR"
log "Starting $BACKUP_TYPE storage backup"
# Rsync exclude patterns for Laravel storage
RSYNC_EXCLUDES="
--exclude=framework/cache/*
--exclude=framework/sessions/*
--exclude=framework/testing/*
--exclude=framework/views/*
--exclude=logs/*.log
--exclude=debugbar/*
--exclude=app/backup/*
--exclude=*/livewire-tmp/*
--exclude=*.tmp
--exclude=.DS_Store
--exclude=Thumbs.db
"
# Function for incremental backup using rsync
incremental_backup() {
local target_dir="$1"
local snapshot_name="$2"
log "Performing incremental backup to $target_dir"
# Create current snapshot directory
local current_snapshot="$SNAPSHOT_DIR/$snapshot_name"
mkdir -p "$current_snapshot"
# Rsync with hard links to previous snapshot for space efficiency
local link_dest_option=""
local latest_snapshot=$(find "$SNAPSHOT_DIR" -maxdepth 1 -type d -name "${BACKUP_TYPE}_*" | sort -r | head -n 1)
if [ -n "$latest_snapshot" ] && [ "$latest_snapshot" != "$current_snapshot" ]; then
link_dest_option="--link-dest=$latest_snapshot"
fi
rsync -av \
--delete \
$RSYNC_EXCLUDES \
$link_dest_option \
"$STORAGE_DIR/" \
"$current_snapshot/" \
2>&1 | tee -a "$LOG_FILE"
# Create compressed archive for long-term storage
local archive_name="${snapshot_name}.tar.gz"
local archive_path="$target_dir/$archive_name"
log "Creating compressed archive: $archive_name"
tar -czf "$archive_path" -C "$SNAPSHOT_DIR" "$snapshot_name" 2>&1 | tee -a "$LOG_FILE"
# Verify archive
if [ -f "$archive_path" ] && [ -s "$archive_path" ]; then
local archive_size=$(du -h "$archive_path" | cut -f1)
log "Archive created successfully: $archive_path ($archive_size)"
# Keep snapshots for recent backups only (last 3)
if [ "$BACKUP_TYPE" = "daily" ]; then
# Clean up old daily snapshots (keep last 3)
find "$SNAPSHOT_DIR" -maxdepth 1 -type d -name "daily_*" | sort -r | tail -n +4 | xargs -r rm -rf
fi
return 0
else
log "ERROR: Archive creation failed or archive is empty"
return 1
fi
}
# Function for full backup (copy entire directory)
full_backup() {
local target_dir="$1"
local backup_name="storage_full_${TIMESTAMP}"
local backup_path="$target_dir/${backup_name}.tar.gz"
log "Performing full storage backup"
# Create temporary directory for clean backup
local temp_dir="/tmp/timebank_storage_backup_$$"
mkdir -p "$temp_dir"
# Copy storage with rsync (excluding unwanted files)
rsync -av $RSYNC_EXCLUDES "$STORAGE_DIR/" "$temp_dir/" 2>&1 | tee -a "$LOG_FILE"
# Create compressed archive
log "Creating compressed archive: $backup_name.tar.gz"
tar -czf "$backup_path" -C "/tmp" "timebank_storage_backup_$$" 2>&1 | tee -a "$LOG_FILE"
# Cleanup temporary directory
rm -rf "$temp_dir"
# Verify archive
if [ -f "$backup_path" ] && [ -s "$backup_path" ]; then
local archive_size=$(du -h "$backup_path" | cut -f1)
log "Full backup completed: $backup_path ($archive_size)"
return 0
else
log "ERROR: Full backup failed"
return 1
fi
}
# Perform backup based on type
case "$BACKUP_TYPE" in
daily)
incremental_backup "$BACKUP_DIR" "daily_${TIMESTAMP}"
;;
weekly)
incremental_backup "$BACKUP_DIR" "weekly_${TIMESTAMP}"
;;
monthly)
incremental_backup "$BACKUP_DIR" "monthly_${TIMESTAMP}"
;;
full)
full_backup "$BACKUP_DIR"
;;
*)
log "ERROR: Invalid backup type: $BACKUP_TYPE (valid: daily, weekly, monthly, full)"
exit 1
;;
esac
# Cleanup function
cleanup_old_backups() {
local backup_dir="$1"
local keep_count="$2"
local pattern="$3"
log "Cleaning up old $BACKUP_TYPE storage backups (keeping $keep_count most recent)"
find "$backup_dir" -name "$pattern" -type f | sort -r | tail -n +$((keep_count + 1)) | while read -r old_backup; do
if [ -f "$old_backup" ]; then
rm "$old_backup"
log "Removed old backup: $(basename "$old_backup")"
fi
done
}
# Apply retention policy
case "$BACKUP_TYPE" in
daily)
cleanup_old_backups "$BACKUP_DIR" 7 "daily_*.tar.gz"
;;
weekly)
cleanup_old_backups "$BACKUP_DIR" 4 "weekly_*.tar.gz"
;;
monthly)
cleanup_old_backups "$BACKUP_DIR" 12 "monthly_*.tar.gz"
;;
full)
cleanup_old_backups "$BACKUP_DIR" 2 "storage_full_*.tar.gz"
;;
esac
# This installation's platform folder (config, theme, overrides, public files; outside git, see
# App\Providers\PlatformServiceProvider). Small, so a full archive each run, with the same retention as above.
PLATFORM_DIR=$(grep -E '^PLATFORM_PATH=' "$PROJECT_ROOT/.env" 2>/dev/null | tail -1 | cut -d= -f2- | tr -d "\"'")
PLATFORM_DIR="${PLATFORM_DIR:-platform}"
[[ "$PLATFORM_DIR" = /* ]] || PLATFORM_DIR="$PROJECT_ROOT/$PLATFORM_DIR"
if [ -d "$PLATFORM_DIR" ]; then
platform_backup_dir="$BACKUP_ROOT_DIR/platform/$BACKUP_TYPE"
platform_archive="$platform_backup_dir/platform_${BACKUP_TYPE}_${TIMESTAMP}.tar.gz"
mkdir -p "$platform_backup_dir"
if tar -czf "$platform_archive" -C "$(dirname "$PLATFORM_DIR")" "$(basename "$PLATFORM_DIR")" 2>>"$LOG_FILE"; then
log "Platform folder backed up: $platform_archive ($(du -h "$platform_archive" | cut -f1))"
case "$BACKUP_TYPE" in
daily) platform_keep=7 ;; weekly) platform_keep=4 ;; monthly) platform_keep=12 ;; *) platform_keep=2 ;;
esac
find "$platform_backup_dir" -name "platform_${BACKUP_TYPE}_*.tar.gz" -type f | sort -r | tail -n +$((platform_keep + 1)) | xargs -r rm -f
else
rm -f "$platform_archive"
log "WARNING: backing up the platform folder $PLATFORM_DIR failed"
fi
fi
# Generate storage usage report
log "Storage backup summary:"
if [ -d "$BACKUP_ROOT_DIR/storage" ]; then
find "$BACKUP_ROOT_DIR/storage" -name "*.tar.gz" -exec basename {} \; | sort | while read -r backup; do
backup_path="$BACKUP_ROOT_DIR/storage"/*/"$backup"
if [ -f $backup_path ]; then
size=$(du -h $backup_path | cut -f1)
log " $backup ($size)"
fi
done
total_size=$(du -sh "$BACKUP_ROOT_DIR/storage" | cut -f1)
log "Total storage backups size: $total_size"
fi
log "$BACKUP_TYPE storage backup completed successfully"
# Send notification
if command -v mail >/dev/null 2>&1; then
echo "Storage backup completed successfully at $(date)" | mail -s "Timebank Storage Backup Success" "${BACKUP_NOTIFY_EMAIL:-$USER@localhost}" 2>/dev/null || true
fi
scripts/check-elasticsearch-security.sh
Executable
+152
View File
@@ -0,0 +1,152 @@
#!/bin/bash
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
echo -e "${BLUE}==========================================${NC}"
echo -e "${BLUE} Elasticsearch Security Check${NC}"
echo -e "${BLUE}==========================================${NC}\n"
# Check if Elasticsearch is running
if ! systemctl is-active --quiet elasticsearch; then
echo -e "${YELLOW}⚠ Elasticsearch is not running${NC}"
exit 0
fi
echo -e "${GREEN}✓ Elasticsearch is running${NC}\n"
# Check listening interfaces
echo -e "${BLUE}Checking network bindings...${NC}"
LISTEN_OUTPUT=$(ss -tlnp 2>/dev/null | grep 9200)
if [ -z "$LISTEN_OUTPUT" ]; then
echo -e "${RED}✗ Elasticsearch not found listening on port 9200${NC}\n"
exit 1
fi
echo "$LISTEN_OUTPUT"
echo ""
# Check if bound to localhost only
# Accept: 127.0.0.1, ::1, and ::ffff:127.0.0.1 (IPv6-mapped IPv4)
if echo "$LISTEN_OUTPUT" | grep -qE "127.0.0.1:9200|::1\]:9200|::ffff:127.0.0.1\]:9200"; then
if echo "$LISTEN_OUTPUT" | grep -qE "0.0.0.0:9200|\*:9200"; then
echo -e "${RED}✗ DANGER: Elasticsearch is bound to ALL interfaces (0.0.0.0)${NC}"
echo -e "${RED} This means it's accessible from the network/internet!${NC}\n"
EXPOSED=true
else
echo -e "${GREEN}✓ SAFE: Elasticsearch is bound to localhost only${NC}"
if echo "$LISTEN_OUTPUT" | grep -q "::ffff:127.0.0.1"; then
echo -e "${GREEN} (Including IPv6-mapped IPv4 localhost)${NC}\n"
else
echo ""
fi
EXPOSED=false
fi
else
if echo "$LISTEN_OUTPUT" | grep -qE "0.0.0.0:9200|\*:9200"; then
echo -e "${RED}✗ DANGER: Elasticsearch is bound to ALL interfaces (0.0.0.0)${NC}"
echo -e "${RED} This means it's accessible from the network/internet!${NC}\n"
EXPOSED=true
else
echo -e "${YELLOW}⚠ WARNING: Elasticsearch might be bound to a specific network interface${NC}\n"
EXPOSED=true
fi
fi
# Check configuration file
echo -e "${BLUE}Checking configuration file...${NC}"
if [ -f /etc/elasticsearch/elasticsearch.yml ]; then
NETWORK_HOST=$(grep "^network.host:" /etc/elasticsearch/elasticsearch.yml 2>/dev/null | awk '{print $2}')
SECURITY_ENABLED=$(grep "^xpack.security.enabled:" /etc/elasticsearch/elasticsearch.yml 2>/dev/null | awk '{print $2}')
if [ -n "$NETWORK_HOST" ]; then
echo -e "network.host: ${YELLOW}$NETWORK_HOST${NC}"
if [ "$NETWORK_HOST" = "127.0.0.1" ] || [ "$NETWORK_HOST" = "localhost" ]; then
echo -e "${GREEN}✓ Configuration: Bound to localhost${NC}"
elif [ "$NETWORK_HOST" = "0.0.0.0" ]; then
echo -e "${RED}✗ Configuration: Bound to ALL interfaces - INSECURE!${NC}"
else
echo -e "${YELLOW}⚠ Configuration: Bound to specific interface${NC}"
fi
else
echo -e "${YELLOW}⚠ network.host not explicitly set (using default)${NC}"
fi
if [ -n "$SECURITY_ENABLED" ]; then
echo -e "xpack.security.enabled: ${YELLOW}$SECURITY_ENABLED${NC}"
if [ "$SECURITY_ENABLED" = "true" ]; then
echo -e "${GREEN}✓ Security: Authentication enabled${NC}"
else
echo -e "${YELLOW}⚠ Security: Authentication disabled${NC}"
if [ "$NETWORK_HOST" = "127.0.0.1" ] || [ "$NETWORK_HOST" = "localhost" ]; then
echo -e "${GREEN} (OK for localhost-only setup)${NC}"
else
echo -e "${RED} (DANGEROUS if accessible from network!)${NC}"
fi
fi
else
echo -e "${YELLOW}⚠ xpack.security.enabled not set (default: false)${NC}"
fi
echo ""
else
echo -e "${RED}✗ Cannot access /etc/elasticsearch/elasticsearch.yml${NC}"
echo -e "${YELLOW} (Run with sudo to read config file)${NC}\n"
fi
# Test local access
echo -e "${BLUE}Testing local access...${NC}"
LOCAL_RESPONSE=$(curl -s http://localhost:9200 2>&1)
if echo "$LOCAL_RESPONSE" | grep -q "\"cluster_name\""; then
echo -e "${GREEN}✓ Accessible locally on localhost:9200${NC}\n"
else
echo -e "${RED}✗ Cannot access locally${NC}\n"
fi
# Get network interfaces
echo -e "${BLUE}Network interfaces:${NC}"
ip addr show | grep "inet " | grep -v "127.0.0.1" | awk '{print " " $2}'
echo ""
# Try to test external access (if we have an external IP)
EXTERNAL_IP=$(ip addr show | grep "inet " | grep -v "127.0.0.1" | head -1 | awk '{print $2}' | cut -d'/' -f1)
if [ -n "$EXTERNAL_IP" ]; then
echo -e "${BLUE}Testing external access from $EXTERNAL_IP...${NC}"
EXTERNAL_RESPONSE=$(curl -s --connect-timeout 2 "http://$EXTERNAL_IP:9200" 2>&1)
if echo "$EXTERNAL_RESPONSE" | grep -q "\"cluster_name\""; then
echo -e "${RED}✗✗✗ DANGER: Elasticsearch IS ACCESSIBLE from $EXTERNAL_IP ✗✗✗${NC}"
echo -e "${RED} Anyone on your network can access your database!${NC}\n"
EXPOSED=true
elif echo "$EXTERNAL_RESPONSE" | grep -q "Connection refused"; then
echo -e "${GREEN}✓ SAFE: Not accessible from $EXTERNAL_IP${NC}\n"
else
echo -e "${YELLOW}⚠ Unable to test (got: $EXTERNAL_RESPONSE)${NC}\n"
fi
fi
# Final verdict
echo -e "${BLUE}==========================================${NC}"
if [ "$EXPOSED" = true ]; then
echo -e "${RED} SECURITY RISK DETECTED!${NC}"
echo -e "${BLUE}==========================================${NC}\n"
echo -e "${RED}Action Required:${NC}"
echo -e "1. Stop Elasticsearch: ${YELLOW}sudo systemctl stop elasticsearch${NC}"
echo -e "2. Edit config: ${YELLOW}sudo nano /etc/elasticsearch/elasticsearch.yml${NC}"
echo -e "3. Set: ${YELLOW}network.host: 127.0.0.1${NC}"
echo -e "4. Set: ${YELLOW}xpack.security.enabled: true${NC}"
echo -e "5. Restart: ${YELLOW}sudo systemctl start elasticsearch${NC}"
echo -e "\nSee ${BLUE}references/ELASTICSEARCH_SETUP.md${NC} for detailed security guide\n"
exit 1
else
echo -e "${GREEN} System appears secure${NC}"
echo -e "${BLUE}==========================================${NC}\n"
if [ "$SECURITY_ENABLED" != "true" ]; then
echo -e "${YELLOW}Recommendation: Enable authentication for additional security${NC}"
echo -e "See ${BLUE}references/ELASTICSEARCH_SETUP.md${NC} for instructions\n"
fi
exit 0
fi
scripts/check-post-visibility.php
Executable
+55
View File
@@ -0,0 +1,55 @@
<?php
require __DIR__.'/vendor/autoload.php';
$app = require_once __DIR__.'/bootstrap/app.php';
$app->make(\Illuminate\Contracts\Console\Kernel::class)->bootstrap();
$posts = App\Models\Post::with('translations', 'category')
->whereIn('category_id', [4,5,6,7,8,113])
->get();
echo "Posts in allowed categories ({$posts->count()} total):\n";
echo str_repeat('=', 100) . "\n";
foreach ($posts as $post) {
$enTranslation = $post->translations->where('locale', 'en')->first();
if (!$enTranslation) {
echo "ID: {$post->id} | Cat: {$post->category_id} | NO ENGLISH TRANSLATION\n";
continue;
}
$now = now();
$fromDate = $enTranslation->from;
$tillDate = $enTranslation->till;
$deletedAt = $enTranslation->deleted_at;
$isVisible = true;
$reason = 'Visible';
// From date MUST exist and be in the past (null = NOT published)
if (!$fromDate) {
$isVisible = false;
$reason = "No publication date (from is null = unpublished)";
} elseif ($now->lt($fromDate)) {
$isVisible = false;
$reason = "Not yet published (from: {$fromDate})";
}
// Till date can be null (never expires) or must be in the future
if ($tillDate && $now->gt($tillDate)) {
$isVisible = false;
$reason = "Publication ended (till: {$tillDate})";
}
// Deleted date can be null (not deleted) or must be in the future
if ($deletedAt && $now->gte($deletedAt)) {
$isVisible = false;
$reason = 'Scheduled deletion';
}
$status = $isVisible ? '✓ VISIBLE' : '✗ HIDDEN';
$title = substr($enTranslation->title, 0, 40);
echo "ID: {$post->id} | Cat: {$post->category_id} | Title: {$title} | Status: {$status} | Reason: {$reason}\n";
}
scripts/cleanup-backups.sh
Executable
+780
View File
@@ -0,0 +1,780 @@
#!/bin/bash
# Laravel Timebank Backup Cleanup Script
# Cleans up old backups based on retention policies and monitors disk usage
# Usage: ./cleanup-backups.sh [options]
# Options: --dry-run, --force, --verbose
set -e # Exit on any error
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
BACKUP_ROOT_DIR="$PROJECT_ROOT/backups"
LOG_FILE="$BACKUP_ROOT_DIR/backup.log"
# Configuration file path
CONFIG_FILE="$SCRIPT_DIR/backup-retention.conf"
# Default retention policies (days) - used as fallback if config file is missing or invalid
DEFAULT_DAILY_RETENTION=7
DEFAULT_WEEKLY_RETENTION=28 # 4 weeks
DEFAULT_MONTHLY_RETENTION=365 # 12 months
DEFAULT_PRE_RESTORE_RETENTION=30
DEFAULT_LOG_RETENTION=30
DEFAULT_FULL_RETENTION=14
# Default count limits - used as fallback
DEFAULT_DAILY_COUNT_LIMIT=7
DEFAULT_WEEKLY_COUNT_LIMIT=4
DEFAULT_MONTHLY_COUNT_LIMIT=12
DEFAULT_PRE_RESTORE_COUNT_LIMIT=5
DEFAULT_SNAPSHOT_COUNT_LIMIT=3
# Whole-storage archives: the largest single files in the tree.
DEFAULT_FULL_COUNT_LIMIT=2
# Default disk usage thresholds
DEFAULT_DISK_WARNING_THRESHOLD=85 # Warn at 85% disk usage
DEFAULT_DISK_CRITICAL_THRESHOLD=95 # Force cleanup at 95% disk usage
# Default settings
DEFAULT_EMAIL_NOTIFICATIONS_ENABLED=true
DEFAULT_CLEANUP_MODE="both"
DEFAULT_CLEANUP_EMPTY_DIRS=true
DEFAULT_VERIFY_BEFORE_DELETE=false
# Function to load configuration with validation and fallbacks
load_config() {
# Set defaults first
DAILY_RETENTION=$DEFAULT_DAILY_RETENTION
WEEKLY_RETENTION=$DEFAULT_WEEKLY_RETENTION
MONTHLY_RETENTION=$DEFAULT_MONTHLY_RETENTION
PRE_RESTORE_RETENTION=$DEFAULT_PRE_RESTORE_RETENTION
LOG_RETENTION=$DEFAULT_LOG_RETENTION
DAILY_COUNT_LIMIT=$DEFAULT_DAILY_COUNT_LIMIT
WEEKLY_COUNT_LIMIT=$DEFAULT_WEEKLY_COUNT_LIMIT
MONTHLY_COUNT_LIMIT=$DEFAULT_MONTHLY_COUNT_LIMIT
PRE_RESTORE_COUNT_LIMIT=$DEFAULT_PRE_RESTORE_COUNT_LIMIT
SNAPSHOT_COUNT_LIMIT=$DEFAULT_SNAPSHOT_COUNT_LIMIT
FULL_COUNT_LIMIT=$DEFAULT_FULL_COUNT_LIMIT
FULL_RETENTION=$DEFAULT_FULL_RETENTION
# Empty = same as MONTHLY_COUNT_LIMIT (see below).
STORAGE_MONTHLY_COUNT_LIMIT=""
DISK_WARNING_THRESHOLD=$DEFAULT_DISK_WARNING_THRESHOLD
DISK_CRITICAL_THRESHOLD=$DEFAULT_DISK_CRITICAL_THRESHOLD
EMAIL_NOTIFICATIONS_ENABLED=$DEFAULT_EMAIL_NOTIFICATIONS_ENABLED
CLEANUP_MODE=$DEFAULT_CLEANUP_MODE
CLEANUP_EMPTY_DIRS=$DEFAULT_CLEANUP_EMPTY_DIRS
VERIFY_BEFORE_DELETE=$DEFAULT_VERIFY_BEFORE_DELETE
# Load config file if it exists
if [ -f "$CONFIG_FILE" ]; then
log "INFO" "Loading retention configuration from: $CONFIG_FILE"
# Source the config file with error handling
if source "$CONFIG_FILE" 2>/dev/null; then
log "INFO" "Configuration loaded successfully"
# Validate numeric values and reset to defaults if invalid
validate_numeric "DAILY_RETENTION" "$DEFAULT_DAILY_RETENTION" 1 365
validate_numeric "WEEKLY_RETENTION" "$DEFAULT_WEEKLY_RETENTION" 1 730
validate_numeric "MONTHLY_RETENTION" "$DEFAULT_MONTHLY_RETENTION" 1 2555 # ~7 years
validate_numeric "PRE_RESTORE_RETENTION" "$DEFAULT_PRE_RESTORE_RETENTION" 1 365
validate_numeric "LOG_RETENTION" "$DEFAULT_LOG_RETENTION" 1 365
validate_numeric "DAILY_COUNT_LIMIT" "$DEFAULT_DAILY_COUNT_LIMIT" 1 100
validate_numeric "WEEKLY_COUNT_LIMIT" "$DEFAULT_WEEKLY_COUNT_LIMIT" 1 100
validate_numeric "MONTHLY_COUNT_LIMIT" "$DEFAULT_MONTHLY_COUNT_LIMIT" 1 100
validate_numeric "PRE_RESTORE_COUNT_LIMIT" "$DEFAULT_PRE_RESTORE_COUNT_LIMIT" 1 50
validate_numeric "SNAPSHOT_COUNT_LIMIT" "$DEFAULT_SNAPSHOT_COUNT_LIMIT" 1 20
validate_numeric "FULL_COUNT_LIMIT" "$DEFAULT_FULL_COUNT_LIMIT" 1 20
# Monthly storage archives (~490MB each) can be kept fewer than
# monthly database dumps (~5MB each), which are the history worth
# keeping. Unset means the shared MONTHLY_COUNT_LIMIT.
if [ -z "$STORAGE_MONTHLY_COUNT_LIMIT" ]; then
STORAGE_MONTHLY_COUNT_LIMIT=$MONTHLY_COUNT_LIMIT
fi
validate_numeric "STORAGE_MONTHLY_COUNT_LIMIT" "$MONTHLY_COUNT_LIMIT" 1 100
validate_numeric "FULL_RETENTION" "$DEFAULT_FULL_RETENTION" 1 365
validate_numeric "DISK_WARNING_THRESHOLD" "$DEFAULT_DISK_WARNING_THRESHOLD" 50 99
validate_numeric "DISK_CRITICAL_THRESHOLD" "$DEFAULT_DISK_CRITICAL_THRESHOLD" 60 100
# Ensure critical threshold is higher than warning threshold
if [ $DISK_CRITICAL_THRESHOLD -le $DISK_WARNING_THRESHOLD ]; then
log "WARNING" "Critical threshold ($DISK_CRITICAL_THRESHOLD%) must be higher than warning threshold ($DISK_WARNING_THRESHOLD%). Resetting to defaults."
DISK_WARNING_THRESHOLD=$DEFAULT_DISK_WARNING_THRESHOLD
DISK_CRITICAL_THRESHOLD=$DEFAULT_DISK_CRITICAL_THRESHOLD
fi
# Validate boolean values
validate_boolean "EMAIL_NOTIFICATIONS_ENABLED" "$DEFAULT_EMAIL_NOTIFICATIONS_ENABLED"
validate_boolean "CLEANUP_EMPTY_DIRS" "$DEFAULT_CLEANUP_EMPTY_DIRS"
validate_boolean "VERIFY_BEFORE_DELETE" "$DEFAULT_VERIFY_BEFORE_DELETE"
# Validate cleanup mode
if [[ ! "$CLEANUP_MODE" =~ ^(age_only|count_only|both)$ ]]; then
log "WARNING" "Invalid CLEANUP_MODE: $CLEANUP_MODE. Using default: $DEFAULT_CLEANUP_MODE"
CLEANUP_MODE=$DEFAULT_CLEANUP_MODE
fi
else
log "WARNING" "Error loading config file. Using default values."
fi
else
log "INFO" "Config file not found: $CONFIG_FILE. Using default retention policies."
log "INFO" "Run: cp $CONFIG_FILE.example $CONFIG_FILE to create a customizable config file."
fi
}
# Function to validate numeric configuration values
validate_numeric() {
local var_name="$1"
local default_val="$2"
local min_val="$3"
local max_val="$4"
local current_val
eval "current_val=\$$var_name"
# Check if it's a valid number
if ! [[ "$current_val" =~ ^[0-9]+$ ]]; then
log "WARNING" "Invalid $var_name: '$current_val'. Using default: $default_val"
eval "$var_name=$default_val"
return
fi
# Check range
if [ "$current_val" -lt "$min_val" ] || [ "$current_val" -gt "$max_val" ]; then
log "WARNING" "$var_name ($current_val) out of range ($min_val-$max_val). Using default: $default_val"
eval "$var_name=$default_val"
fi
}
# Function to validate boolean configuration values
validate_boolean() {
local var_name="$1"
local default_val="$2"
local current_val
eval "current_val=\$$var_name"
# Normalize to lowercase
current_val=$(echo "$current_val" | tr '[:upper:]' '[:lower:]')
case "$current_val" in
true|yes|1|on)
eval "$var_name=true"
;;
false|no|0|off)
eval "$var_name=false"
;;
*)
log "WARNING" "Invalid $var_name: '$current_val'. Using default: $default_val"
eval "$var_name=$default_val"
;;
esac
}
# Create log directory
mkdir -p "$BACKUP_ROOT_DIR/logs"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Logging function
log() {
local level="$1"
local message="$2"
local timestamp="[$(date '+%Y-%m-%d %H:%M:%S')]"
case "$level" in
"INFO")
echo -e "${timestamp} ${BLUE}[INFO]${NC} $message" | tee -a "$LOG_FILE"
;;
"SUCCESS")
echo -e "${timestamp} ${GREEN}[SUCCESS]${NC} $message" | tee -a "$LOG_FILE"
;;
"WARNING")
echo -e "${timestamp} ${YELLOW}[WARNING]${NC} $message" | tee -a "$LOG_FILE"
;;
"ERROR")
echo -e "${timestamp} ${RED}[ERROR]${NC} $message" | tee -a "$LOG_FILE"
;;
*)
echo "$timestamp $message" | tee -a "$LOG_FILE"
;;
esac
}
# Function to send email notifications
send_notification() {
local subject="$1"
local message="$2"
# Check if email notifications are enabled
if [ "$EMAIL_NOTIFICATIONS_ENABLED" != "true" ]; then
log "INFO" "Email notifications disabled in config, skipping: $subject"
return 0
fi
if command -v mail >/dev/null 2>&1; then
echo "$message" | mail -s "$subject" "${BACKUP_NOTIFY_EMAIL:-$USER@localhost}" 2>/dev/null || true
log "INFO" "Email notification sent: $subject"
else
log "WARNING" "Mail command not available, notification not sent"
fi
}
# Function to show usage
show_usage() {
# Load config to show current values
load_config
echo "Usage: $0 [options]"
echo ""
echo "Options:"
echo " --dry-run - Show what would be deleted without actually deleting"
echo " --force - Force cleanup even if disk usage is not critical"
echo " --verbose - Show detailed information"
echo " --help - Show this help message"
echo ""
echo "Current Configuration:"
echo " Config file: $CONFIG_FILE"
if [ -f "$CONFIG_FILE" ]; then
echo " Config status: Loaded"
else
echo " Config status: Using defaults (file not found)"
fi
echo ""
echo "Time-based Retention (days):"
echo " Daily backups: $DAILY_RETENTION days"
echo " Weekly backups: $WEEKLY_RETENTION days"
echo " Monthly backups: $MONTHLY_RETENTION days"
echo " Pre-restore: $PRE_RESTORE_RETENTION days"
echo " Log files: $LOG_RETENTION days"
echo ""
echo "Count-based Retention (keep N most recent):"
echo " Daily backups: $DAILY_COUNT_LIMIT files"
echo " Weekly backups: $WEEKLY_COUNT_LIMIT files"
echo " Monthly backups: $MONTHLY_COUNT_LIMIT files (database)"
echo " Monthly storage: ${STORAGE_MONTHLY_COUNT_LIMIT:-$MONTHLY_COUNT_LIMIT} files"
echo " Pre-restore: $PRE_RESTORE_COUNT_LIMIT files"
echo " Storage snapshots: $SNAPSHOT_COUNT_LIMIT files"
echo " Full storage: $FULL_COUNT_LIMIT files"
echo ""
echo "Disk Space Thresholds:"
echo " Warning threshold: $DISK_WARNING_THRESHOLD%"
echo " Critical threshold: $DISK_CRITICAL_THRESHOLD%"
echo ""
echo "Settings:"
echo " Email notifications: $EMAIL_NOTIFICATIONS_ENABLED"
echo " Cleanup mode: $CLEANUP_MODE"
echo " Clean empty dirs: $CLEANUP_EMPTY_DIRS"
echo " Verify before delete: $VERIFY_BEFORE_DELETE"
echo ""
echo "To customize retention policies:"
echo " Edit: $CONFIG_FILE"
echo ""
exit 0
}
# Function to check disk usage
check_disk_usage() {
local backup_dir="$1"
local disk_usage=$(df "$backup_dir" | awk 'NR==2 {print substr($5,1,length($5)-1)}')
echo "$disk_usage"
}
# Function to get human readable size
get_size() {
local path="$1"
if [ -f "$path" ]; then
du -h "$path" | cut -f1
elif [ -d "$path" ]; then
du -sh "$path" | cut -f1
else
echo "0B"
fi
}
# Function to cleanup old backups by age
cleanup_by_age() {
local backup_dir="$1"
local retention_days="$2"
local backup_type="$3"
local pattern="$4"
local dry_run="$5"
if [ ! -d "$backup_dir" ]; then
return 0
fi
log "INFO" "Cleaning up $backup_type backups older than $retention_days days"
# Never let age-based cleanup empty a directory. If every archive in here
# is older than the retention window - a backup job that stopped running,
# a restore from an old snapshot - deleting them all would leave no backup
# at all, which is far worse than keeping a stale one. The count-based
# pass that follows still trims down to its own limit.
local total_matching
total_matching=$(find "$backup_dir" -maxdepth 1 -mindepth 1 -name "$pattern" \
\( -type f -o -type d \) 2>/dev/null | wc -l)
local older_matching
older_matching=$(find "$backup_dir" -maxdepth 1 -mindepth 1 -name "$pattern" \
\( -type f -o -type d \) -mtime "+$retention_days" 2>/dev/null | wc -l)
if [ "$total_matching" -gt 0 ] && [ "$older_matching" -ge "$total_matching" ]; then
log "WARNING" "$backup_type: all $total_matching backups are older than $retention_days days; keeping them so this directory is not left empty"
return 0
fi
local files_found=0
local files_deleted=0
local total_size_deleted=0
# Find files older than retention period
while IFS= read -r -d '' backup_file; do
files_found=$((files_found + 1))
local file_size_kb=$(du -k "$backup_file" | cut -f1)
local file_size_hr=$(get_size "$backup_file")
if [ "$dry_run" = true ]; then
if [ "$VERBOSE" = true ]; then
log "INFO" "Would delete: $(basename "$backup_file") ($file_size_hr)"
fi
else
if [ "$VERBOSE" = true ]; then
log "INFO" "Deleting: $(basename "$backup_file") ($file_size_hr)"
fi
rm "$backup_file"
files_deleted=$((files_deleted + 1))
total_size_deleted=$((total_size_deleted + file_size_kb))
fi
done < <(find "$backup_dir" -name "$pattern" -type f -mtime +$retention_days -print0 2>/dev/null)
if [ $files_found -gt 0 ]; then
local total_size_hr=$(echo "$total_size_deleted" | awk '{printf "%.1fMB", $1/1024}')
if [ "$dry_run" = true ]; then
log "INFO" "$backup_type: Would delete $files_found files ($total_size_hr)"
else
log "SUCCESS" "$backup_type: Deleted $files_deleted files ($total_size_hr)"
fi
else
log "INFO" "$backup_type: No old files to clean up"
fi
return 0
}
# Function to cleanup by count (keep only N most recent)
cleanup_by_count() {
local backup_dir="$1"
local keep_count="$2"
local backup_type="$3"
local pattern="$4"
local dry_run="$5"
if [ ! -d "$backup_dir" ]; then
return 0
fi
log "INFO" "Ensuring only $keep_count most recent $backup_type backups are kept"
# -maxdepth 1 and "! -path" so only direct children are considered: a
# snapshot entry is a DIRECTORY, and without the depth limit find would
# descend into it and match the files inside instead.
#
# ( -type f -o -type d ) rather than -type f alone: storage/snapshots/*
# entries are directories, so the old -type f could never match them and
# 30 snapshots accumulated against a limit of 3.
#
# Sorted by mtime (newest first) rather than by name: snapshot directory
# names are not guaranteed to sort chronologically, and keeping the
# "newest N" is what the retention policy actually means.
local files_to_delete=()
while IFS= read -r line; do
[ -n "$line" ] && files_to_delete+=("${line#* }")
done < <(find "$backup_dir" -maxdepth 1 -mindepth 1 -name "$pattern" \
\( -type f -o -type d \) -printf '%T@ %p\n' \
| sort -rn | tail -n +$((keep_count + 1)))
if [ ${#files_to_delete[@]} -gt 0 ]; then
local total_size_deleted=0
for backup_file in "${files_to_delete[@]}"; do
# -s so a directory reports its whole tree, not one line per file.
local file_size_kb=$(du -sk "$backup_file" | cut -f1)
local file_size_hr=$(du -sh "$backup_file" | cut -f1)
# Counted in both modes: the total was previously only summed on
# the real path, so --dry-run always reported 0.0MB and looked as
# though there was nothing to reclaim.
total_size_deleted=$((total_size_deleted + file_size_kb))
if [ "$dry_run" = true ]; then
if [ "$VERBOSE" = true ]; then
log "INFO" "Would delete (count): $(basename "$backup_file") ($file_size_hr)"
fi
else
if [ "$VERBOSE" = true ]; then
log "INFO" "Deleting (count): $(basename "$backup_file") ($file_size_hr)"
fi
# -r because a snapshot entry is a directory; -f so a
# vanished file (a concurrent run, a manual delete) is not
# a fatal error under set -e.
rm -rf "$backup_file"
fi
done
local total_size_hr=$(echo "$total_size_deleted" | awk '{printf "%.1fMB", $1/1024}')
if [ "$dry_run" = true ]; then
log "INFO" "$backup_type: Would delete ${#files_to_delete[@]} excess files ($total_size_hr)"
else
log "SUCCESS" "$backup_type: Deleted ${#files_to_delete[@]} excess files ($total_size_hr)"
fi
else
log "INFO" "$backup_type: No excess files to clean up"
fi
return 0
}
# Function to clean up empty directories
cleanup_empty_dirs() {
local backup_root="$1"
local dry_run="$2"
log "INFO" "Cleaning up empty directories"
local empty_dirs=($(find "$backup_root" -type d -empty 2>/dev/null))
if [ ${#empty_dirs[@]} -gt 0 ]; then
for empty_dir in "${empty_dirs[@]}"; do
# Skip the root backup directories
if [[ "$empty_dir" =~ (database|storage|logs)$ ]]; then
continue
fi
if [ "$dry_run" = true ]; then
if [ "$VERBOSE" = true ]; then
log "INFO" "Would remove empty directory: $empty_dir"
fi
else
if [ "$VERBOSE" = true ]; then
log "INFO" "Removing empty directory: $empty_dir"
fi
rmdir "$empty_dir" 2>/dev/null || true
fi
done
if [ "$dry_run" = true ]; then
log "INFO" "Would remove ${#empty_dirs[@]} empty directories"
else
log "SUCCESS" "Removed empty directories"
fi
else
log "INFO" "No empty directories to clean up"
fi
}
# Function to generate cleanup report
generate_report() {
local backup_root="$1"
log "INFO" "Backup storage report:"
if [ -d "$backup_root" ]; then
# Database backups
if [ -d "$backup_root/database" ]; then
local db_count=$(find "$backup_root/database" -name "*.sql*" -type f | wc -l)
local db_size=$(get_size "$backup_root/database")
log "INFO" " Database backups: $db_count files ($db_size)"
fi
# Storage backups
if [ -d "$backup_root/storage" ]; then
local storage_count=$(find "$backup_root/storage" -name "*.tar.gz" -type f | wc -l)
local storage_size=$(get_size "$backup_root/storage")
log "INFO" " Storage backups: $storage_count files ($storage_size)"
fi
# Total size
local total_size=$(get_size "$backup_root")
log "INFO" " Total backup size: $total_size"
# Disk usage
local disk_usage=$(check_disk_usage "$backup_root")
local available_space=$(df -h "$backup_root" | awk 'NR==2 {print $4}')
log "INFO" " Disk usage: ${disk_usage}% (${available_space} available)"
# Oldest and newest backups (check both .sql* and .gz files)
local oldest_backup=$(find "$backup_root" \( -name "*.gz" -o -name "*.sql" \) -type f -printf '%T@ %p\n' 2>/dev/null | sort -n | head -n 1 | cut -d' ' -f2-)
local newest_backup=$(find "$backup_root" \( -name "*.gz" -o -name "*.sql" \) -type f -printf '%T@ %p\n' 2>/dev/null | sort -n | tail -n 1 | cut -d' ' -f2-)
if [ -n "$oldest_backup" ]; then
local oldest_date=$(date -r "$oldest_backup" '+%Y-%m-%d %H:%M:%S')
log "INFO" " Oldest backup: $(basename "$oldest_backup") ($oldest_date)"
fi
if [ -n "$newest_backup" ]; then
local newest_date=$(date -r "$newest_backup" '+%Y-%m-%d %H:%M:%S')
log "INFO" " Newest backup: $(basename "$newest_backup") ($newest_date)"
fi
else
log "WARNING" "No backup directory found"
fi
}
# Parse command line arguments
DRY_RUN=false
FORCE=false
VERBOSE=false
while [[ $# -gt 0 ]]; do
case $1 in
--dry-run)
DRY_RUN=true
shift
;;
--force)
FORCE=true
shift
;;
--verbose)
VERBOSE=true
shift
;;
--help)
show_usage
;;
*)
log "ERROR" "Unknown option: $1"
show_usage
;;
esac
done
# Main execution
main() {
# Load configuration before starting
load_config
log "INFO" "============================================"
if [ "$DRY_RUN" = true ]; then
log "INFO" "Starting backup cleanup (DRY RUN)"
else
log "INFO" "Starting backup cleanup"
fi
log "INFO" "Time: $(date)"
log "INFO" "============================================"
local start_time=$(date +%s)
# Rotate backup.log if it exceeds 5 MB
if [ -f "$LOG_FILE" ] && [ $(stat -c%s "$LOG_FILE" 2>/dev/null || echo 0) -gt 5242880 ]; then
mv "$LOG_FILE" "${LOG_FILE}.1"
# Keep only the two most recent log files
rm -f "${LOG_FILE}.2"
mv -f "${LOG_FILE}.1" "${LOG_FILE}.old" 2>/dev/null || true
log "INFO" "backup.log rotated (exceeded 5 MB)"
fi
# Check if backup directory exists
if [ ! -d "$BACKUP_ROOT_DIR" ]; then
log "WARNING" "Backup directory not found: $BACKUP_ROOT_DIR"
exit 0
fi
# Check disk usage
local disk_usage=$(check_disk_usage "$BACKUP_ROOT_DIR")
log "INFO" "Current disk usage: ${disk_usage}%"
local should_cleanup=false
if [ $disk_usage -ge $DISK_CRITICAL_THRESHOLD ]; then
log "WARNING" "Disk usage critical (${disk_usage}% >= ${DISK_CRITICAL_THRESHOLD}%), forcing cleanup"
send_notification "Timebank Cleanup Critical - Aggressive Cleanup Started" \
"CRITICAL: Disk usage has reached ${disk_usage}% (>= ${DISK_CRITICAL_THRESHOLD}%)
Aggressive backup cleanup has been automatically triggered to free disk space.
Location: $BACKUP_ROOT_DIR
Current usage: ${disk_usage}%
Critical threshold: ${DISK_CRITICAL_THRESHOLD}%
Time: $(date)
This is an automated cleanup to prevent disk space issues. Please check the system if this occurs frequently."
should_cleanup=true
elif [ $disk_usage -ge $DISK_WARNING_THRESHOLD ]; then
log "WARNING" "Disk usage high (${disk_usage}% >= ${DISK_WARNING_THRESHOLD}%)"
send_notification "Timebank Cleanup Warning - Low Disk Space" \
"WARNING: Disk usage is getting high at ${disk_usage}% (>= ${DISK_WARNING_THRESHOLD}%)
Backup cleanup will be performed to prevent potential disk space issues.
Location: $BACKUP_ROOT_DIR
Current usage: ${disk_usage}%
Warning threshold: ${DISK_WARNING_THRESHOLD}%
Critical threshold: ${DISK_CRITICAL_THRESHOLD}%
Time: $(date)
Please monitor disk usage and consider expanding storage if warnings occur frequently."
should_cleanup=true
elif [ "$FORCE" = true ]; then
log "INFO" "Force cleanup requested"
should_cleanup=true
else
log "INFO" "Disk usage normal, running standard cleanup"
should_cleanup=true
fi
if [ "$should_cleanup" = true ]; then
# Cleanup database backups
if [ -d "$BACKUP_ROOT_DIR/database" ]; then
# Load environment for database name
if [ -f "$PROJECT_ROOT/.env" ]; then
source "$SCRIPT_DIR/load-env.sh"
load_env "$PROJECT_ROOT/.env"
fi
# Match both compressed (.sql.gz) and uncompressed (.sql) backups
local db_pattern="*.sql*"
if [ -n "$DB_DATABASE" ]; then
db_pattern="${DB_DATABASE}_*.sql*"
fi
local pre_restore_pattern="*.sql*"
cleanup_by_age "$BACKUP_ROOT_DIR/database/daily" $DAILY_RETENTION "daily database" "$db_pattern" "$DRY_RUN"
cleanup_by_age "$BACKUP_ROOT_DIR/database/weekly" $WEEKLY_RETENTION "weekly database" "$db_pattern" "$DRY_RUN"
cleanup_by_age "$BACKUP_ROOT_DIR/database/monthly" $MONTHLY_RETENTION "monthly database" "$db_pattern" "$DRY_RUN"
cleanup_by_age "$BACKUP_ROOT_DIR/database/pre-restore" $PRE_RESTORE_RETENTION "pre-restore database" "$pre_restore_pattern" "$DRY_RUN"
# Also enforce count limits
cleanup_by_count "$BACKUP_ROOT_DIR/database/daily" $DAILY_COUNT_LIMIT "daily database" "$db_pattern" "$DRY_RUN"
cleanup_by_count "$BACKUP_ROOT_DIR/database/weekly" $WEEKLY_COUNT_LIMIT "weekly database" "$db_pattern" "$DRY_RUN"
cleanup_by_count "$BACKUP_ROOT_DIR/database/monthly" $MONTHLY_COUNT_LIMIT "monthly database" "$db_pattern" "$DRY_RUN"
cleanup_by_count "$BACKUP_ROOT_DIR/database/pre-restore" $PRE_RESTORE_COUNT_LIMIT "pre-restore database" "$pre_restore_pattern" "$DRY_RUN"
fi
# Cleanup storage backups
if [ -d "$BACKUP_ROOT_DIR/storage" ]; then
cleanup_by_age "$BACKUP_ROOT_DIR/storage/daily" $DAILY_RETENTION "daily storage" "*.tar.gz" "$DRY_RUN"
cleanup_by_age "$BACKUP_ROOT_DIR/storage/weekly" $WEEKLY_RETENTION "weekly storage" "*.tar.gz" "$DRY_RUN"
cleanup_by_age "$BACKUP_ROOT_DIR/storage/monthly" $MONTHLY_RETENTION "monthly storage" "*.tar.gz" "$DRY_RUN"
cleanup_by_age "$BACKUP_ROOT_DIR/storage/pre-restore" $PRE_RESTORE_RETENTION "pre-restore storage" "*.tar.gz" "$DRY_RUN"
# Count limits for storage
cleanup_by_count "$BACKUP_ROOT_DIR/storage/daily" $DAILY_COUNT_LIMIT "daily storage" "*.tar.gz" "$DRY_RUN"
cleanup_by_count "$BACKUP_ROOT_DIR/storage/weekly" $WEEKLY_COUNT_LIMIT "weekly storage" "*.tar.gz" "$DRY_RUN"
cleanup_by_count "$BACKUP_ROOT_DIR/storage/monthly" ${STORAGE_MONTHLY_COUNT_LIMIT:-$MONTHLY_COUNT_LIMIT} "monthly storage" "*.tar.gz" "$DRY_RUN"
# storage/full had no rule at all until 2026-09-22 and had grown
# to 979M untouched. These are whole-storage archives, so a small
# count limit matters more here than anywhere else.
cleanup_by_age "$BACKUP_ROOT_DIR/storage/full" $FULL_RETENTION "full storage" "*.tar.gz" "$DRY_RUN"
cleanup_by_count "$BACKUP_ROOT_DIR/storage/full" $FULL_COUNT_LIMIT "full storage" "*.tar.gz" "$DRY_RUN"
# Cleanup old snapshots
cleanup_by_count "$BACKUP_ROOT_DIR/storage/snapshots" $SNAPSHOT_COUNT_LIMIT "storage snapshots" "*" "$DRY_RUN"
fi
# Cleanup empty directories
cleanup_empty_dirs "$BACKUP_ROOT_DIR" "$DRY_RUN"
# Cleanup old log files
if [ -d "$BACKUP_ROOT_DIR/logs" ]; then
cleanup_by_age "$BACKUP_ROOT_DIR/logs" $LOG_RETENTION "log files" "*.log" "$DRY_RUN"
fi
fi
# Generate final report
generate_report "$BACKUP_ROOT_DIR"
# Calculate execution time
local end_time=$(date +%s)
local execution_time=$((end_time - start_time))
local execution_time_formatted=$(date -d@$execution_time -u +%H:%M:%S)
# Send completion summary email if cleanup was triggered by disk space conditions
local final_disk_usage=$(check_disk_usage "$BACKUP_ROOT_DIR")
if [ "$should_cleanup" = true ] && ([ $disk_usage -ge $DISK_WARNING_THRESHOLD ] || [ "$FORCE" = true ]); then
local cleanup_type="Standard"
if [ $disk_usage -ge $DISK_CRITICAL_THRESHOLD ]; then
cleanup_type="Aggressive (Critical)"
elif [ $disk_usage -ge $DISK_WARNING_THRESHOLD ]; then
cleanup_type="Warning Level"
elif [ "$FORCE" = true ]; then
cleanup_type="Manual Force"
fi
# Get backup statistics
local db_count=0
local db_size="0B"
local storage_count=0
local storage_size="0B"
local total_size="Unknown"
if [ -d "$BACKUP_ROOT_DIR/database" ]; then
db_count=$(find "$BACKUP_ROOT_DIR/database" -name "*.sql.gz" 2>/dev/null | wc -l)
db_size=$(get_size "$BACKUP_ROOT_DIR/database")
fi
if [ -d "$BACKUP_ROOT_DIR/storage" ]; then
storage_count=$(find "$BACKUP_ROOT_DIR/storage" -name "*.tar.gz" 2>/dev/null | wc -l)
storage_size=$(get_size "$BACKUP_ROOT_DIR/storage")
fi
if [ -d "$BACKUP_ROOT_DIR" ]; then
total_size=$(get_size "$BACKUP_ROOT_DIR")
fi
local mode_text="cleanup completed"
if [ "$DRY_RUN" = true ]; then
mode_text="cleanup analysis completed (DRY RUN)"
fi
send_notification "Timebank Backup Cleanup Completed" \
"Backup $mode_text successfully.
Cleanup Details:
- Type: $cleanup_type cleanup
- Duration: $execution_time_formatted
- Initial disk usage: ${disk_usage}%
- Final disk usage: ${final_disk_usage}%
Current Backup Status:
- Database backups: $db_count files ($db_size)
- Storage backups: $storage_count files ($storage_size)
- Total backup size: $total_size
Location: $BACKUP_ROOT_DIR
Completed: $(date)
For detailed logs, check: $LOG_FILE"
fi
if [ "$DRY_RUN" = true ]; then
log "SUCCESS" "Backup cleanup analysis completed in $execution_time_formatted"
else
log "SUCCESS" "Backup cleanup completed in $execution_time_formatted"
fi
log "INFO" "============================================"
log "INFO" "Cleanup process finished"
log "INFO" "============================================"
}
# Run main function
main
scripts/create-restricted-db-user-safe.sh
Executable
+351
View File
@@ -0,0 +1,351 @@
#!/bin/bash
#
# Create Restricted Database User - Safe Script
#
# This script creates a database user with restricted permissions to enforce
# transaction immutability. The user can:
# - SELECT and INSERT on all tables
# - UPDATE and DELETE on all tables EXCEPT transactions and transaction_types
#
# This enforces financial transaction immutability at the database level.
#
# Usage:
# ./scripts/create-restricted-db-user-safe.sh [username] [password]
#
# If username/password not provided, will prompt interactively.
set -e # Exit on error
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
echo -e "${BLUE}========================================${NC}"
echo -e "${BLUE}Create Restricted Database User${NC}"
echo -e "${BLUE}========================================${NC}"
echo ""
# Load database name from .env
if [ ! -f .env ]; then
echo -e "${RED}Error: .env file not found${NC}"
exit 1
fi
DB_DATABASE=$(grep "^DB_DATABASE=" .env | cut -d '=' -f2 | sed 's/#.*//' | tr -d '"' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | sed 's/\r$//')
DB_HOST=$(grep "^DB_HOST=" .env | cut -d '=' -f2 | sed 's/#.*//' | tr -d '"' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | sed 's/\r$//')
# Default to localhost if not set
if [ -z "$DB_HOST" ]; then
DB_HOST="localhost"
fi
echo -e "${BLUE}Database:${NC} $DB_DATABASE"
echo -e "${BLUE}Host:${NC} $DB_HOST"
echo ""
# Determine authentication method
USE_SUDO=false
ROOT_USER="root"
ROOT_PASSWORD=""
# If running as root, try socket authentication first
if [ "$EUID" -eq 0 ]; then
echo -e "${YELLOW}Running as root user, testing socket authentication...${NC}"
if mysql -e "SELECT 1;" &> /dev/null; then
echo -e "${GREEN}✓ Socket authentication successful${NC}"
USE_SUDO=true
else
echo -e "${YELLOW}Socket authentication failed, falling back to credential-based auth${NC}"
fi
fi
# If not using socket auth, get credentials
if [ "$USE_SUDO" = false ]; then
echo ""
echo -e "${YELLOW}Please provide MySQL root credentials to create the restricted user:${NC}"
read -p "Root username (default: root): " ROOT_USER_INPUT
ROOT_USER=${ROOT_USER_INPUT:-root}
read -s -p "Root password: " ROOT_PASSWORD
echo ""
# Test root connection
echo -e "${YELLOW}Testing database connection...${NC}"
if ! MYSQL_PWD="$ROOT_PASSWORD" mysql -h"$DB_HOST" -u"$ROOT_USER" -e "SELECT 1;" 2>/dev/null; then
echo -e "${RED}Error: Cannot connect to database with provided credentials${NC}"
echo ""
echo -e "${YELLOW}Troubleshooting:${NC}"
echo -e " 1. Check your MySQL root password"
echo -e " 2. Verify MySQL is running: ${BLUE}systemctl status mysql${NC}"
echo -e " 3. Check MySQL is listening on ${DB_HOST}: ${BLUE}netstat -tlnp | grep mysql${NC}"
echo -e " 4. Verify user '${ROOT_USER}' can connect from '${DB_HOST}'"
echo ""
echo -e "${YELLOW}Debug: Attempting to connect as:${NC}"
echo -e " User: ${ROOT_USER}"
echo -e " Host: ${DB_HOST}"
exit 1
fi
echo -e "${GREEN}✓ Database connection successful${NC}"
fi
echo ""
# Check if user has CREATE USER privilege
echo -e "${YELLOW}Checking for CREATE USER privilege...${NC}"
if [ "$USE_SUDO" = true ]; then
HAS_CREATE_USER=$(mysql -N -e "SELECT COUNT(*) FROM information_schema.user_privileges WHERE PRIVILEGE_TYPE='CREATE USER' AND GRANTEE LIKE '%root%';" 2>/dev/null || echo "0")
else
HAS_CREATE_USER=$(MYSQL_PWD="$ROOT_PASSWORD" mysql -h"$DB_HOST" -u"$ROOT_USER" -N -e "SELECT COUNT(*) FROM information_schema.user_privileges WHERE PRIVILEGE_TYPE='CREATE USER' AND GRANTEE LIKE '%${ROOT_USER}%';" 2>/dev/null || echo "0")
fi
if [ "$HAS_CREATE_USER" = "0" ]; then
echo -e "${RED}Error: User '${ROOT_USER}' does not have CREATE USER privilege${NC}"
echo ""
echo -e "${YELLOW}Solutions:${NC}"
echo ""
echo -e " ${BLUE}Option 1: Use sudo mysql (recommended for servers)${NC}"
echo -e " If your server uses socket authentication for root, run:"
echo -e " ${GREEN}sudo ./scripts/create-restricted-db-user-safe.sh${NC}"
echo ""
echo -e " ${BLUE}Option 2: Grant CREATE USER privilege${NC}"
echo -e " Connect as a user with GRANT privilege and run:"
echo -e " ${GREEN}GRANT CREATE USER ON *.* TO '${ROOT_USER}'@'${DB_HOST}';${NC}"
echo -e " ${GREEN}FLUSH PRIVILEGES;${NC}"
echo ""
echo -e " ${BLUE}Option 3: Use MySQL root user${NC}"
echo -e " On Ubuntu/Debian servers, try running without password:"
echo -e " ${GREEN}sudo mysql -u root${NC}"
echo -e " Then manually run the SQL commands from:"
echo -e " ${GREEN}scripts/create-restricted-db-user.sql${NC}"
echo ""
exit 1
fi
echo -e "${GREEN}✓ CREATE USER privilege confirmed${NC}"
echo ""
# Get new user details
if [ -z "$1" ]; then
read -p "Enter new restricted username (default: ${DB_DATABASE}_dev): " NEW_USER
NEW_USER=${NEW_USER:-${DB_DATABASE}_dev}
else
NEW_USER="$1"
fi
if [ -z "$2" ]; then
echo -e "${YELLOW}Generate strong password for ${NEW_USER}? (y/n)${NC}"
read -p "> " GENERATE_PASSWORD
if [[ "$GENERATE_PASSWORD" =~ ^[Yy]$ ]]; then
# Generate strong random password
NEW_PASSWORD=$(openssl rand -base64 32 | tr -d "=+/" | cut -c1-32)
echo -e "${GREEN}Generated password: ${NEW_PASSWORD}${NC}"
else
read -s -p "Enter password for ${NEW_USER}: " NEW_PASSWORD
echo ""
read -s -p "Confirm password: " NEW_PASSWORD_CONFIRM
echo ""
if [ "$NEW_PASSWORD" != "$NEW_PASSWORD_CONFIRM" ]; then
echo -e "${RED}Error: Passwords do not match${NC}"
exit 1
fi
fi
else
NEW_PASSWORD="$2"
fi
echo ""
echo -e "${BLUE}Creating user:${NC} ${NEW_USER}@${DB_HOST}"
echo -e "${BLUE}For database:${NC} ${DB_DATABASE}"
echo ""
# Check if user already exists
if [ "$USE_SUDO" = true ]; then
USER_EXISTS=$(mysql -N -e "SELECT COUNT(*) FROM mysql.user WHERE user='$NEW_USER' AND host='$DB_HOST';" 2>/dev/null || echo "0")
else
USER_EXISTS=$(MYSQL_PWD="$ROOT_PASSWORD" mysql -h"$DB_HOST" -u"$ROOT_USER" -N -e "SELECT COUNT(*) FROM mysql.user WHERE user='$NEW_USER' AND host='$DB_HOST';" 2>/dev/null || echo "0")
fi
if [ "$USER_EXISTS" != "0" ]; then
echo -e "${YELLOW}Warning: User '${NEW_USER}'@'${DB_HOST}' already exists${NC}"
read -p "Drop and recreate user? (y/n): " RECREATE
if [[ "$RECREATE" =~ ^[Yy]$ ]]; then
echo -e "${YELLOW}Dropping existing user...${NC}"
if [ "$USE_SUDO" = true ]; then
mysql <<EOF
DROP USER IF EXISTS '${NEW_USER}'@'${DB_HOST}';
DROP USER IF EXISTS '${NEW_USER}'@'localhost';
DROP USER IF EXISTS '${NEW_USER}'@'127.0.0.1';
FLUSH PRIVILEGES;
EOF
else
MYSQL_PWD="$ROOT_PASSWORD" mysql -h"$DB_HOST" -u"$ROOT_USER" <<EOF
DROP USER IF EXISTS '${NEW_USER}'@'${DB_HOST}';
DROP USER IF EXISTS '${NEW_USER}'@'localhost';
DROP USER IF EXISTS '${NEW_USER}'@'127.0.0.1';
FLUSH PRIVILEGES;
EOF
fi
echo -e "${GREEN}✓ Existing user dropped${NC}"
else
echo "Exiting without changes"
exit 0
fi
fi
echo ""
echo -e "${YELLOW}Creating restricted database user...${NC}"
echo ""
# Get all tables
if [ "$USE_SUDO" = true ]; then
ALL_TABLES=$(mysql -N -e "SHOW TABLES FROM ${DB_DATABASE};" | tr '\n' ' ')
else
ALL_TABLES=$(MYSQL_PWD="$ROOT_PASSWORD" mysql -h"$DB_HOST" -u"$ROOT_USER" -N -e "SHOW TABLES FROM ${DB_DATABASE};" | tr '\n' ' ')
fi
# Count tables
TABLE_COUNT=$(echo "$ALL_TABLES" | wc -w)
# Immutable tables (cannot UPDATE or DELETE)
IMMUTABLE_TABLES="transactions transaction_types"
# Count immutable tables
IMMUTABLE_COUNT=$(echo "$IMMUTABLE_TABLES" | wc -w)
# Calculate mutable tables count
MUTABLE_COUNT=$((TABLE_COUNT - IMMUTABLE_COUNT))
echo -e "${BLUE}Total tables:${NC} $TABLE_COUNT"
echo -e "${BLUE}Immutable tables:${NC} $IMMUTABLE_COUNT (transactions, transaction_types)"
echo -e "${BLUE}Mutable tables:${NC} $MUTABLE_COUNT"
echo ""
# Create SQL script
SQL_SCRIPT=$(cat <<EOF
-- Create restricted database user for ${DB_DATABASE}
-- This user enforces transaction immutability at the database level
-- Create user for all common connection types
CREATE USER IF NOT EXISTS '${NEW_USER}'@'${DB_HOST}' IDENTIFIED BY '${NEW_PASSWORD}';
CREATE USER IF NOT EXISTS '${NEW_USER}'@'localhost' IDENTIFIED BY '${NEW_PASSWORD}';
CREATE USER IF NOT EXISTS '${NEW_USER}'@'127.0.0.1' IDENTIFIED BY '${NEW_PASSWORD}';
-- Grant SELECT and INSERT on all tables (basic read/write access)
GRANT SELECT, INSERT ON ${DB_DATABASE}.* TO '${NEW_USER}'@'${DB_HOST}';
GRANT SELECT, INSERT ON ${DB_DATABASE}.* TO '${NEW_USER}'@'localhost';
GRANT SELECT, INSERT ON ${DB_DATABASE}.* TO '${NEW_USER}'@'127.0.0.1';
-- Grant UPDATE and DELETE on each mutable table
EOF
)
# Add UPDATE/DELETE grants for each table except immutable ones
for table in $ALL_TABLES; do
# Skip immutable tables
if [[ " $IMMUTABLE_TABLES " =~ " $table " ]]; then
continue
fi
SQL_SCRIPT+=$(cat <<EOF
GRANT UPDATE, DELETE ON ${DB_DATABASE}.${table} TO '${NEW_USER}'@'${DB_HOST}';
GRANT UPDATE, DELETE ON ${DB_DATABASE}.${table} TO '${NEW_USER}'@'localhost';
GRANT UPDATE, DELETE ON ${DB_DATABASE}.${table} TO '${NEW_USER}'@'127.0.0.1';
EOF
)
done
SQL_SCRIPT+=$(cat <<EOF
-- Apply permission changes
FLUSH PRIVILEGES;
-- Display created user
SELECT
'User created successfully' as status,
'${NEW_USER}' as username,
GROUP_CONCAT(DISTINCT host) as hosts
FROM mysql.user
WHERE user = '${NEW_USER}';
EOF
)
# Execute SQL script
echo -e "${YELLOW}Executing SQL commands...${NC}"
if [ "$USE_SUDO" = true ]; then
echo "$SQL_SCRIPT" | mysql
else
echo "$SQL_SCRIPT" | MYSQL_PWD="$ROOT_PASSWORD" mysql -h"$DB_HOST" -u"$ROOT_USER"
fi
if [ $? -eq 0 ]; then
echo ""
echo -e "${GREEN}========================================${NC}"
echo -e "${GREEN}✓ User created successfully!${NC}"
echo -e "${GREEN}========================================${NC}"
echo ""
echo -e "${BLUE}User Details:${NC}"
echo -e " Username: ${GREEN}${NEW_USER}${NC}"
echo -e " Password: ${GREEN}${NEW_PASSWORD}${NC}"
echo -e " Database: ${GREEN}${DB_DATABASE}${NC}"
echo -e " Hosts: ${GREEN}${DB_HOST}, localhost, 127.0.0.1${NC}"
echo ""
echo -e "${BLUE}Permissions:${NC}"
echo -e " ${GREEN}✓${NC} SELECT, INSERT on ${TABLE_COUNT} tables"
echo -e " ${GREEN}✓${NC} UPDATE, DELETE on ${MUTABLE_COUNT} mutable tables"
echo -e " ${RED}✗${NC} UPDATE, DELETE on ${IMMUTABLE_COUNT} immutable tables (transactions, transaction_types)"
echo ""
echo -e "${YELLOW}Next Steps:${NC}"
echo ""
echo -e "1. ${BLUE}Backup your current .env file:${NC}"
echo -e " cp .env .env.backup-\$(date +'%Y%m%d-%H%M%S')"
echo ""
echo -e "2. ${BLUE}Update your .env file:${NC}"
echo -e " DB_USERNAME=${NEW_USER}"
echo -e " DB_PASSWORD=${NEW_PASSWORD}"
echo ""
echo -e "3. ${BLUE}Clear Laravel configuration cache:${NC}"
echo -e " php artisan config:clear"
echo ""
echo -e "4. ${BLUE}Test the connection:${NC}"
echo -e " php artisan tinker --execute=\"DB::connection()->getPdo();\""
echo ""
echo -e "5. ${BLUE}Verify transaction immutability:${NC}"
echo -e " ./scripts/test-transaction-immutability.sh"
echo ""
echo -e "${YELLOW}IMPORTANT:${NC}"
echo -e " • For database migrations, use root credentials:"
echo -e " DB_USERNAME=root php artisan migrate"
echo -e " • Keep root credentials secure and separate"
echo -e " • Save the new user credentials securely"
echo ""
# Save credentials to a secure file
CREDENTIALS_FILE=".credentials-${NEW_USER}"
cat > "$CREDENTIALS_FILE" <<EOC
# Database credentials for ${NEW_USER}
# Created: $(date)
# Database: ${DB_DATABASE}
DB_USERNAME=${NEW_USER}
DB_PASSWORD=${NEW_PASSWORD}
EOC
chmod 600 "$CREDENTIALS_FILE"
echo -e "${GREEN}✓ Credentials saved to: ${CREDENTIALS_FILE}${NC}"
echo -e "${YELLOW} (This file has been created with restricted permissions: 600)${NC}"
echo ""
else
echo ""
echo -e "${RED}========================================${NC}"
echo -e "${RED}✗ Error creating user${NC}"
echo -e "${RED}========================================${NC}"
echo ""
exit 1
fi
scripts/cron-backup.conf
+70
View File
@@ -0,0 +1,70 @@
# Laravel Timebank Backup Cron Configuration
# Copy this file and customize for your server setup
# To install:
# 1. Edit paths and email addresses
# 2. Copy to /etc/cron.d/timebank-backup (requires root)
# OR add entries to user crontab: crontab -e
# Set PATH and environment variables for cron (UPDATE THIS EMAIL)
PATH=/usr/local/bin:/usr/bin:/bin
MAILTO=admin@yourdomain.org
# Project directory of this installation (UPDATE THIS PATH, e.g. /var/www/timebank_cc)
PROJECT_DIR=/var/www/your-app
# Daily backup at 2:00 AM every day
0 2 * * * root cd $PROJECT_DIR && ./scripts/backup-all.sh daily --verify --notify >> /var/log/timebank-backup.log 2>&1
# Weekly backup at 3:00 AM every Sunday (includes full storage backup)
0 3 * * 0 root cd $PROJECT_DIR && ./scripts/backup-all.sh weekly --verify --notify >> /var/log/timebank-backup.log 2>&1
# Monthly backup at 4:00 AM on the 1st of each month
0 4 1 * * root cd $PROJECT_DIR && ./scripts/backup-all.sh monthly --verify --notify >> /var/log/timebank-backup.log 2>&1
# Full storage backup every Sunday at 5:00 AM (additional full backup)
0 5 * * 0 root cd $PROJECT_DIR && ./scripts/backup-storage.sh full >> /var/log/timebank-backup.log 2>&1
# Cleanup and health check every day at 6:00 AM
0 6 * * * root cd $PROJECT_DIR && ./scripts/cleanup-backups.sh >> /var/log/timebank-backup.log 2>&1
# Log rotation for backup logs (optional)
# 0 7 * * 0 root /usr/sbin/logrotate -f /etc/logrotate.d/timebank-backup
###############################################################################
# Alternative User Crontab Entries (if not using system cron)
# Run: crontab -e
# Then add these lines (adjust paths as needed):
###############################################################################
# # Daily backup at 2:00 AM
# 0 2 * * * cd /path/to/your-app && ./scripts/backup-all.sh daily --verify --notify
# # Weekly backup at 3:00 AM every Sunday
# 0 3 * * 0 cd /path/to/your-app && ./scripts/backup-all.sh weekly --verify --notify
# # Monthly backup at 4:00 AM on the 1st of each month
# 0 4 1 * * cd /path/to/your-app && ./scripts/backup-all.sh monthly --verify --notify
###############################################################################
# Cron Schedule Reference
###############################################################################
# Format: minute hour day_of_month month day_of_week command
#
# minute (0-59)
# hour (0-23)
# day_of_month (1-31)
# month (1-12)
# day_of_week (0-7, where 0 and 7 are Sunday)
#
# Special characters:
# * = any value
# , = list separator (e.g., 1,3,5)
# - = range (e.g., 1-5)
# / = step values (e.g., */5 = every 5)
#
# Examples:
# 0 2 * * * = Every day at 2:00 AM
# 0 */6 * * * = Every 6 hours
# 30 1 * * 0 = Every Sunday at 1:30 AM
# 0 0 1 * * = First day of every month at midnight
###############################################################################
scripts/debug-db-connection.sh
Executable
+82
View File
@@ -0,0 +1,82 @@
#!/bin/bash
# Debug script to test database connection and show parsed values
echo "=== Database Connection Debug ==="
echo ""
if [ ! -f .env ]; then
echo "ERROR: .env file not found"
exit 1
fi
# Parse values exactly as the test script does (with comment stripping)
DB_DATABASE=$(grep "^DB_DATABASE=" .env | cut -d '=' -f2 | sed 's/#.*//' | tr -d '"' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | sed 's/\r$//')
DB_USERNAME=$(grep "^DB_USERNAME=" .env | cut -d '=' -f2 | sed 's/#.*//' | tr -d '"' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | sed 's/\r$//')
DB_PASSWORD=$(grep "^DB_PASSWORD=" .env | cut -d '=' -f2- | sed 's/#.*//' | sed 's/^"//' | sed 's/"$//' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | sed 's/\r$//')
DB_HOST=$(grep "^DB_HOST=" .env | cut -d '=' -f2 | sed 's/#.*//' | tr -d '"' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | sed 's/\r$//')
if [ -z "$DB_HOST" ]; then
DB_HOST="localhost"
fi
echo "Parsed values from .env:"
echo " DB_DATABASE: [$DB_DATABASE]"
echo " DB_USERNAME: [$DB_USERNAME]"
echo " DB_PASSWORD length: ${#DB_PASSWORD} characters"
echo " DB_PASSWORD first char: [${DB_PASSWORD:0:1}]"
echo " DB_PASSWORD last char: [${DB_PASSWORD: -1}]"
echo " DB_HOST: [$DB_HOST]"
echo ""
# Try different connection methods
echo "Test 1: Trying connection with MYSQL_PWD..."
if MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" -e "SELECT 1 AS test;" 2>/tmp/mysql_error.log; then
echo " ✓ SUCCESS with MYSQL_PWD"
else
echo " ✗ FAILED with MYSQL_PWD"
echo " Error output:"
cat /tmp/mysql_error.log
fi
echo ""
echo "Test 2: Trying connection to localhost instead of $DB_HOST..."
if MYSQL_PWD="$DB_PASSWORD" mysql -hlocalhost -u"$DB_USERNAME" "$DB_DATABASE" -e "SELECT 1 AS test;" 2>/tmp/mysql_error2.log; then
echo " ✓ SUCCESS with localhost"
else
echo " ✗ FAILED with localhost"
echo " Error output:"
cat /tmp/mysql_error2.log
fi
echo ""
echo "Test 3: Trying without specifying host (socket connection)..."
if MYSQL_PWD="$DB_PASSWORD" mysql -u"$DB_USERNAME" "$DB_DATABASE" -e "SELECT 1 AS test;" 2>/tmp/mysql_error3.log; then
echo " ✓ SUCCESS with socket connection"
else
echo " ✗ FAILED with socket connection"
echo " Error output:"
cat /tmp/mysql_error3.log
fi
echo ""
echo "Test 4: Check MySQL is listening..."
echo " MySQL processes:"
ps aux | grep mysql | grep -v grep
echo ""
echo " MySQL network listeners:"
netstat -tlnp 2>/dev/null | grep mysql || ss -tlnp 2>/dev/null | grep mysql
echo ""
echo "Test 5: Check if mysql command works at all..."
if mysql --version >/dev/null 2>&1; then
echo " ✓ mysql command is available: $(mysql --version)"
else
echo " ✗ mysql command not found"
fi
echo ""
# Cleanup
rm -f /tmp/mysql_error.log /tmp/mysql_error2.log /tmp/mysql_error3.log
echo "=== Debug Complete ==="
scripts/deploy-apk.sh
Executable
+76
View File
@@ -0,0 +1,76 @@
#!/usr/bin/env bash
# Deploy the latest signed APK to the server and update AppVersionController.php
set -euo pipefail
# Server and paths: override with environment variables for another server, e.g.
# APK_REMOTE_HOST=example.org APK_REMOTE_APP_DIR=/var/www/myorg scripts/deploy-apk.sh
REMOTE_USER="${APK_REMOTE_USER:-root}"
REMOTE_HOST="${APK_REMOTE_HOST:-timebank.cc}"
REMOTE_APP_DIR="${APK_REMOTE_APP_DIR:-/var/www/timebank_cc}"
REMOTE_APK_DIR="$REMOTE_APP_DIR/storage/app/public/apk"
REMOTE_CONTROLLER="$REMOTE_APP_DIR/app/Http/Controllers/AppVersionController.php"
# The local checkout is the one this script lives in
LOCAL_APP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
APK_DIR="$LOCAL_APP_DIR/storage/app/public/apk"
CONTROLLER="$LOCAL_APP_DIR/app/Http/Controllers/AppVersionController.php"
# --- Find the most recently modified APK in the local storage folder ----------
LOCAL_APK=$(ls -t "$APK_DIR"/timebank-cc-*.apk 2>/dev/null | head -1)
if [[ -z "$LOCAL_APK" ]]; then
echo "ERROR: No timebank-cc-*.apk found in $APK_DIR" >&2
exit 1
fi
REMOTE_FILENAME=$(basename "$LOCAL_APK")
# Read versionCode and versionName from build.gradle.kts — single source of truth
GRADLE="${APK_GRADLE_FILE:-$HOME/AndroidStudioProjects/Timebankcc/app/build.gradle.kts}"
VERSION_CODE=$(grep -oP 'versionCode\s*=\s*\K[0-9]+' "$GRADLE" | head -1)
VERSION_NAME=$(grep -oP 'versionName\s*=\s*"\K[^"]+' "$GRADLE" | head -1)
if [[ -z "$VERSION_CODE" || -z "$VERSION_NAME" ]]; then
echo "ERROR: Could not read version from $GRADLE" >&2
exit 1
fi
# Sanity check: filename should match the versionName in build.gradle.kts
EXPECTED_FILENAME="timebank-cc-${VERSION_NAME}.apk"
if [[ "$REMOTE_FILENAME" != "$EXPECTED_FILENAME" ]]; then
echo "WARNING: APK filename '$REMOTE_FILENAME' does not match build.gradle.kts versionName '$VERSION_NAME'"
echo " Expected: $EXPECTED_FILENAME"
read -r -p "Continue anyway? [y/N] " confirm
[[ "$confirm" =~ ^[Yy]$ ]] || exit 1
fi
echo "APK : $LOCAL_APK"
echo "Version: $VERSION_NAME (code $VERSION_CODE)"
echo "Remote : ${REMOTE_USER}@${REMOTE_HOST}:${REMOTE_APK_DIR}/${REMOTE_FILENAME}"
echo ""
# --- Upload APK ---------------------------------------------------------------
echo "Uploading APK..."
scp "$LOCAL_APK" "${REMOTE_USER}@${REMOTE_HOST}:${REMOTE_APK_DIR}/"
echo "Upload complete."
# --- Update AppVersionController on the server --------------------------------
echo "Updating AppVersionController..."
ssh "${REMOTE_USER}@${REMOTE_HOST}" bash <<EOF
sed -i "s/\\\$versionCode = [0-9]*;/\\\$versionCode = ${VERSION_CODE};/" "${REMOTE_CONTROLLER}"
sed -i "s/\\\$versionName = '[^']*';/\\\$versionName = '${VERSION_NAME}';/" "${REMOTE_CONTROLLER}"
echo "Controller updated:"
grep -E "versionCode|versionName" "${REMOTE_CONTROLLER}"
EOF
# --- Update local AppVersionController to keep it in sync ---------------------
sed -i "s/\$versionCode = [0-9]*;/\$versionCode = ${VERSION_CODE};/" "$CONTROLLER"
sed -i "s/\$versionName = '[^']*';/\$versionName = '${VERSION_NAME}';/" "$CONTROLLER"
echo "Local controller updated."
echo ""
echo "Done. Users on versionCode < ${VERSION_CODE} will be offered the update."
scripts/expire-user-session.php
Executable
+96
View File
@@ -0,0 +1,96 @@
<?php
/**
* Script to manually expire all sessions for a specific user
* Usage: php scripts/expire-user-session.php [user_id]
*/
require __DIR__.'/../vendor/autoload.php';
$app = require_once __DIR__.'/../bootstrap/app.php';
$kernel = $app->make(Illuminate\Contracts\Console\Kernel::class);
$kernel->bootstrap();
if ($argc < 2) {
echo "Usage: php expire-user-session.php [user_id]\n";
exit(1);
}
$userId = (int) $argv[1];
$user = \App\Models\User::find($userId);
if (!$user) {
echo "User {$userId} not found\n";
exit(1);
}
echo "User found: {$user->name} (ID: {$user->id})\n";
echo "Session driver: " . config('session.driver') . "\n\n";
$sessionDriver = config('session.driver');
$deleted = 0;
if ($sessionDriver === 'redis') {
echo "Scanning Redis for user sessions...\n";
$redis = \Illuminate\Support\Facades\Redis::connection(config('session.connection') ?: 'default');
$prefix = config('database.redis.options.prefix', '');
// Get all session keys
$cursor = '0';
$allKeys = [];
do {
$result = $redis->scan($cursor, ['match' => $prefix . 'laravel_session:*', 'count' => 100]);
$cursor = $result[0];
$keys = $result[1] ?? [];
$allKeys = array_merge($allKeys, $keys);
} while ($cursor !== '0');
echo "Found " . count($allKeys) . " total sessions\n";
// Check each session for the user ID
foreach ($allKeys as $key) {
$sessionData = $redis->get($key);
if ($sessionData) {
// Check if this session belongs to our user
// Laravel stores user ID in the session data
if (strpos($sessionData, '"login_web_59ba36addc2b2f9401580f014c7f58ea4e30989d";i:' . $userId) !== false ||
strpos($sessionData, 's:7:"user_id";i:' . $userId) !== false) {
$redis->del($key);
$deleted++;
echo "✓ Deleted session: " . str_replace($prefix, '', $key) . "\n";
}
}
}
} elseif ($sessionDriver === 'database') {
echo "Scanning database for user sessions...\n";
$table = config('session.table', 'sessions');
$sessions = DB::table($table)
->where('user_id', $userId)
->get();
echo "Found " . count($sessions) . " sessions for user {$userId}\n";
foreach ($sessions as $session) {
DB::table($table)->where('id', $session->id)->delete();
$deleted++;
echo "✓ Deleted session: {$session->id}\n";
}
} else {
echo "Session driver '{$sessionDriver}' is not supported by this script\n";
exit(1);
}
echo "\n";
echo "====================================\n";
echo "Total sessions deleted: {$deleted}\n";
echo "User {$user->name} has been logged out from all devices\n";
echo "====================================\n";
scripts/find-all-missing-translations.php
Executable
+157
View File
@@ -0,0 +1,157 @@
<?php
/**
* Find all translation strings in PHP and Blade files that are not in en.json
*/
echo "=== COMPREHENSIVE TRANSLATION STRING SCANNER ===\n\n";
// Load current en.json
$enFile = 'resources/lang/en.json';
$existing = json_decode(file_get_contents($enFile), true);
echo "Current en.json has " . count($existing) . " keys\n\n";
// Find all PHP and Blade files
$files = [];
$directories = ['app', 'resources/views'];
foreach ($directories as $dir) {
if (!is_dir($dir)) continue;
$iterator = new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($dir)
);
foreach ($iterator as $file) {
if ($file->isFile()) {
$ext = $file->getExtension();
if ($ext === 'php' || $ext === 'blade') {
$files[] = $file->getPathname();
}
}
}
}
echo "Scanning " . count($files) . " files (PHP + Blade)...\n\n";
// Find all translation calls
$foundStrings = [];
$patterns = [
'/__\(\s*[\'"]([^\'"]+)[\'"]\s*\)/', // __('string')
'/__\(\s*"([^"]+)"\s*\)/', // __("string")
'/trans\(\s*[\'"]([^\'"]+)[\'"]\s*\)/', // trans('string')
'/@lang\(\s*[\'"]([^\'"]+)[\'"]\s*\)/', // @lang('string')
'/\{\{\s*__\(\s*[\'"]([^\'"]+)[\'"]\s*\)\s*\}\}/', // {{ __('string') }}
'/\{\{\s*trans\(\s*[\'"]([^\'"]+)[\'"]\s*\)\s*\}\}/', // {{ trans('string') }}
];
$totalFound = 0;
foreach ($files as $file) {
$content = file_get_contents($file);
$relativePath = str_replace(getcwd() . '/', '', $file);
foreach ($patterns as $pattern) {
if (preg_match_all($pattern, $content, $matches)) {
foreach ($matches[1] as $string) {
// Skip if it's a variable or complex expression
if (strpos($string, '$') !== false) continue;
if (strpos($string, '{') !== false) continue;
if (strpos($string, '.') === 0) continue;
if (empty(trim($string))) continue;
$totalFound++;
if (!isset($foundStrings[$string])) {
$foundStrings[$string] = [
'string' => $string,
'files' => []
];
}
$foundStrings[$string]['files'][] = $relativePath;
}
}
}
}
echo "Found " . $totalFound . " translation calls\n";
echo "Found " . count($foundStrings) . " unique translation strings\n\n";
// Find missing strings
$missing = [];
foreach ($foundStrings as $string => $info) {
if (!isset($existing[$string])) {
$missing[$string] = $info;
}
}
echo "Missing from en.json: " . count($missing) . " strings\n";
echo str_repeat('=', 100) . "\n\n";
if (count($missing) > 0) {
echo "Missing translation strings:\n";
echo str_repeat('-', 100) . "\n";
// Group by first occurrence file
foreach ($missing as $string => $info) {
$firstFile = $info['files'][0];
$fileCount = count($info['files']);
$fileInfo = $fileCount > 1 ? " (used in $fileCount files)" : "";
echo sprintf("%-60s %s%s\n",
substr($string, 0, 60),
basename($firstFile),
$fileInfo
);
}
echo "\n" . str_repeat('=', 100) . "\n\n";
// Automatically add them
echo "Adding " . count($missing) . " new keys to en.json...\n";
// Add missing strings to en.json
foreach ($missing as $string => $info) {
$existing[$string] = $string; // Use the string itself as the value
}
// Sort alphabetically
ksort($existing);
// Backup first
copy($enFile, $enFile . '.backup');
// Save to file
file_put_contents(
$enFile,
json_encode($existing, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES) . PHP_EOL
);
echo "\n✓ Added " . count($missing) . " new keys to en.json\n";
echo "✓ Backup created: en.json.backup\n";
echo "✓ Total keys in en.json: " . count($existing) . "\n\n";
// Save detailed report
$report = "Missing translation strings found:\n\n";
foreach ($missing as $string => $info) {
$report .= "Key: $string\n";
$report .= "Used in:\n";
foreach ($info['files'] as $file) {
$report .= " - $file\n";
}
$report .= "\n";
}
file_put_contents('/tmp/missing-translations-report.txt', $report);
echo "Detailed report saved to: /tmp/missing-translations-report.txt\n\n";
echo "Next steps:\n";
echo "1. Review the new keys in en.json and edit values if needed\n";
echo "2. Run: ./translate-new-keys.sh\n";
echo "3. Clear cache: php artisan config:clear && php artisan cache:clear\n";
} else {
echo "✓ All translation strings are already in en.json!\n";
echo "\nSummary:\n";
echo " - Scanned " . count($files) . " files\n";
echo " - Found " . $totalFound . " translation calls\n";
echo " - All " . count($foundStrings) . " unique strings are in en.json\n";
}
scripts/find-missing-translations.php
Executable
+124
View File
@@ -0,0 +1,124 @@
<?php
/**
* Find all translation strings in PHP files that are not in en.json
*/
echo "=== FINDING MISSING TRANSLATION STRINGS ===\n\n";
// Load current en.json
$enFile = 'resources/lang/en.json';
$existing = json_decode(file_get_contents($enFile), true);
echo "Current en.json has " . count($existing) . " keys\n\n";
// Find all PHP files in app directory
$phpFiles = [];
$directories = ['app/Http/Controllers', 'app/Http/Livewire', 'app/Models', 'app'];
foreach ($directories as $dir) {
if (!is_dir($dir)) continue;
$iterator = new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($dir)
);
foreach ($iterator as $file) {
if ($file->isFile() && $file->getExtension() === 'php') {
$phpFiles[] = $file->getPathname();
}
}
}
echo "Scanning " . count($phpFiles) . " PHP files...\n\n";
// Find all translation calls
$foundStrings = [];
$patterns = [
'/__\([\'"]([^\'"]+)[\'"]\)/', // __('string')
'/trans\([\'"]([^\'"]+)[\'"]\)/', // trans('string')
'/@lang\([\'"]([^\'"]+)[\'"]\)/', // @lang('string')
];
foreach ($phpFiles as $file) {
$content = file_get_contents($file);
foreach ($patterns as $pattern) {
if (preg_match_all($pattern, $content, $matches)) {
foreach ($matches[1] as $string) {
// Skip if it's a variable or complex expression
if (strpos($string, '$') !== false) continue;
if (strpos($string, '.') === 0) continue; // Starts with dot
$foundStrings[$string] = [
'file' => str_replace(getcwd() . '/', '', $file),
'string' => $string
];
}
}
}
}
echo "Found " . count($foundStrings) . " unique translation strings in PHP files\n\n";
// Find missing strings
$missing = [];
foreach ($foundStrings as $string => $info) {
if (!isset($existing[$string])) {
$missing[$string] = $info;
}
}
echo "Missing from en.json: " . count($missing) . " strings\n";
echo str_repeat('=', 80) . "\n\n";
if (count($missing) > 0) {
echo "Missing translation strings:\n";
echo str_repeat('-', 80) . "\n";
foreach ($missing as $string => $info) {
echo sprintf("%-50s (from %s)\n", $string, $info['file']);
}
echo "\n" . str_repeat('=', 80) . "\n\n";
echo "Would you like to add these to en.json? (y/n): ";
$handle = fopen("php://stdin", "r");
$line = fgets($handle);
fclose($handle);
if (trim($line) === 'y' || trim($line) === 'yes') {
// Add missing strings to en.json
foreach ($missing as $string => $info) {
$existing[$string] = $string; // Use the string itself as the value
}
// Sort alphabetically
ksort($existing);
// Save to file
file_put_contents(
$enFile,
json_encode($existing, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES) . PHP_EOL
);
echo "\n✓ Added " . count($missing) . " new keys to en.json\n";
echo "✓ Total keys in en.json: " . count($existing) . "\n\n";
echo "Next steps:\n";
echo "1. Review the new keys in en.json and edit values if needed\n";
echo "2. Run: ./translate-new-keys.sh\n";
echo "3. Clear cache: php artisan config:clear && php artisan cache:clear\n";
} else {
echo "\nNo changes made.\n";
// Save list to file for review
$output = "Missing translation strings:\n\n";
foreach ($missing as $string => $info) {
$output .= sprintf("%-50s (from %s)\n", $string, $info['file']);
}
file_put_contents('/tmp/missing-translations.txt', $output);
echo "List saved to: /tmp/missing-translations.txt\n";
}
} else {
echo "✓ All translation strings in PHP files are already in en.json!\n";
}
scripts/load-env.sh
+48
View File
@@ -0,0 +1,48 @@
#!/bin/bash
# Robust .env file loader for backup scripts
# This handles comments, quotes, and special characters properly
load_env() {
local env_file="$1"
if [ ! -f "$env_file" ]; then
return 1
fi
# Read .env file line by line and export valid variables
while IFS= read -r line; do
# Skip empty lines and comments
if [[ -z "$line" || "$line" =~ ^[[:space:]]*# ]]; then
continue
fi
# Match valid environment variable pattern
if [[ "$line" =~ ^[A-Za-z_][A-Za-z0-9_]*= ]]; then
# Remove inline comments (everything after # that's not inside quotes)
local clean_line
if [[ "$line" =~ ^([^#]*[\"\']).*(#.*)$ ]]; then
# Line has quotes, need to be more careful with comment removal
clean_line="$line"
else
# Simple case, remove everything after #
clean_line="${line%%#*}"
fi
# Remove trailing whitespace
clean_line="${clean_line%"${clean_line##*[![:space:]]}"}"
# A quoted value: keep what is between the quotes (a password may hold # or spaces), drop the
# quotes and any comment after them. Without this the quotes became part of the value.
local key="${clean_line%%=*}" value="${clean_line#*=}"
if [[ "$value" =~ ^\"([^\"]*)\"([[:space:]]+#.*)?$ ]] || [[ "$value" =~ ^\'([^\']*)\'([[:space:]]+#.*)?$ ]]; then
clean_line="${key}=${BASH_REMATCH[1]}"
fi
# Export the variable
export "$clean_line"
fi
done < "$env_file"
return 0
}
scripts/log.sh
Executable
+38
View File
@@ -0,0 +1,38 @@
#!/bin/bash
# WARNING: This script is for development purposes only.
# Do include in production folder, unless execution rights are restricted. See below.
# Change the ownership of the file to your user:
# chown user:user log.sh
# Set the permissions to allow only the owner to read and execute the file:
# sudo chmod 700 log.sh
# Ask for the number of lines to display
echo "How many lines of the log would you like to display?"
read num_lines
# Display the specified number of lines from the log
tail -n $num_lines storage/logs/laravel.log
# Ask if the log should be cleared
echo "Would you like to clear the log? (Y/N)"
read clear_log
# Clear the log if the user answered 'Y' or 'y'
if [ "$clear_log" = "Y" ] || [ "$clear_log" = "y" ]; then
> storage/logs/laravel.log
echo ""
echo "Log cleared."
echo ""
# Clear the log and append the date and time it was cleared
echo "" > storage/logs/laravel.log
echo "[$(date '+%Y-%m-%d %H:%M:%S')] LOG CLEARED" >> storage/logs/laravel.log
echo "" >> storage/logs/laravel.log
fi
# Exit the script
exit 0
scripts/mail-real.env.example
+9
View File
@@ -0,0 +1,9 @@
# Real SMTP server settings
# Update these values when the production SMTP server details change
MAIL_MAILER=smtp
MAIL_HOST=your-smtp-server.com
MAIL_PORT=587
MAIL_USERNAME=your-username
MAIL_PASSWORD=your-password
MAIL_ENCRYPTION=tls
MAIL_BOUNCE_ADDRESS=bounces@domain.org
scripts/mail-switch.sh
Executable
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env bash
# mail-switch.sh — toggle between Mailpit (local testing) and real SMTP server
# Usage: ./scripts/mail-switch.sh [mailpit|real]
# Without argument: auto-detects current mode and toggles
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
APP_DIR="$(dirname "$SCRIPT_DIR")"
ENV_FILE="$APP_DIR/.env"
REAL_ENV="$SCRIPT_DIR/mail-real.env"
if [[ ! -f "$ENV_FILE" ]]; then
echo "ERROR: .env file not found at $ENV_FILE"
exit 1
fi
if [[ ! -f "$REAL_ENV" ]]; then
echo "ERROR: mail-real.env not found at $REAL_ENV"
echo "Create it with your real SMTP settings."
exit 1
fi
# Read current MAIL_HOST from .env
current_host=$(grep -E "^MAIL_HOST=" "$ENV_FILE" | cut -d= -f2 | tr -d '"' | tr -d "'")
# Determine target mode
if [[ "${1:-}" == "mailpit" ]]; then
target="mailpit"
elif [[ "${1:-}" == "real" ]]; then
target="real"
elif [[ "$current_host" == "localhost" || "$current_host" == "127.0.0.1" ]]; then
target="real"
echo "Current mode: Mailpit → switching to real SMTP"
else
target="mailpit"
echo "Current mode: real SMTP ($current_host) → switching to Mailpit"
fi
set_env_value() {
local key="$1"
local value="$2"
# Replace the key=value line in .env (handles empty values too)
sed -i "s|^${key}=.*|${key}=${value}|" "$ENV_FILE"
}
if [[ "$target" == "mailpit" ]]; then
set_env_value "MAIL_MAILER" "smtp"
set_env_value "MAIL_HOST" "localhost"
set_env_value "MAIL_PORT" "1025"
set_env_value "MAIL_USERNAME" ""
set_env_value "MAIL_PASSWORD" ""
set_env_value "MAIL_ENCRYPTION" "null"
echo "Mail switched to: Mailpit (localhost:1025)"
else
# Load real SMTP settings from mail-real.env
while IFS='=' read -r key value; do
# Skip comments and empty lines
[[ "$key" =~ ^#.*$ || -z "$key" ]] && continue
set_env_value "$key" "$value"
done < "$REAL_ENV"
real_host=$(grep -E "^MAIL_HOST=" "$REAL_ENV" | cut -d= -f2)
echo "Mail switched to: real SMTP ($real_host)"
fi
# Clear config cache and restart queue workers
cd "$APP_DIR"
php artisan config:clear
php artisan queue:restart
echo "Done. Config cleared and queue workers restarted."
scripts/migrate-to-example-configs.sh
Executable
+144
View File
@@ -0,0 +1,144 @@
#!/bin/bash
#
# Migration Script: Move to .example Config Pattern
#
# This script migrates the configuration system to use .example files
# for white-label protection. Run this once per repository.
#
# What it does:
# 1. Removes config files from git tracking (keeps local files)
# 2. Adds .example files to git tracking
# 3. Updates .gitignore
# 4. Creates a commit with these changes
#
set -e # Exit on error
# Colors
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
RED='\033[0;31m'
NC='\033[0m'
echo -e "${BLUE}========================================${NC}"
echo -e "${BLUE} Config Migration to .example Pattern ${NC}"
echo -e "${BLUE}========================================${NC}\n"
# Check if in git repository
if [ ! -d .git ]; then
echo -e "${RED}Error: Must be run from repository root${NC}"
exit 1
fi
# Files to migrate
config_files=(
"config/themes.php"
"config/timebank-default.php"
"config/timebank_cc.php"
)
echo -e "${YELLOW}This script will:${NC}"
echo "1. Remove config files from git tracking (local files remain)"
echo "2. Add .example template files to git"
echo "3. Update .gitignore if needed"
echo -e "\n${YELLOW}Your custom config files will be preserved locally.${NC}\n"
read -p "Continue? (y/n) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
echo "Migration cancelled."
exit 0
fi
echo -e "\n${BLUE}Step 1: Verifying .example files exist${NC}"
for config_file in "${config_files[@]}"; do
if [ ! -f "${config_file}.example" ]; then
if [ -f "$config_file" ]; then
echo -e "${YELLOW}Creating ${config_file}.example from current file...${NC}"
cp "$config_file" "${config_file}.example"
else
echo -e "${RED}Warning: $config_file doesn't exist, skipping...${NC}"
continue
fi
else
echo -e "${GREEN}✓ ${config_file}.example exists${NC}"
fi
done
echo -e "\n${BLUE}Step 2: Removing config files from git tracking${NC}"
for config_file in "${config_files[@]}"; do
if git ls-files --error-unmatch "$config_file" > /dev/null 2>&1; then
echo -e "${YELLOW}Removing $config_file from git (keeping local file)...${NC}"
git rm --cached "$config_file"
echo -e "${GREEN}✓ Removed $config_file from git tracking${NC}"
else
echo -e "${YELLOW}$config_file not tracked in git, skipping...${NC}"
fi
done
echo -e "\n${BLUE}Step 3: Adding .example files to git${NC}"
for config_file in "${config_files[@]}"; do
if [ -f "${config_file}.example" ]; then
git add "${config_file}.example"
echo -e "${GREEN}✓ Added ${config_file}.example to git${NC}"
fi
done
echo -e "\n${BLUE}Step 4: Verifying .gitignore${NC}"
if grep -q "/config/themes.php" .gitignore; then
echo -e "${GREEN}✓ .gitignore already updated${NC}"
else
echo -e "${YELLOW}Updating .gitignore...${NC}"
git add .gitignore
echo -e "${GREEN}✓ Added .gitignore changes${NC}"
fi
echo -e "\n${BLUE}Step 5: Adding enhanced deploy.sh${NC}"
if grep -q "CHECKING CONFIGURATION FILES" deploy.sh; then
git add deploy.sh
echo -e "${GREEN}✓ Added deploy.sh changes${NC}"
else
echo -e "${YELLOW}Warning: deploy.sh doesn't have config checking logic${NC}"
fi
echo -e "\n${BLUE}Step 6: Creating git commit${NC}"
cat > /tmp/migration_commit_msg.txt << 'EOF'
Migrate to .example config pattern for white-label protection
- Add .example template files for themes and platform configs
- Remove actual config files from git tracking (gitignored)
- Update .gitignore to protect custom configurations
- Enhance deploy.sh to copy .example files if configs missing
- Add WHITE_LABEL_CONFIG.md documentation
This ensures white-label installations can customize configs
without git conflicts during deployments.
Config files affected:
- config/themes.php
- config/timebank-default.php
- config/timebank_cc.php
Local config files are preserved. Future git pulls will not
overwrite installation-specific customizations.
EOF
git commit -F /tmp/migration_commit_msg.txt
rm /tmp/migration_commit_msg.txt
echo -e "\n${GREEN}========================================${NC}"
echo -e "${GREEN} Migration Complete!${NC}"
echo -e "${GREEN}========================================${NC}\n"
echo -e "${YELLOW}What changed:${NC}"
echo "✓ .example files are now tracked in git (templates)"
echo "✓ Actual config files are gitignored (your customizations)"
echo "✓ deploy.sh will auto-create configs from templates if missing"
echo ""
echo -e "${YELLOW}Next steps:${NC}"
echo "1. Push these changes: git push origin main"
echo "2. On other installations, pull updates: git pull origin main"
echo "3. Their custom configs will be preserved automatically"
echo ""
echo -e "${BLUE}For more info, see: references/MULTI_PLATFORM_INSTANCES.md${NC}"
scripts/patch-vendor-firefox.sh
Executable
+31
View File
@@ -0,0 +1,31 @@
#!/bin/bash
# Patch vendor JS files to fix Firefox beta parser bugs.
# Re-applied automatically after composer install/update.
set -e
# 1. Copy Livewire JS to public/ for static serving (bypasses PHP which causes Firefox issues)
LIVEWIRE_SRC="vendor/livewire/livewire/dist/livewire.js"
LIVEWIRE_DST="public/livewire-dev.js"
if [ -f "$LIVEWIRE_SRC" ]; then
cp "$LIVEWIRE_SRC" "$LIVEWIRE_DST"
echo " [patch] Copied $LIVEWIRE_SRC -> $LIVEWIRE_DST"
else
echo " [patch] WARNING: $LIVEWIRE_SRC not found, skipping"
fi
# 2. Add trailing newline to WireUI JS if missing (Firefox beta requires it)
WIREUI_JS="vendor/wireui/wireui/dist/wireui.js"
if [ -f "$WIREUI_JS" ]; then
LAST_BYTE=$(tail -c1 "$WIREUI_JS" | xxd -p)
if [ "$LAST_BYTE" != "0a" ]; then
echo "" >> "$WIREUI_JS"
echo " [patch] Added trailing newline to $WIREUI_JS"
else
echo " [patch] $WIREUI_JS already has trailing newline"
fi
else
echo " [patch] WARNING: $WIREUI_JS not found, skipping"
fi
scripts/prepare-for-translation.php
Executable
+32
View File
@@ -0,0 +1,32 @@
<?php
/**
* Remove untranslated keys (where key === value) from all language files
* This prepares them for AI translation
*/
$locales = ['nl', 'de', 'es', 'fr'];
echo "Preparing files for translation...\n\n";
foreach ($locales as $locale) {
$file = "resources/lang/{$locale}.json";
$data = json_decode(file_get_contents($file), true);
$before = count($data);
// Remove entries where key === value (untranslated)
$filtered = array_filter($data, function($value, $key) {
return $value !== $key;
}, ARRAY_FILTER_USE_BOTH);
$after = count($filtered);
$removed = $before - $after;
ksort($filtered);
file_put_contents($file, json_encode($filtered, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES) . PHP_EOL);
echo "{$locale}.json: Removed {$removed} untranslated keys ({$before} → {$after} keys)\n";
}
echo "\nFiles prepared! Now ready for translation.\n";
echo "Next: Run translation for each language\n";
scripts/publish-public-snapshot.sh
Executable
+159
View File
@@ -0,0 +1,159 @@
#!/bin/bash
#
# Publish a history-free snapshot of a branch to the public repository.
#
# The public repo never receives the private history: each publish is ONE new
# commit whose tree is the chosen ref minus the paths in .public-exclude. Its
# only parent is the previous public snapshot, so the public history is just
# the list of snapshots.
#
# Before anything is pushed, the snapshot is scanned for secrets: the values of
# PASSWORD/SECRET/KEY/TOKEN variables in the local .env, private keys, service
# account files and common token formats. Any hit aborts the publish.
#
# Usage (from the repository root):
# scripts/publish-public-snapshot.sh [ref] # dry run: build + scan only
# scripts/publish-public-snapshot.sh [ref] --push # build, scan and push
#
# ref defaults to main. Plain `git push public` is disabled on purpose
# (remote.public.pushurl), so this script is the only way to publish.
set -euo pipefail
PUBLIC_URL=https://gitea.thesesidesup.nl/ronald/timebank-cc-public.git
PUBLIC_BRANCH=main
REF=main
PUSH=0
for arg in "$@"; do
case "$arg" in
--push) PUSH=1 ;;
-*) echo "Unknown option: $arg" >&2; exit 2 ;;
*) REF=$arg ;;
esac
done
cd "$(git rev-parse --show-toplevel)"
COMMIT=$(git rev-parse --verify "$REF^{commit}")
SHORT=$(git rev-parse --short "$COMMIT")
echo "Source: $REF ($SHORT)"
# --- Build the tree without the excluded paths ------------------------------
INDEX=$(mktemp)
trap 'rm -f "$INDEX"' EXIT
export GIT_INDEX_FILE=$INDEX
git read-tree "$COMMIT"
EXCLUDES=()
ALLOWED=() # '!path' lines: reviewed, exempt from the file-name check
if git cat-file -e "$COMMIT:.public-exclude" 2>/dev/null; then
while IFS= read -r line; do
line=${line%%#*}
line=$(printf '%s' "$line" | sed -E 's/^[[:space:]]+|[[:space:]]+$//g')
case "$line" in
'') ;;
'!'*) ALLOWED+=("${line#!}") ;;
*) EXCLUDES+=("$line") ;;
esac
done < <(git show "$COMMIT:.public-exclude")
fi
for p in "${EXCLUDES[@]}"; do
git rm -r -q --cached --ignore-unmatch -- "$p" >/dev/null
done
TREE=$(git write-tree)
unset GIT_INDEX_FILE
echo "Excluded: ${EXCLUDES[*]:-(nothing)}"
echo "Files in snapshot: $(git ls-tree -r --name-only "$TREE" | wc -l)"
# --- Secret scan --------------------------------------------------------------
hits=0
report() { echo " SECRET: $1"; hits=$((hits + 1)); }
# 1. Values of secret-like variables in the local .env (never printed). In a
# linked worktree without its own .env, use the main checkout's.
ENV_FILE=.env
[ -f "$ENV_FILE" ] || ENV_FILE="$(dirname "$(git rev-parse --path-format=absolute --git-common-dir)")/.env"
if [ -f "$ENV_FILE" ]; then
echo "Scanning for values from: $ENV_FILE"
else
echo "WARNING: no .env found, skipping the value scan"
fi
if [ -f "$ENV_FILE" ]; then
while IFS= read -r line; do
key=${line%%=*}
val=$(printf '%s' "${line#*=}" | sed -E 's/[[:space:]]+#.*//; s/^"(.*)"$/\1/; s/^'\''(.*)'\''$/\1/')
[ ${#val} -ge 8 ] || continue
case "$val" in *'${'*|null|true|false) continue ;; esac
# Client-side keys are public by design (they ship in the JS bundle).
case "$key" in VITE_*|MIX_*|PUSHER_APP_KEY|REVERB_APP_KEY) continue ;; esac
while IFS= read -r f; do
report "value of $key from .env in ${f#*:}"
done < <(git grep -l -F -e "$val" "$TREE" -- || true)
done < <(grep -E '^[A-Z0-9_]*(PASSWORD|PASS|SECRET|KEY|TOKEN|DSN)[A-Z0-9_]*=' "$ENV_FILE" || true)
fi
# 2. Files that should never be public.
while IFS= read -r f; do
for a in "${ALLOWED[@]}"; do [ "$f" = "$a" ] && continue 2; done
report "sensitive file name: $f"
done < <(git ls-tree -r --name-only "$TREE" \
| grep -E '(^|/)\.env$|(^|/)\.env\.(local|production|prod|backup)|\.pem$|\.p12$|\.pfx$|(^|/)id_(rsa|ed25519|ecdsa)$|firebase-adminsdk|\.htpasswd|\.sql(\.gz)?$' || true)
# 3. Common secret formats. Placeholders (your_..., ..._here, example) are skipped.
PATTERNS=(
-e "-----BEGIN [A-Z ]*PRIVATE KEY-----"
-e "-p'[^'\$ <]{6,}'"
-e "MYSQL_PWD=[^\$\" <]{6,}"
-e "IDENTIFIED BY '[^'<\$]{6,}'"
-e "AKIA[0-9A-Z]{16}"
-e "sk_live_[0-9a-zA-Z]{10,}"
-e "gh[pousr]_[A-Za-z0-9]{30,}"
-e "glpat-[A-Za-z0-9_-]{20}"
-e "xox[abprs]-[A-Za-z0-9-]{10,}"
-e "AIza[0-9A-Za-z_-]{35}"
-e "://[^/ :@\"'<]+:[^/ @\"'\$<]{6,}@"
)
while IFS= read -r m; do
m=${m#*:} # strip the tree id
report "pattern match in ${m%%:*}:$(cut -d: -f2 <<<"$m") (line not shown)"
done < <(git grep -n -I -E "${PATTERNS[@]}" "$TREE" -- ':!*.lock' ':!package-lock.json' \
| grep -v -i -E 'your[_-]|_here|example|placeholder|changeme|xxxx|\.\.\.' || true)
if [ $hits -ne 0 ]; then
echo
echo "ABORTED: $hits possible secret(s) in the snapshot. Fix them in $REF, or add the"
echo "path to .public-exclude, then run again. Nothing was pushed."
exit 1
fi
echo "Secret scan: clean"
# --- Commit on top of the previous public snapshot ----------------------------
PARENT=$(git ls-remote "$PUBLIC_URL" "refs/heads/$PUBLIC_BRANCH" | awk '{print $1}')
PARENT_ARGS=()
if [ -n "$PARENT" ]; then
git fetch -q "$PUBLIC_URL" "$PUBLIC_BRANCH"
if [ "$(git rev-parse "$PARENT^{tree}")" = "$TREE" ]; then
echo "Public repo already has exactly this content. Nothing to do."
exit 0
fi
PARENT_ARGS=(-p "$PARENT")
echo "Previous public snapshot: $(git log -1 --format='%h %s' "$PARENT")"
else
echo "Public repo is empty: this becomes the first commit."
fi
MSG="Snapshot of $REF at $SHORT ($(git log -1 --format=%cd --date=short "$COMMIT"))"
SNAP=$(git commit-tree "$TREE" "${PARENT_ARGS[@]}" -m "$MSG")
echo "Snapshot commit: $(git rev-parse --short "$SNAP") \"$MSG\""
if [ $PUSH -eq 1 ]; then
git push "$PUBLIC_URL" "$SNAP:refs/heads/$PUBLIC_BRANCH"
echo "Published to $PUBLIC_URL ($PUBLIC_BRANCH)"
else
echo
echo "Dry run, nothing pushed. Inspect with: git ls-tree -r --name-only $SNAP | less"
echo "Publish with: $0 $REF --push"
fi
scripts/re-index-search.sh
Executable
+208
View File
@@ -0,0 +1,208 @@
#!/bin/bash
# Colors for output
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[0;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color
# Function for printing section headers
section_header() {
printf "\n${BLUE}===========================================================${NC}\n"
printf "${BLUE} $1${NC}\n"
printf "${BLUE}===========================================================${NC}\n\n"
}
# Error handling
error_exit() {
printf "${RED}ERROR: $1${NC}\n" >&2
exit 1
}
# Load .env variables
set -a
if [ -f .env ]; then
. ./.env 2>/dev/null
fi
set +a
# Set Elasticsearch authentication flag if credentials exist
if [ -n "$ELASTICSEARCH_USER" ] && [ -n "$ELASTICSEARCH_PASSWORD" ]; then
ES_AUTH="-u $ELASTICSEARCH_USER:$ELASTICSEARCH_PASSWORD"
else
ES_AUTH=""
fi
section_header "🗑️ CLEANING UP OLD INDICES AND ALIASES"
# Function to delete all indices matching a pattern
delete_indices_by_pattern() {
local pattern=$1
local alias_name=$2
printf "${YELLOW}Cleaning up pattern: ${pattern}${NC}\n"
# Get all indices matching the pattern (both direct indices and versioned)
local all_indices=$(curl $ES_AUTH -s -X GET "localhost:9200/_cat/indices?h=index" | grep -E "^${pattern}(_[0-9]+)?$" | sort)
printf "${YELLOW}Found indices for pattern '${pattern}': ${all_indices}${NC}\n"
# Check if there's a direct index with the alias name (this causes conflicts)
local direct_index=$(echo "$all_indices" | grep -E "^${pattern}$")
if [ -n "$direct_index" ]; then
printf "${YELLOW}Found conflicting direct index: $direct_index (this will be deleted)${NC}\n"
delete_response=$(curl $ES_AUTH -s -X DELETE "localhost:9200/$direct_index")
printf "${YELLOW}Delete response: $delete_response${NC}\n"
if echo "$delete_response" | grep -q '"acknowledged":true'; then
printf "${GREEN}Successfully deleted conflicting index: $direct_index${NC}\n"
else
printf "${RED}Failed to delete conflicting index $direct_index — attempting force delete${NC}\n"
curl $ES_AUTH -s -X DELETE "localhost:9200/$direct_index?ignore_unavailable=true"
fi
fi
# Delete ALL versioned indices — scout:import will create a fresh one
local versioned_indices=$(echo "$all_indices" | grep -E "^${pattern}_[0-9]+$")
if [ -n "$versioned_indices" ]; then
for index in $versioned_indices; do
printf "${YELLOW}Deleting versioned index: $index${NC}\n"
curl $ES_AUTH -s -X DELETE "localhost:9200/$index?ignore_unavailable=true" || printf "${RED}Failed to delete $index${NC}\n"
done
fi
# Remove any existing alias (safe operation)
printf "${YELLOW}Removing existing alias: $alias_name${NC}\n"
curl $ES_AUTH -s -X POST "localhost:9200/_aliases" -H 'Content-Type: application/json' -d"{\"actions\":[{\"remove\":{\"alias\":\"$alias_name\",\"index\":\"*\"}}]}" 2>/dev/null || printf "${YELLOW}Alias $alias_name didn't exist${NC}\n"
}
# Define index patterns (using regular variables instead of associative array)
INDEX_PATTERNS="users_index organizations_index banks_index posts_index calls_index transactions_index"
# Clean up old indices for each pattern
for alias in $INDEX_PATTERNS; do
delete_indices_by_pattern "$alias" "$alias"
done
section_header "⏳ WAITING FOR ELASTICSEARCH CLUSTER HEALTH"
# Wait for Elasticsearch to be ready — require green (all shards assigned) before importing.
# On production single-node clusters this can take up to 2 minutes after a restart.
printf "${YELLOW}Waiting for green cluster health (up to 120s)...${NC}\n"
timeout=120
counter=0
cluster_ready=false
while [ $counter -lt $timeout ]; do
health_response=$(curl $ES_AUTH -s "localhost:9200/_cluster/health" 2>/dev/null)
if echo "$health_response" | grep -q '"status":"green"'; then
printf "${GREEN}Cluster is green — all shards active!${NC}\n"
cluster_ready=true
break
fi
# Also accept yellow on single-node clusters where replicas can never be assigned
unassigned=$(echo "$health_response" | grep -o '"unassigned_shards":[0-9]*' | grep -o '[0-9]*$')
initializing=$(echo "$health_response" | grep -o '"initializing_shards":[0-9]*' | grep -o '[0-9]*$')
if echo "$health_response" | grep -q '"status":"yellow"' && [ "${unassigned:-1}" = "0" ] && [ "${initializing:-1}" = "0" ]; then
printf "${GREEN}Cluster is yellow with 0 unassigned/initializing shards — safe to proceed.${NC}\n"
cluster_ready=true
break
fi
printf "${YELLOW}Waiting for cluster (${counter}/${timeout}) — status: $(echo "$health_response" | grep -o '"status":"[^"]*"'), unassigned: ${unassigned:-?}...${NC}\n"
sleep 3
counter=$((counter + 3))
done
if [ "$cluster_ready" = false ]; then
printf "${RED}ERROR: Elasticsearch cluster did not reach a healthy state after ${timeout}s.${NC}\n"
printf "${RED}Check: curl -s localhost:9200/_cluster/health?pretty${NC}\n"
printf "${RED}Check: curl -s localhost:9200/_cat/shards?v | grep -v STARTED${NC}\n"
error_exit "Elasticsearch cluster not healthy — aborting re-index to prevent partial data"
fi
# Helper: import a model with up to 3 retries (15s pause between attempts)
import_with_retry() {
local model="$1"
local attempts=3
local attempt=1
while [ $attempt -le $attempts ]; do
printf "${GREEN}Importing %s (attempt %d/%d)...${NC}\n" "$model" "$attempt" "$attempts"
if SCOUT_QUEUE=false php artisan scout:import "$model"; then
printf "${GREEN}Successfully imported %s${NC}\n" "$model"
return 0
fi
printf "${YELLOW}Import of %s failed on attempt %d.${NC}\n" "$model" "$attempt"
if [ $attempt -lt $attempts ]; then
printf "${YELLOW}Waiting 15s before retry...${NC}\n"
sleep 15
fi
attempt=$((attempt + 1))
done
error_exit "Failed to import $model after $attempts attempts"
}
section_header "🧹 SKIPPING FLUSH - DIRECT IMPORT WILL CREATE FRESH INDICES"
# Skip flush since we already cleaned up indices above and import will create fresh ones
printf "${YELLOW}Note: Skipping scout:flush commands to avoid shard conflicts.${NC}\n"
printf "${YELLOW}The scout:import commands will create fresh indices automatically.${NC}\n"
section_header "📥 IMPORTING ALL MODELS TO ELASTICSEARCH (WITHOUT QUEUE)"
import_with_retry "App\Models\User"
sleep 5
import_with_retry "App\Models\Organization"
sleep 5
import_with_retry "App\Models\Bank"
sleep 5
import_with_retry "App\Models\Post"
sleep 5
import_with_retry "App\Models\Call"
sleep 5
import_with_retry "App\Models\Transaction"
# Wait a moment for Elasticsearch to process (shorter wait since we're not using queue)
printf "${YELLOW}Waiting for Elasticsearch to process...${NC}\n"
sleep 2
section_header "🔗 CREATING ALIASES FOR STABLE INDEX NAMES"
# Create aliases pointing to the new timestamped indices
for alias in $INDEX_PATTERNS; do
printf "${GREEN}Creating alias for: $alias${NC}\n"
# Find the latest index for this pattern
latest_index=$(curl $ES_AUTH -s -X GET "localhost:9200/_cat/indices?h=index" | grep "^${alias}_" | sort | tail -n 1)
if [ -n "$latest_index" ]; then
printf "${GREEN}Found latest index: $latest_index${NC}\n"
# Create alias
curl $ES_AUTH -s -X POST "localhost:9200/_aliases" -H 'Content-Type: application/json' -d"
{
\"actions\": [
{ \"add\": { \"index\": \"${latest_index}\", \"alias\": \"${alias}\" } }
]
}
" && printf "${GREEN}✅ Created alias ${alias} -> ${latest_index}${NC}\n" || printf "${RED}❌ Failed to create alias ${alias}${NC}\n"
else
printf "${RED}❌ No index found for pattern ${alias}_*${NC}\n"
fi
done
section_header "📋 FINAL ELASTICSEARCH INDICES AND ALIASES"
printf "${BLUE}Indices:${NC}\n"
curl $ES_AUTH -s -X GET "localhost:9200/_cat/indices?v"
printf "\n${BLUE}Aliases:${NC}\n"
curl $ES_AUTH -s -X GET "localhost:9200/_cat/aliases?v"
section_header "✅ REINDEXING COMPLETE"
printf "${GREEN}All models have been reindexed successfully!${NC}\n"
printf "${GREEN}You can now use the stable alias names (users_index, organizations_index, etc.) in your application.${NC}\n"
scripts/restore-all.sh
Executable
+525
View File
@@ -0,0 +1,525 @@
#!/bin/bash
# Laravel Timebank Complete Restoration Script
# Restores both database and storage from backups
# Usage: ./restore-all.sh [options]
set -e # Exit on any error
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
BACKUP_ROOT_DIR="$PROJECT_ROOT/backups"
LOG_FILE="$BACKUP_ROOT_DIR/restore.log"
# Create log directory
mkdir -p "$BACKUP_ROOT_DIR/logs"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Logging function
log() {
local level="$1"
local message="$2"
local timestamp="[$(date '+%Y-%m-%d %H:%M:%S')]"
case "$level" in
"INFO")
echo -e "${timestamp} ${BLUE}[INFO]${NC} $message" | tee -a "$LOG_FILE"
;;
"SUCCESS")
echo -e "${timestamp} ${GREEN}[SUCCESS]${NC} $message" | tee -a "$LOG_FILE"
;;
"WARNING")
echo -e "${timestamp} ${YELLOW}[WARNING]${NC} $message" | tee -a "$LOG_FILE"
;;
"ERROR")
echo -e "${timestamp} ${RED}[ERROR]${NC} $message" | tee -a "$LOG_FILE"
;;
*)
echo "$timestamp $message" | tee -a "$LOG_FILE"
;;
esac
}
# Function to show usage
show_usage() {
echo "Usage: $0 [options]"
echo ""
echo "Options:"
echo " --latest - Restore from latest backups"
echo " --database-file FILE - Specify database backup file"
echo " --storage-file FILE - Specify storage backup file"
echo " --database-latest - Use latest database backup only"
echo " --storage-latest - Use latest storage backup only"
echo " --confirm - Skip confirmation prompts"
echo " --list-backups - List available backups"
echo " --help - Show this help message"
echo ""
echo "Restore modes:"
echo " Complete restore: Both database and storage"
echo " Database only: --database-file or --database-latest"
echo " Storage only: --storage-file or --storage-latest"
echo ""
echo "Examples:"
echo " $0 --latest # Restore latest database and storage"
echo " $0 --database-latest --storage-latest # Same as above"
echo " $0 --database-file db.sql.gz --storage-file storage.tar.gz"
echo " $0 --list-backups # Show available backups"
echo ""
exit 0
}
# Function to list available backups
list_backups() {
log "INFO" "Available backups for restoration:"
echo ""
if [ ! -d "$BACKUP_ROOT_DIR" ]; then
log "WARNING" "No backup directory found"
return 0
fi
local backup_found=false
# Database backups
if [ -d "$BACKUP_ROOT_DIR/database" ]; then
echo -e "${BLUE}Database backups:${NC}"
for backup_type in daily weekly monthly; do
local backup_dir="$BACKUP_ROOT_DIR/database/$backup_type"
if [ -d "$backup_dir" ]; then
local backups=($(find "$backup_dir" -name "*.sql.gz" -type f | sort -r | head -3))
if [ ${#backups[@]} -gt 0 ]; then
echo -e " ${backup_type}:"
for backup in "${backups[@]}"; do
local size=$(du -h "$backup" | cut -f1)
local date_created=$(date -r "$backup" '+%Y-%m-%d %H:%M:%S')
echo " $(basename "$backup") ($size, $date_created)"
done
backup_found=true
fi
fi
done
echo ""
fi
# Storage backups
if [ -d "$BACKUP_ROOT_DIR/storage" ]; then
echo -e "${BLUE}Storage backups:${NC}"
for backup_type in daily weekly monthly; do
local backup_dir="$BACKUP_ROOT_DIR/storage/$backup_type"
if [ -d "$backup_dir" ]; then
local backups=($(find "$backup_dir" -name "*.tar.gz" -type f | sort -r | head -3))
if [ ${#backups[@]} -gt 0 ]; then
echo -e " ${backup_type}:"
for backup in "${backups[@]}"; do
local size=$(du -h "$backup" | cut -f1)
local date_created=$(date -r "$backup" '+%Y-%m-%d %H:%M:%S')
echo " $(basename "$backup") ($size, $date_created)"
done
backup_found=true
fi
fi
done
echo ""
fi
if [ "$backup_found" = false ]; then
log "WARNING" "No backups found"
fi
}
# Function to find latest backup
get_latest_backup() {
local backup_type="$1" # database or storage
local extension="$2" # sql.gz or tar.gz
local latest_backup=""
local latest_time=0
if [ -d "$BACKUP_ROOT_DIR/$backup_type" ]; then
while IFS= read -r -d '' backup; do
local backup_time=$(stat -c %Y "$backup")
if [ "$backup_time" -gt "$latest_time" ]; then
latest_time=$backup_time
latest_backup=$backup
fi
done < <(find "$BACKUP_ROOT_DIR/$backup_type" -name "*.$extension" -type f -print0)
fi
echo "$latest_backup"
}
# Function to confirm restoration
confirm_restore() {
local db_file="$1"
local storage_file="$2"
echo ""
echo -e "${YELLOW}WARNING: This will REPLACE current data!${NC}"
echo ""
if [ -n "$db_file" ]; then
echo -e "${RED}Database restore:${NC}"
echo -e " File: $(basename "$db_file")"
echo -e " Size: $(du -h "$db_file" | cut -f1)"
echo -e " Date: $(date -r "$db_file" '+%Y-%m-%d %H:%M:%S')"
echo ""
fi
if [ -n "$storage_file" ]; then
echo -e "${RED}Storage restore:${NC}"
echo -e " File: $(basename "$storage_file")"
echo -e " Size: $(du -h "$storage_file" | cut -f1)"
echo -e " Date: $(date -r "$storage_file" '+%Y-%m-%d %H:%M:%S')"
echo ""
fi
read -p "Are you sure you want to continue? [y/N]: " -n 1 -r
echo ""
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
log "INFO" "Restoration cancelled by user"
exit 0
fi
}
# Function to validate backup files
validate_backup_file() {
local file="$1"
local type="$2"
if [ ! -f "$file" ] || [ ! -r "$file" ]; then
log "ERROR" "$type backup file not found or not readable: $file"
return 1
fi
# Test if backup file is valid
case "$type" in
"database")
if ! gzip -t "$file" 2>/dev/null; then
log "ERROR" "Database backup file is corrupted or not a valid gzip file"
return 1
fi
;;
"storage")
if ! tar -tzf "$file" >/dev/null 2>&1; then
log "ERROR" "Storage backup file is corrupted or not a valid tar.gz file"
return 1
fi
;;
esac
return 0
}
# Function to restore database
restore_database() {
local db_file="$1"
log "INFO" "Starting database restoration..."
if ! validate_backup_file "$db_file" "database"; then
return 1
fi
# Use the database restore script
if [ -x "$SCRIPT_DIR/restore-database.sh" ]; then
log "INFO" "Using database restore script"
if "$SCRIPT_DIR/restore-database.sh" "$db_file" --confirm; then
log "SUCCESS" "Database restoration completed"
return 0
else
log "ERROR" "Database restoration failed"
return 1
fi
else
log "ERROR" "Database restore script not found or not executable"
return 1
fi
}
# Function to restore storage
restore_storage() {
local storage_file="$1"
log "INFO" "Starting storage restoration..."
if ! validate_backup_file "$storage_file" "storage"; then
return 1
fi
# Use the storage restore script
if [ -x "$SCRIPT_DIR/restore-storage.sh" ]; then
log "INFO" "Using storage restore script"
if "$SCRIPT_DIR/restore-storage.sh" "$storage_file" --confirm; then
log "SUCCESS" "Storage restoration completed"
return 0
else
log "ERROR" "Storage restoration failed"
return 1
fi
else
log "ERROR" "Storage restore script not found or not executable"
return 1
fi
}
# Function to perform post-restore tasks
post_restore_tasks() {
log "INFO" "Running post-restore tasks..."
# Laravel optimization commands
if [ -f "$PROJECT_ROOT/artisan" ]; then
log "INFO" "Running Laravel optimization commands..."
# Change to project directory
cd "$PROJECT_ROOT"
# Clear caches
php artisan config:clear 2>/dev/null || true
php artisan cache:clear 2>/dev/null || true
php artisan view:clear 2>/dev/null || true
php artisan route:clear 2>/dev/null || true
# Check migration status
log "INFO" "Checking database migration status..."
php artisan migrate:status 2>/dev/null || log "WARNING" "Could not check migration status"
log "SUCCESS" "Laravel optimization completed"
else
log "WARNING" "Laravel artisan not found, skipping optimization"
fi
}
# Parse command line arguments
DATABASE_FILE=""
STORAGE_FILE=""
DATABASE_LATEST=false
STORAGE_LATEST=false
LATEST=false
CONFIRM=false
LIST_BACKUPS=false
while [[ $# -gt 0 ]]; do
case $1 in
--database-file)
DATABASE_FILE="$2"
shift 2
;;
--storage-file)
STORAGE_FILE="$2"
shift 2
;;
--database-latest)
DATABASE_LATEST=true
shift
;;
--storage-latest)
STORAGE_LATEST=true
shift
;;
--latest)
LATEST=true
shift
;;
--confirm)
CONFIRM=true
shift
;;
--list-backups)
LIST_BACKUPS=true
shift
;;
--help)
show_usage
;;
*)
log "ERROR" "Unknown option: $1"
show_usage
;;
esac
done
# Handle list backups option
if [ "$LIST_BACKUPS" = true ]; then
list_backups
exit 0
fi
# Handle latest option
if [ "$LATEST" = true ]; then
DATABASE_LATEST=true
STORAGE_LATEST=true
fi
# Determine backup files
if [ "$DATABASE_LATEST" = true ]; then
DATABASE_FILE=$(get_latest_backup "database" "sql.gz")
if [ -z "$DATABASE_FILE" ]; then
log "ERROR" "No database backups found"
exit 1
fi
log "INFO" "Using latest database backup: $(basename "$DATABASE_FILE")"
fi
if [ "$STORAGE_LATEST" = true ]; then
STORAGE_FILE=$(get_latest_backup "storage" "tar.gz")
if [ -z "$STORAGE_FILE" ]; then
log "ERROR" "No storage backups found"
exit 1
fi
log "INFO" "Using latest storage backup: $(basename "$STORAGE_FILE")"
fi
# Validate that at least one restoration is requested
if [ -z "$DATABASE_FILE" ] && [ -z "$STORAGE_FILE" ]; then
log "ERROR" "No restoration specified. Use --help for usage information."
exit 1
fi
# Resolve relative paths
if [ -n "$DATABASE_FILE" ] && [[ "$DATABASE_FILE" != /* ]]; then
if [ -f "$BACKUP_ROOT_DIR/$DATABASE_FILE" ]; then
DATABASE_FILE="$BACKUP_ROOT_DIR/$DATABASE_FILE"
elif [ -f "$BACKUP_ROOT_DIR/database/daily/$DATABASE_FILE" ]; then
DATABASE_FILE="$BACKUP_ROOT_DIR/database/daily/$DATABASE_FILE"
elif [ -f "$BACKUP_ROOT_DIR/database/weekly/$DATABASE_FILE" ]; then
DATABASE_FILE="$BACKUP_ROOT_DIR/database/weekly/$DATABASE_FILE"
elif [ -f "$BACKUP_ROOT_DIR/database/monthly/$DATABASE_FILE" ]; then
DATABASE_FILE="$BACKUP_ROOT_DIR/database/monthly/$DATABASE_FILE"
elif [ -f "$DATABASE_FILE" ]; then
DATABASE_FILE="$(realpath "$DATABASE_FILE")"
else
log "ERROR" "Database backup file not found: $DATABASE_FILE"
log "INFO" "Checked locations:"
log "INFO" " $BACKUP_ROOT_DIR/$DATABASE_FILE"
log "INFO" " $BACKUP_ROOT_DIR/database/daily/$DATABASE_FILE"
log "INFO" " $BACKUP_ROOT_DIR/database/weekly/$DATABASE_FILE"
log "INFO" " $BACKUP_ROOT_DIR/database/monthly/$DATABASE_FILE"
log "INFO" " ./$DATABASE_FILE"
exit 1
fi
fi
if [ -n "$STORAGE_FILE" ] && [[ "$STORAGE_FILE" != /* ]]; then
if [ -f "$BACKUP_ROOT_DIR/$STORAGE_FILE" ]; then
STORAGE_FILE="$BACKUP_ROOT_DIR/$STORAGE_FILE"
elif [ -f "$BACKUP_ROOT_DIR/storage/daily/$STORAGE_FILE" ]; then
STORAGE_FILE="$BACKUP_ROOT_DIR/storage/daily/$STORAGE_FILE"
elif [ -f "$BACKUP_ROOT_DIR/storage/weekly/$STORAGE_FILE" ]; then
STORAGE_FILE="$BACKUP_ROOT_DIR/storage/weekly/$STORAGE_FILE"
elif [ -f "$BACKUP_ROOT_DIR/storage/monthly/$STORAGE_FILE" ]; then
STORAGE_FILE="$BACKUP_ROOT_DIR/storage/monthly/$STORAGE_FILE"
elif [ -f "$STORAGE_FILE" ]; then
STORAGE_FILE="$(realpath "$STORAGE_FILE")"
else
log "ERROR" "Storage backup file not found: $STORAGE_FILE"
log "INFO" "Checked locations:"
log "INFO" " $BACKUP_ROOT_DIR/$STORAGE_FILE"
log "INFO" " $BACKUP_ROOT_DIR/storage/daily/$STORAGE_FILE"
log "INFO" " $BACKUP_ROOT_DIR/storage/weekly/$STORAGE_FILE"
log "INFO" " $BACKUP_ROOT_DIR/storage/monthly/$STORAGE_FILE"
log "INFO" " ./$STORAGE_FILE"
exit 1
fi
fi
# Main execution
main() {
log "INFO" "============================================"
log "INFO" "Starting complete system restoration"
log "INFO" "Time: $(date)"
log "INFO" "============================================"
local start_time=$(date +%s)
local overall_success=true
# Confirm restoration unless --confirm was specified
if [ "$CONFIRM" = false ]; then
confirm_restore "$DATABASE_FILE" "$STORAGE_FILE"
fi
# Restore database
if [ -n "$DATABASE_FILE" ]; then
if ! restore_database "$DATABASE_FILE"; then
overall_success=false
log "ERROR" "Database restoration failed"
fi
else
log "INFO" "Skipping database restoration (not requested)"
fi
# Restore storage
if [ -n "$STORAGE_FILE" ]; then
if ! restore_storage "$STORAGE_FILE"; then
overall_success=false
log "ERROR" "Storage restoration failed"
fi
else
log "INFO" "Skipping storage restoration (not requested)"
fi
# Post-restore tasks
if [ "$overall_success" = true ]; then
post_restore_tasks
local end_time=$(date +%s)
local execution_time=$((end_time - start_time))
local execution_time_formatted=$(date -d@$execution_time -u +%H:%M:%S)
log "SUCCESS" "Complete system restoration finished in $execution_time_formatted"
# Send notification
if command -v mail >/dev/null 2>&1; then
local restored_items=""
if [ -n "$DATABASE_FILE" ]; then
restored_items="Database: $(basename "$DATABASE_FILE")"
fi
if [ -n "$STORAGE_FILE" ]; then
if [ -n "$restored_items" ]; then
restored_items="$restored_items, Storage: $(basename "$STORAGE_FILE")"
else
restored_items="Storage: $(basename "$STORAGE_FILE")"
fi
fi
echo "Complete system restoration finished successfully at $(date). Restored: $restored_items" | mail -s "Timebank Complete Restore Success" "${BACKUP_NOTIFY_EMAIL:-$USER@localhost}" 2>/dev/null || true
fi
echo ""
log "INFO" "Restoration summary:"
if [ -n "$DATABASE_FILE" ]; then
log "INFO" " Database restored from: $(basename "$DATABASE_FILE")"
fi
if [ -n "$STORAGE_FILE" ]; then
log "INFO" " Storage restored from: $(basename "$STORAGE_FILE")"
fi
echo ""
log "INFO" "Recommended next steps:"
log "INFO" " 1. Test application functionality"
log "INFO" " 2. Verify file permissions are correct"
log "INFO" " 3. Check that all services are running properly"
log "INFO" " 4. Review any Laravel logs for issues"
else
log "ERROR" "System restoration completed with errors"
exit 1
fi
log "INFO" "============================================"
log "INFO" "Restoration process finished"
log "INFO" "============================================"
}
# Load environment loader for database credentials (if needed)
if [ -f "$SCRIPT_DIR/load-env.sh" ]; then
source "$SCRIPT_DIR/load-env.sh"
fi
# Run main function
main
scripts/restore-database.sh
Executable
+441
View File
@@ -0,0 +1,441 @@
#!/bin/bash
# Laravel Timebank Database Restore Script
# Restores MySQL database from compressed backup
# Usage: ./restore-database.sh [backup_file] [options]
# Options: --confirm, --list-backups, --latest
set -e # Exit on any error
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
BACKUP_ROOT_DIR="$PROJECT_ROOT/backups"
LOG_FILE="$BACKUP_ROOT_DIR/restore.log"
# Create log directory
mkdir -p "$BACKUP_ROOT_DIR/logs"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Logging function
log() {
local level="$1"
local message="$2"
local timestamp="[$(date '+%Y-%m-%d %H:%M:%S')]"
case "$level" in
"INFO")
echo -e "${timestamp} ${BLUE}[INFO]${NC} $message" | tee -a "$LOG_FILE"
;;
"SUCCESS")
echo -e "${timestamp} ${GREEN}[SUCCESS]${NC} $message" | tee -a "$LOG_FILE"
;;
"WARNING")
echo -e "${timestamp} ${YELLOW}[WARNING]${NC} $message" | tee -a "$LOG_FILE"
;;
"ERROR")
echo -e "${timestamp} ${RED}[ERROR]${NC} $message" | tee -a "$LOG_FILE"
;;
*)
echo "$timestamp $message" | tee -a "$LOG_FILE"
;;
esac
}
# Function to show usage
show_usage() {
echo "Usage: $0 [backup_file] [options]"
echo ""
echo "Arguments:"
echo " backup_file - Path to backup file (relative to backup directory or absolute)"
echo ""
echo "Options:"
echo " --confirm - Skip confirmation prompt"
echo " --list-backups - List available backups"
echo " --latest - Restore from latest backup"
echo " --help - Show this help message"
echo ""
echo "Examples:"
echo " $0 --list-backups # List available backups"
echo " $0 --latest # Restore latest backup"
echo " $0 database/daily/mydb_daily_20240101.sql.gz # Restore specific backup"
echo ""
exit 0
}
# Function to list available backups
list_backups() {
log "INFO" "Available database backups:"
echo ""
if [ ! -d "$BACKUP_ROOT_DIR/database" ]; then
log "WARNING" "No backup directory found"
return 0
fi
local backup_found=false
for backup_type in daily weekly monthly; do
local backup_dir="$BACKUP_ROOT_DIR/database/$backup_type"
if [ -d "$backup_dir" ]; then
local backups=($(find "$backup_dir" -name "*.sql.gz" -type f | sort -r))
if [ ${#backups[@]} -gt 0 ]; then
echo -e "${BLUE}$backup_type backups:${NC}"
for backup in "${backups[@]}"; do
local size=$(du -h "$backup" | cut -f1)
local date_created=$(date -r "$backup" '+%Y-%m-%d %H:%M:%S')
echo " $(basename "$backup") ($size, $date_created)"
done
echo ""
backup_found=true
fi
fi
done
if [ "$backup_found" = false ]; then
log "WARNING" "No database backups found"
fi
}
# Function to find latest backup
get_latest_backup() {
local latest_backup=""
local latest_time=0
if [ -d "$BACKUP_ROOT_DIR/database" ]; then
while IFS= read -r -d '' backup; do
local backup_time=$(stat -c %Y "$backup")
if [ "$backup_time" -gt "$latest_time" ]; then
latest_time=$backup_time
latest_backup=$backup
fi
done < <(find "$BACKUP_ROOT_DIR/database" -name "*.sql.gz" -type f -print0)
fi
echo "$latest_backup"
}
# Function to confirm restore
confirm_restore() {
local backup_file="$1"
echo ""
echo -e "${YELLOW}WARNING: This will REPLACE the current database!${NC}"
echo -e "Database: ${RED}$DB_DATABASE${NC}"
echo -e "Backup file: ${BLUE}$(basename "$backup_file")${NC}"
echo -e "Backup size: $(du -h "$backup_file" | cut -f1)"
echo -e "Backup date: $(date -r "$backup_file" '+%Y-%m-%d %H:%M:%S')"
echo ""
read -p "Are you sure you want to continue? [y/N]: " -n 1 -r
echo ""
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
log "INFO" "Restore cancelled by user"
exit 0
fi
}
# Function to create database backup before restore
create_pre_restore_backup() {
log "INFO" "Creating pre-restore backup of current database"
local pre_restore_dir="$BACKUP_ROOT_DIR/database/pre-restore"
mkdir -p "$pre_restore_dir"
local timestamp=$(date '+%Y%m%d_%H%M%S')
local backup_file="$pre_restore_dir/${DB_DATABASE}_pre_restore_${timestamp}.sql.gz"
# Use the database backup script if available
if [ -x "$SCRIPT_DIR/backup-database.sh" ]; then
log "INFO" "Using backup script for pre-restore backup"
# Create MySQL configuration file for secure password handling
local mysql_cnf_file="/tmp/mysql_pre_restore_$$.cnf"
cat > "$mysql_cnf_file" <<EOF
[mysqldump]
host=${DB_HOST:-127.0.0.1}
port=${DB_PORT:-3306}
user=$DB_USERNAME
password=$DB_PASSWORD
EOF
# Set secure permissions on the config file
chmod 600 "$mysql_cnf_file"
# Create a temporary backup.
#
# No --events: it needs the EVENT privilege the application user does
# not have, and the schema defines no events. See backup-database.sh.
#
# pipefail matters here: with `mysqldump | gzip` the pipeline reports
# gzip's status, so a failed dump still looked successful and wrote a
# valid gzip of a TRUNCATED dump - the worst possible outcome for a
# pre-restore safety net, since it is taken precisely when the database
# is about to be overwritten.
set +e
set -o pipefail
mysqldump \
--defaults-extra-file="$mysql_cnf_file" \
--single-transaction \
--routines \
--triggers \
--add-drop-database \
--databases "$DB_DATABASE" 2> "${backup_file}.err" | gzip > "$backup_file"
local dump_status=$?
set +o pipefail
set -e
# Clean up the temporary config file
rm -f "$mysql_cnf_file"
if [ $dump_status -ne 0 ]; then
log "ERROR" "Pre-restore backup failed: $(tr '\n' ' ' < "${backup_file}.err")"
rm -f "$backup_file" "${backup_file}.err"
return 1
fi
rm -f "${backup_file}.err"
# A pre-restore backup that is truncated is worse than none: it would
# be trusted at exactly the moment it is needed.
if ! gzip -dc "$backup_file" | tail -c 200 | grep -q "Dump completed"; then
log "ERROR" "Pre-restore backup is truncated - discarding $backup_file"
rm -f "$backup_file"
return 1
fi
if [ -f "$backup_file" ] && [ -s "$backup_file" ]; then
log "SUCCESS" "Pre-restore backup created: $backup_file"
else
log "ERROR" "Pre-restore backup failed"
return 1
fi
fi
}
# Function to restore database
restore_database() {
local backup_file="$1"
log "INFO" "Starting database restore from: $(basename "$backup_file")"
# Verify backup file exists and is readable
if [ ! -f "$backup_file" ] || [ ! -r "$backup_file" ]; then
log "ERROR" "Backup file not found or not readable: $backup_file"
return 1
fi
# Test if backup file is valid gzip
if ! gzip -t "$backup_file" 2>/dev/null; then
log "ERROR" "Backup file is corrupted or not a valid gzip file"
return 1
fi
# Extract and restore
log "INFO" "Decompressing and restoring backup..."
# The dump drops and recreates the database and its tables. The application user may not do that
# (no DROP privilege, and UPDATE/DELETE only per table), so restore as the privileged user deploy.sh
# uses for migrations: DB_DEPLOY_USERNAME / DB_DEPLOY_PASSWORD from .env, or ask for one.
local restore_user="${DB_DEPLOY_USERNAME:-}"
local restore_password="${DB_DEPLOY_PASSWORD:-}"
if [ -z "$restore_user" ] && [ -t 0 ]; then
echo "A MySQL user that may drop and create the database is needed (root or a deployment user)."
read -r -p "MySQL username [root]: " restore_user
restore_user="${restore_user:-root}"
fi
if [ -n "$restore_user" ] && [ -z "$restore_password" ] && [ -t 0 ]; then
read -r -s -p "MySQL password for $restore_user: " restore_password
echo ""
fi
if [ -z "$restore_user" ]; then
log "WARNING" "No DB_DEPLOY_USERNAME and no terminal to ask for one: restoring as $DB_USERNAME, which usually lacks the DROP privilege"
restore_user="$DB_USERNAME"
restore_password="$DB_PASSWORD"
fi
log "INFO" "Restoring as MySQL user $restore_user"
# Create MySQL configuration file for secure password handling (quoted: passwords may hold # or ;)
local mysql_cnf_file="/tmp/mysql_restore_$$.cnf"
local quoted_password="${restore_password//\\/\\\\}"
quoted_password="${quoted_password//\"/\\\"}"
( umask 077; printf '[mysql]\nhost=%s\nport=%s\nuser=%s\npassword="%s"\n' \
"${DB_HOST:-127.0.0.1}" "${DB_PORT:-3306}" "$restore_user" "$quoted_password" > "$mysql_cnf_file" )
# Set secure permissions on the config file
chmod 600 "$mysql_cnf_file"
# Perform the restore using the config file
if gzip -dc "$backup_file" | mysql --defaults-extra-file="$mysql_cnf_file"; then
log "SUCCESS" "Database restore completed successfully"
rm -f "$mysql_cnf_file"
return 0
else
log "ERROR" "Database restore failed"
rm -f "$mysql_cnf_file"
return 1
fi
}
# Parse command line arguments
BACKUP_FILE=""
CONFIRM=false
LIST_BACKUPS=false
LATEST=false
while [[ $# -gt 0 ]]; do
case $1 in
--confirm)
CONFIRM=true
shift
;;
--list-backups)
LIST_BACKUPS=true
shift
;;
--latest)
LATEST=true
shift
;;
--help)
show_usage
;;
-*)
log "ERROR" "Unknown option: $1"
show_usage
;;
*)
if [ -z "$BACKUP_FILE" ]; then
BACKUP_FILE="$1"
else
log "ERROR" "Multiple backup files specified"
show_usage
fi
shift
;;
esac
done
# Load environment loader
source "$SCRIPT_DIR/load-env.sh"
# Load environment variables
if ! load_env "$PROJECT_ROOT/.env"; then
log "ERROR" ".env file not found in $PROJECT_ROOT"
exit 1
fi
# Validate required environment variables
if [ -z "$DB_DATABASE" ] || [ -z "$DB_USERNAME" ] || [ -z "$DB_HOST" ]; then
log "ERROR" "Required database environment variables not found (DB_DATABASE, DB_USERNAME, DB_HOST)"
exit 1
fi
# Handle list backups option
if [ "$LIST_BACKUPS" = true ]; then
list_backups
exit 0
fi
# Handle latest backup option
if [ "$LATEST" = true ]; then
BACKUP_FILE=$(get_latest_backup)
if [ -z "$BACKUP_FILE" ]; then
log "ERROR" "No backups found"
exit 1
fi
log "INFO" "Using latest backup: $(basename "$BACKUP_FILE")"
fi
# Validate backup file argument
if [ -z "$BACKUP_FILE" ]; then
log "ERROR" "No backup file specified"
show_usage
fi
# Resolve backup file path
if [[ "$BACKUP_FILE" != /* ]]; then
# Relative path - check if it exists relative to backup directory
if [ -f "$BACKUP_ROOT_DIR/$BACKUP_FILE" ]; then
BACKUP_FILE="$BACKUP_ROOT_DIR/$BACKUP_FILE"
elif [ -f "$BACKUP_ROOT_DIR/database/daily/$BACKUP_FILE" ]; then
BACKUP_FILE="$BACKUP_ROOT_DIR/database/daily/$BACKUP_FILE"
elif [ -f "$BACKUP_ROOT_DIR/database/weekly/$BACKUP_FILE" ]; then
BACKUP_FILE="$BACKUP_ROOT_DIR/database/weekly/$BACKUP_FILE"
elif [ -f "$BACKUP_ROOT_DIR/database/monthly/$BACKUP_FILE" ]; then
BACKUP_FILE="$BACKUP_ROOT_DIR/database/monthly/$BACKUP_FILE"
elif [ -f "$BACKUP_FILE" ]; then
# Exists relative to current directory
BACKUP_FILE="$(realpath "$BACKUP_FILE")"
else
log "ERROR" "Database backup file not found: $BACKUP_FILE"
log "INFO" "Checked locations:"
log "INFO" " $BACKUP_ROOT_DIR/$BACKUP_FILE"
log "INFO" " $BACKUP_ROOT_DIR/database/daily/$BACKUP_FILE"
log "INFO" " $BACKUP_ROOT_DIR/database/weekly/$BACKUP_FILE"
log "INFO" " $BACKUP_ROOT_DIR/database/monthly/$BACKUP_FILE"
log "INFO" " ./$BACKUP_FILE"
exit 1
fi
fi
# Main execution
main() {
log "INFO" "============================================"
log "INFO" "Starting database restore process"
log "INFO" "Time: $(date)"
log "INFO" "============================================"
local start_time=$(date +%s)
# Confirm restore unless --confirm was specified
if [ "$CONFIRM" = false ]; then
confirm_restore "$BACKUP_FILE"
fi
# Create pre-restore backup
if ! create_pre_restore_backup; then
log "WARNING" "Pre-restore backup failed, but continuing with restore"
fi
# Perform restore
if restore_database "$BACKUP_FILE"; then
local end_time=$(date +%s)
local execution_time=$((end_time - start_time))
local execution_time_formatted=$(date -d@$execution_time -u +%H:%M:%S)
log "SUCCESS" "Database restore completed successfully in $execution_time_formatted"
log "INFO" "Restored from: $(basename "$BACKUP_FILE")"
# Send notification
if command -v mail >/dev/null 2>&1; then
echo "Database restore completed successfully at $(date). Restored from: $(basename "$BACKUP_FILE")" | mail -s "Timebank DB Restore Success" "${BACKUP_NOTIFY_EMAIL:-$USER@localhost}" 2>/dev/null || true
fi
# Recommend running Laravel commands
echo ""
log "INFO" "Recommended post-restore steps:"
log "INFO" " 1. php artisan migrate:status"
log "INFO" " 2. php artisan config:clear"
log "INFO" " 3. php artisan cache:clear"
log "INFO" " 4. Test application functionality"
else
log "ERROR" "Database restore failed"
exit 1
fi
log "INFO" "============================================"
log "INFO" "Restore process finished"
log "INFO" "============================================"
}
# Run main function
main
scripts/restore-storage.sh
Executable
+457
View File
@@ -0,0 +1,457 @@
#!/bin/bash
# Laravel Timebank Storage Restore Script
# Restores storage directory from compressed backup
# Usage: ./restore-storage.sh [backup_file] [options]
# Options: --confirm, --list-backups, --latest, --merge
set -e # Exit on any error
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
STORAGE_DIR="$PROJECT_ROOT/storage"
BACKUP_ROOT_DIR="$PROJECT_ROOT/backups"
LOG_FILE="$BACKUP_ROOT_DIR/restore.log"
# Create log directory
mkdir -p "$BACKUP_ROOT_DIR/logs"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Logging function
log() {
local level="$1"
local message="$2"
local timestamp="[$(date '+%Y-%m-%d %H:%M:%S')]"
case "$level" in
"INFO")
echo -e "${timestamp} ${BLUE}[INFO]${NC} $message" | tee -a "$LOG_FILE"
;;
"SUCCESS")
echo -e "${timestamp} ${GREEN}[SUCCESS]${NC} $message" | tee -a "$LOG_FILE"
;;
"WARNING")
echo -e "${timestamp} ${YELLOW}[WARNING]${NC} $message" | tee -a "$LOG_FILE"
;;
"ERROR")
echo -e "${timestamp} ${RED}[ERROR]${NC} $message" | tee -a "$LOG_FILE"
;;
*)
echo "$timestamp $message" | tee -a "$LOG_FILE"
;;
esac
}
# Function to show usage
show_usage() {
echo "Usage: $0 [backup_file] [options]"
echo ""
echo "Arguments:"
echo " backup_file - Path to backup file (relative to backup directory or absolute)"
echo ""
echo "Options:"
echo " --confirm - Skip confirmation prompt"
echo " --list-backups - List available backups"
echo " --latest - Restore from latest backup"
echo " --merge - Merge with existing storage (don't delete existing files)"
echo " --help - Show this help message"
echo ""
echo "Examples:"
echo " $0 --list-backups # List available backups"
echo " $0 --latest # Restore latest backup"
echo " $0 --latest --merge # Merge latest backup with existing"
echo " $0 storage/daily/daily_20240101_120000.tar.gz # Restore specific backup"
echo ""
exit 0
}
# Function to list available backups
list_backups() {
log "INFO" "Available storage backups:"
echo ""
if [ ! -d "$BACKUP_ROOT_DIR/storage" ]; then
log "WARNING" "No storage backup directory found"
return 0
fi
local backup_found=false
for backup_type in daily weekly monthly; do
local backup_dir="$BACKUP_ROOT_DIR/storage/$backup_type"
if [ -d "$backup_dir" ]; then
local backups=($(find "$backup_dir" -name "*.tar.gz" -type f | sort -r))
if [ ${#backups[@]} -gt 0 ]; then
echo -e "${BLUE}$backup_type backups:${NC}"
for backup in "${backups[@]}"; do
local size=$(du -h "$backup" | cut -f1)
local date_created=$(date -r "$backup" '+%Y-%m-%d %H:%M:%S')
echo " $(basename "$backup") ($size, $date_created)"
done
echo ""
backup_found=true
fi
fi
done
if [ "$backup_found" = false ]; then
log "WARNING" "No storage backups found"
fi
}
# Function to find latest backup
get_latest_backup() {
local latest_backup=""
local latest_time=0
if [ -d "$BACKUP_ROOT_DIR/storage" ]; then
while IFS= read -r -d '' backup; do
local backup_time=$(stat -c %Y "$backup")
if [ "$backup_time" -gt "$latest_time" ]; then
latest_time=$backup_time
latest_backup=$backup
fi
done < <(find "$BACKUP_ROOT_DIR/storage" -name "*.tar.gz" -type f -print0)
fi
echo "$latest_backup"
}
# Function to confirm restore
confirm_restore() {
local backup_file="$1"
local merge_mode="$2"
echo ""
if [ "$merge_mode" = true ]; then
echo -e "${YELLOW}WARNING: This will MERGE the backup with existing storage files!${NC}"
echo -e "Mode: ${GREEN}Merge${NC} (existing files will be preserved unless overwritten)"
else
echo -e "${RED}WARNING: This will REPLACE the entire storage directory!${NC}"
echo -e "Mode: ${RED}Full Replace${NC} (all existing files will be deleted)"
fi
echo -e "Storage directory: ${BLUE}$STORAGE_DIR${NC}"
echo -e "Backup file: ${BLUE}$(basename "$backup_file")${NC}"
echo -e "Backup size: $(du -h "$backup_file" | cut -f1)"
echo -e "Backup date: $(date -r "$backup_file" '+%Y-%m-%d %H:%M:%S')"
echo ""
read -p "Are you sure you want to continue? [y/N]: " -n 1 -r
echo ""
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
log "INFO" "Restore cancelled by user"
exit 0
fi
}
# Function to create storage backup before restore
create_pre_restore_backup() {
log "INFO" "Creating pre-restore backup of current storage"
if [ ! -d "$STORAGE_DIR" ]; then
log "WARNING" "Storage directory doesn't exist, skipping pre-restore backup"
return 0
fi
local pre_restore_dir="$BACKUP_ROOT_DIR/storage/pre-restore"
mkdir -p "$pre_restore_dir"
local timestamp=$(date '+%Y%m%d_%H%M%S')
local backup_file="$pre_restore_dir/storage_pre_restore_${timestamp}.tar.gz"
log "INFO" "Creating compressed archive of current storage..."
if tar -czf "$backup_file" -C "$PROJECT_ROOT" "storage" 2>/dev/null; then
if [ -f "$backup_file" ] && [ -s "$backup_file" ]; then
local backup_size=$(du -h "$backup_file" | cut -f1)
log "SUCCESS" "Pre-restore backup created: $(basename "$backup_file") ($backup_size)"
return 0
fi
fi
log "WARNING" "Pre-restore backup failed"
return 1
}
# Function to restore storage
restore_storage() {
local backup_file="$1"
local merge_mode="$2"
log "INFO" "Starting storage restore from: $(basename "$backup_file")"
# Verify backup file exists and is readable
if [ ! -f "$backup_file" ] || [ ! -r "$backup_file" ]; then
log "ERROR" "Backup file not found or not readable: $backup_file"
return 1
fi
# Test if backup file is valid tar.gz
if ! tar -tzf "$backup_file" >/dev/null 2>&1; then
log "ERROR" "Backup file is corrupted or not a valid tar.gz file"
return 1
fi
# Create temporary extraction directory
local temp_dir="/tmp/timebank_restore_$$"
mkdir -p "$temp_dir"
# Extract backup to temporary directory
log "INFO" "Extracting backup archive..."
if tar -xzf "$backup_file" -C "$temp_dir" 2>/dev/null; then
log "SUCCESS" "Backup extracted successfully"
else
log "ERROR" "Failed to extract backup archive"
rm -rf "$temp_dir"
return 1
fi
# Find the extracted storage directory
local extracted_storage=""
# Check for different possible structures in the archive
log "INFO" "Checking extracted structure in: $temp_dir"
log "INFO" "Contents: $(ls -la "$temp_dir" | head -10)"
if [ -d "$temp_dir/storage" ]; then
log "INFO" "Found direct storage directory"
extracted_storage="$temp_dir/storage"
elif [ -d "$temp_dir/daily_"* ] || [ -d "$temp_dir/weekly_"* ] || [ -d "$temp_dir/monthly_"* ]; then
log "INFO" "Found timestamped directory structure"
# Handle snapshot-based backups - look for storage directory inside timestamped folder
local snapshot_dir=$(find "$temp_dir" -maxdepth 1 -type d \( -name "daily_*" -o -name "weekly_*" -o -name "monthly_*" \) | head -n 1)
log "INFO" "Snapshot directory found: $snapshot_dir"
if [ -n "$snapshot_dir" ] && [ -d "$snapshot_dir/storage" ]; then
log "INFO" "Found storage subdirectory in snapshot"
extracted_storage="$snapshot_dir/storage"
elif [ -n "$snapshot_dir" ]; then
log "INFO" "Using snapshot directory contents directly"
# If no storage subdirectory, use the snapshot dir directly (it may contain the files)
extracted_storage="$snapshot_dir"
fi
elif [ -d "$temp_dir/timebank_storage_backup_"* ]; then
log "INFO" "Found full backup structure"
# Handle full backup structure
local full_backup_dir=$(find "$temp_dir" -maxdepth 1 -type d -name "timebank_storage_backup_*" | head -n 1)
if [ -n "$full_backup_dir" ]; then
extracted_storage="$full_backup_dir"
fi
else
log "WARNING" "No recognized backup structure found"
fi
if [ -z "$extracted_storage" ] || [ ! -d "$extracted_storage" ]; then
log "ERROR" "Could not find storage directory in backup archive"
rm -rf "$temp_dir"
return 1
fi
log "INFO" "Found extracted storage at: $extracted_storage"
log "INFO" "Contents of extracted storage: $(ls -la "$extracted_storage" | head -5)"
# Restore storage based on mode
if [ "$merge_mode" = true ]; then
log "INFO" "Performing merge restore (existing files preserved)"
# Create storage directory if it doesn't exist
mkdir -p "$STORAGE_DIR"
# Copy files, preserving existing ones
if rsync -av --ignore-existing "$extracted_storage/" "$STORAGE_DIR/" 2>&1 | tee -a "$LOG_FILE"; then
log "SUCCESS" "Storage merge completed"
else
log "ERROR" "Storage merge failed"
rm -rf "$temp_dir"
return 1
fi
else
log "INFO" "Performing full restore (replacing existing storage)"
# Create storage directory (rsync will handle the rest)
mkdir -p "$STORAGE_DIR"
# Copy extracted storage contents to final location
if rsync -av --delete "$extracted_storage/" "$STORAGE_DIR/" 2>&1 | tee -a "$LOG_FILE"; then
log "SUCCESS" "Storage restore completed"
else
log "ERROR" "Storage restore failed"
rm -rf "$temp_dir"
return 1
fi
fi
# Set correct permissions
log "INFO" "Setting storage permissions..."
chmod -R 755 "$STORAGE_DIR"
find "$STORAGE_DIR" -type f -exec chmod 644 {} \;
# Create required Laravel storage directories if they don't exist
mkdir -p "$STORAGE_DIR"/{app/public,framework/{cache,sessions,testing,views},logs}
# Clean up temporary directory
rm -rf "$temp_dir"
log "SUCCESS" "Storage restore process completed"
return 0
}
# Parse command line arguments
BACKUP_FILE=""
CONFIRM=false
LIST_BACKUPS=false
LATEST=false
MERGE=false
while [[ $# -gt 0 ]]; do
case $1 in
--confirm)
CONFIRM=true
shift
;;
--list-backups)
LIST_BACKUPS=true
shift
;;
--latest)
LATEST=true
shift
;;
--merge)
MERGE=true
shift
;;
--help)
show_usage
;;
-*)
log "ERROR" "Unknown option: $1"
show_usage
;;
*)
if [ -z "$BACKUP_FILE" ]; then
BACKUP_FILE="$1"
else
log "ERROR" "Multiple backup files specified"
show_usage
fi
shift
;;
esac
done
# Handle list backups option
if [ "$LIST_BACKUPS" = true ]; then
list_backups
exit 0
fi
# Handle latest backup option
if [ "$LATEST" = true ]; then
BACKUP_FILE=$(get_latest_backup)
if [ -z "$BACKUP_FILE" ]; then
log "ERROR" "No storage backups found"
exit 1
fi
log "INFO" "Using latest backup: $(basename "$BACKUP_FILE")"
fi
# Validate backup file argument
if [ -z "$BACKUP_FILE" ]; then
log "ERROR" "No backup file specified"
show_usage
fi
# Resolve backup file path
if [[ "$BACKUP_FILE" != /* ]]; then
# Relative path - check if it exists relative to backup directory
if [ -f "$BACKUP_ROOT_DIR/$BACKUP_FILE" ]; then
BACKUP_FILE="$BACKUP_ROOT_DIR/$BACKUP_FILE"
elif [ -f "$BACKUP_ROOT_DIR/storage/daily/$BACKUP_FILE" ]; then
BACKUP_FILE="$BACKUP_ROOT_DIR/storage/daily/$BACKUP_FILE"
elif [ -f "$BACKUP_ROOT_DIR/storage/weekly/$BACKUP_FILE" ]; then
BACKUP_FILE="$BACKUP_ROOT_DIR/storage/weekly/$BACKUP_FILE"
elif [ -f "$BACKUP_ROOT_DIR/storage/monthly/$BACKUP_FILE" ]; then
BACKUP_FILE="$BACKUP_ROOT_DIR/storage/monthly/$BACKUP_FILE"
elif [ -f "$BACKUP_FILE" ]; then
# Exists relative to current directory
BACKUP_FILE="$(realpath "$BACKUP_FILE")"
else
log "ERROR" "Storage backup file not found: $BACKUP_FILE"
log "INFO" "Checked locations:"
log "INFO" " $BACKUP_ROOT_DIR/$BACKUP_FILE"
log "INFO" " $BACKUP_ROOT_DIR/storage/daily/$BACKUP_FILE"
log "INFO" " $BACKUP_ROOT_DIR/storage/weekly/$BACKUP_FILE"
log "INFO" " $BACKUP_ROOT_DIR/storage/monthly/$BACKUP_FILE"
log "INFO" " ./$BACKUP_FILE"
exit 1
fi
fi
# Main execution
main() {
log "INFO" "============================================"
log "INFO" "Starting storage restore process"
log "INFO" "Time: $(date)"
log "INFO" "============================================"
local start_time=$(date +%s)
# Confirm restore unless --confirm was specified
if [ "$CONFIRM" = false ]; then
confirm_restore "$BACKUP_FILE" "$MERGE"
fi
# Create pre-restore backup
if ! create_pre_restore_backup; then
log "WARNING" "Pre-restore backup failed, but continuing with restore"
fi
# Perform restore
if restore_storage "$BACKUP_FILE" "$MERGE"; then
local end_time=$(date +%s)
local execution_time=$((end_time - start_time))
local execution_time_formatted=$(date -d@$execution_time -u +%H:%M:%S)
log "SUCCESS" "Storage restore completed successfully in $execution_time_formatted"
log "INFO" "Restored from: $(basename "$BACKUP_FILE")"
# Send notification
if command -v mail >/dev/null 2>&1; then
echo "Storage restore completed successfully at $(date). Restored from: $(basename "$BACKUP_FILE")" | mail -s "Timebank Storage Restore Success" "${BACKUP_NOTIFY_EMAIL:-$USER@localhost}" 2>/dev/null || true
fi
# Recommend running Laravel commands
echo ""
log "INFO" "Recommended post-restore steps:"
log "INFO" " 1. php artisan storage:link"
log "INFO" " 2. php artisan config:clear"
log "INFO" " 3. php artisan cache:clear"
log "INFO" " 4. Check file permissions and ownership"
log "INFO" " 5. Test file uploads and media functionality"
# Check if storage link exists
if [ ! -L "$PROJECT_ROOT/public/storage" ]; then
log "WARNING" "Storage symlink missing. Run: php artisan storage:link"
fi
else
log "ERROR" "Storage restore failed"
exit 1
fi
log "INFO" "============================================"
log "INFO" "Restore process finished"
log "INFO" "============================================"
}
# Run main function
main
scripts/send-all-test-emails.sh
Executable
+10
View File
@@ -0,0 +1,10 @@
#!/usr/bin/env bash
# send-all-test-emails.sh — Send all test transactional emails in all 5 languages non-interactively
# Usage: ./scripts/send-all-test-emails.sh
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR/.."
printf "6\ny\n" | bash scripts/test-transactional-emails.sh
scripts/send-test-warnings.php
Executable
+113
View File
@@ -0,0 +1,113 @@
<?php
/**
* Manual Test Script for Inactive profile warning Emails
*
* Usage in tinker:
* php artisan tinker
* include 'send-test-warnings.php';
* sendTestWarnings(102); // Replace 102 with your user ID
*/
function sendTestWarnings($userId) {
$user = \App\Models\User::find($userId);
if (!$user) {
echo "❌ User not found with ID: {$userId}\n";
return;
}
echo " Sending test warning emails for user: {$user->name} (ID: {$userId})\n";
echo " Email: {$user->email}\n";
echo " Language: {$user->lang_preference}\n\n";
// Get accounts and balances
$accounts = [];
$totalBalance = 0;
$profileAccounts = $user->accounts()->active()->notRemoved()->get();
foreach ($profileAccounts as $account) {
// Clear cache to get fresh balance
\Cache::forget("account_balance_{$account->id}");
$balance = $account->balance; // Property, not method
$totalBalance += $balance;
$accounts[] = [
'id' => $account->id,
'name' => $account->name,
'balance' => $balance,
'balanceFormatted' => tbFormat($balance),
];
}
// Test data for each warning level
$warnings = [
'warning_1' => [
'class' => \App\Mail\InactiveProfileWarning1Mail::class,
'timeRemaining' => '2 weeks',
'daysRemaining' => 14,
'daysSinceLogin' => 351,
],
'warning_2' => [
'class' => \App\Mail\InactiveProfileWarning2Mail::class,
'timeRemaining' => '1 week',
'daysRemaining' => 7,
'daysSinceLogin' => 358,
],
'warning_final' => [
'class' => \App\Mail\InactiveProfileWarningFinalMail::class,
'timeRemaining' => '24 hours',
'daysRemaining' => 1,
'daysSinceLogin' => 365,
],
];
// Send all three warning emails
foreach ($warnings as $warningType => $data) {
echo " Dispatching {$warningType}...\n";
$mailClass = $data['class'];
\Illuminate\Support\Facades\Mail::to($user->email)->queue(
new $mailClass(
$user,
'User',
$data['timeRemaining'],
$data['daysRemaining'],
$accounts,
$totalBalance,
$data['daysSinceLogin']
)
);
}
echo "\n✅ All warning emails dispatched to queue\n";
echo " Total balance: " . tbFormat($totalBalance) . "\n";
echo " Accounts: " . count($accounts) . "\n\n";
echo "🚀 Processing queue...\n";
// Process the queue to actually send the emails
\Illuminate\Support\Facades\Artisan::call('queue:work', [
'--stop-when-empty' => true,
'--tries' => 1,
]);
echo "✅ Queue processed. Check your inbox at: {$user->email}\n";
}
// Example usage (uncomment to run):
// sendTestWarnings(102);
echo "
╔════════════════════════════════════════════════════════════╗
║ Inactive profile warning Email Test Script ║
╚════════════════════════════════════════════════════════════╝
Usage:
sendTestWarnings(102); // Replace 102 with user ID
This will send all 3 warning emails:
• Warning 1 (2 weeks remaining)
• Warning 2 (1 week remaining)
• Warning Final (24 hours remaining)
";
scripts/session-manager.php
Executable
+230
View File
@@ -0,0 +1,230 @@
<?php
/**
* Script to manage user sessions
* Usage:
* php scripts/session-manager.php list [user_id] - List sessions for a user (or all if no user_id)
* php scripts/session-manager.php expire [user_id] - Expire all sessions for a user
*/
require __DIR__.'/../vendor/autoload.php';
$app = require_once __DIR__.'/../bootstrap/app.php';
$kernel = $app->make(Illuminate\Contracts\Console\Kernel::class);
$kernel->bootstrap();
if ($argc < 2) {
echo "Usage:\n";
echo " php session-manager.php list [user_id] - List sessions\n";
echo " php session-manager.php expire [user_id] - Expire sessions\n";
exit(1);
}
$action = $argv[1];
$userId = isset($argv[2]) ? (int) $argv[2] : null;
$sessionDriver = config('session.driver');
echo "Session driver: {$sessionDriver}\n\n";
if ($sessionDriver === 'database') {
// Database sessions
$table = config('session.table', 'sessions');
echo "Querying database sessions from table '{$table}'...\n";
$query = DB::table($table);
if ($userId) {
$query->where('user_id', $userId);
} else {
$query->whereNotNull('user_id'); // Only show logged-in sessions
}
$dbSessions = $query->get();
echo "Found " . count($dbSessions) . " session(s)\n\n";
} elseif ($sessionDriver === 'redis') {
// Redis sessions
$redis = \Illuminate\Support\Facades\Redis::connection(config('session.connection') ?: 'default');
$prefix = config('database.redis.options.prefix', '');
echo "Scanning Redis for sessions...\n";
$cursor = '0';
$allKeys = [];
do {
$result = $redis->scan($cursor, ['match' => '*laravel_session:*', 'count' => 1000]);
$cursor = $result[0];
$keys = $result[1] ?? [];
$allKeys = array_merge($allKeys, $keys);
} while ($cursor !== '0');
echo "Found " . count($allKeys) . " total session keys\n\n";
} else {
echo "Unsupported session driver: {$sessionDriver}\n";
exit(1);
}
function unserializeSession($data) {
$result = [];
if (empty($data)) return $result;
// Try to find user ID in serialized data
if (preg_match('/login_web_[a-f0-9]+";i:(\d+)/', $data, $matches)) {
$result['user_id'] = (int) $matches[1];
}
// Try to find last activity
if (preg_match('/last_activity";i:(\d+)/', $data, $matches)) {
$result['last_activity'] = $matches[1];
}
// Try to find activeProfileType
if (preg_match('/activeProfileType";s:\d+:"([^"]+)"/', $data, $matches)) {
$result['profile_type'] = $matches[1];
}
return $result;
}
$sessions = [];
if ($sessionDriver === 'database') {
// Process database sessions
foreach ($dbSessions as $session) {
$parsed = unserializeSession($session->payload);
// Determine guard from column (if exists) or fallback to session data
$guard = $session->guard ?? $parsed['profile_type'] ?? 'web';
// Map guard to friendly name
$guardMap = [
'web' => 'User',
'bank' => 'Bank',
'organization' => 'Organization',
'admin' => 'Admin',
];
$profileType = $guardMap[$guard] ?? ucfirst($guard);
$sessions[] = [
'id' => $session->id,
'user_id' => $session->user_id,
'guard' => $guard,
'last_activity' => $session->last_activity,
'profile_type' => $profileType,
'ip_address' => $session->ip_address,
'user_agent' => substr($session->user_agent, 0, 50),
];
}
} elseif ($sessionDriver === 'redis') {
// Process Redis sessions
foreach ($allKeys as $key) {
$sessionData = $redis->get($key);
if ($sessionData) {
$parsed = unserializeSession($sessionData);
if (isset($parsed['user_id'])) {
// Filter by user if specified
if ($userId === null || $parsed['user_id'] === $userId) {
$sessions[] = [
'key' => $key,
'user_id' => $parsed['user_id'],
'last_activity' => $parsed['last_activity'] ?? null,
'profile_type' => $parsed['profile_type'] ?? 'User',
'ttl' => $redis->ttl($key),
];
}
}
}
}
}
if ($action === 'list') {
if (empty($sessions)) {
echo "No active sessions found" . ($userId ? " for user {$userId}" : "") . "\n";
} else {
echo "Active sessions" . ($userId ? " for user {$userId}" : "") . ":\n";
echo str_repeat("=", 130) . "\n";
if ($sessionDriver === 'database') {
printf("%-10s %-12s %-20s %-20s %-20s %-30s\n", "User ID", "Guard", "Last Activity", "Profile Type", "IP Address", "User Agent");
} else {
printf("%-10s %-30s %-20s %-15s %s\n", "User ID", "Last Activity", "Profile Type", "TTL (sec)", "Session Key");
}
echo str_repeat("-", 130) . "\n";
foreach ($sessions as $session) {
$lastActivity = $session['last_activity'] ? date('Y-m-d H:i:s', $session['last_activity']) : 'Unknown';
$profileType = basename(str_replace('\\', '/', $session['profile_type']));
if ($sessionDriver === 'database') {
printf("%-10s %-12s %-20s %-20s %-20s %-30s\n",
$session['user_id'] ?? 'N/A',
$session['guard'] ?? 'web',
$lastActivity,
$profileType,
$session['ip_address'],
$session['user_agent']
);
} else {
$sessionKey = substr($session['key'], -20);
printf("%-10d %-30s %-20s %-15s ...%s\n",
$session['user_id'],
$lastActivity,
$profileType,
$session['ttl'],
$sessionKey
);
}
}
echo str_repeat("=", 120) . "\n";
echo "Total: " . count($sessions) . " session(s)\n";
}
} elseif ($action === 'expire') {
if (!$userId) {
echo "Error: User ID is required for expire action\n";
exit(1);
}
$user = \App\Models\User::find($userId);
if (!$user) {
echo "User {$userId} not found\n";
exit(1);
}
echo "User: {$user->name} (ID: {$user->id})\n";
echo "Sessions to expire: " . count($sessions) . "\n\n";
if (empty($sessions)) {
echo "No sessions found for user {$userId}\n";
} else {
$deleted = 0;
if ($sessionDriver === 'database') {
$table = config('session.table', 'sessions');
foreach ($sessions as $session) {
DB::table($table)->where('id', $session['id'])->delete();
$deleted++;
echo "✓ Deleted session: " . substr($session['id'], 0, 40) . "\n";
}
} elseif ($sessionDriver === 'redis') {
foreach ($sessions as $session) {
$redis->del($session['key']);
$deleted++;
echo "✓ Deleted session: " . substr($session['key'], -40) . "\n";
}
}
echo "\n";
echo str_repeat("=", 50) . "\n";
echo "Total sessions deleted: {$deleted}\n";
echo "{$user->name} has been logged out from all devices\n";
echo str_repeat("=", 50) . "\n";
}
} else {
echo "Unknown action: {$action}\n";
echo "Valid actions: list, expire\n";
exit(1);
}
scripts/setup-backups.sh
Executable
+338
View File
@@ -0,0 +1,338 @@
#!/bin/bash
# Laravel Timebank Backup Setup Script
# Initializes the backup system and performs initial configuration
# Usage: ./setup-backups.sh
set -e
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Function to print colored output
print_status() {
local level="$1"
local message="$2"
case "$level" in
"INFO")
echo -e "${BLUE}[INFO]${NC} $message"
;;
"SUCCESS")
echo -e "${GREEN}[SUCCESS]${NC} $message"
;;
"WARNING")
echo -e "${YELLOW}[WARNING]${NC} $message"
;;
"ERROR")
echo -e "${RED}[ERROR]${NC} $message"
;;
esac
}
# Function to check prerequisites
check_prerequisites() {
print_status "INFO" "Checking prerequisites..."
local missing_tools=()
# Check required commands
local required_commands=("mysqldump" "mysql" "rsync" "tar" "gzip" "find" "awk")
for cmd in "${required_commands[@]}"; do
if ! command -v "$cmd" >/dev/null 2>&1; then
missing_tools+=("$cmd")
fi
done
if [ ${#missing_tools[@]} -gt 0 ]; then
print_status "ERROR" "Missing required tools: ${missing_tools[*]}"
print_status "INFO" "Please install missing tools and try again"
return 1
fi
print_status "SUCCESS" "All required tools are available"
return 0
}
# Function to verify environment configuration
check_environment() {
print_status "INFO" "Checking environment configuration..."
if [ ! -f "$PROJECT_ROOT/.env" ]; then
print_status "ERROR" ".env file not found in $PROJECT_ROOT"
return 1
fi
# Load environment loader
source "$SCRIPT_DIR/load-env.sh"
# Load environment variables
if ! load_env "$PROJECT_ROOT/.env"; then
return 1
fi
# Check required database variables
local missing_vars=()
if [ -z "$DB_DATABASE" ]; then missing_vars+=("DB_DATABASE"); fi
if [ -z "$DB_USERNAME" ]; then missing_vars+=("DB_USERNAME"); fi
if [ -z "$DB_HOST" ]; then missing_vars+=("DB_HOST"); fi
if [ ${#missing_vars[@]} -gt 0 ]; then
print_status "ERROR" "Missing required environment variables: ${missing_vars[*]}"
return 1
fi
print_status "SUCCESS" "Environment configuration is valid"
return 0
}
# Function to test database connection
test_database_connection() {
print_status "INFO" "Testing database connection..."
# Create MySQL configuration file for secure password handling
local mysql_cnf_file="/tmp/mysql_test_$$.cnf"
cat > "$mysql_cnf_file" <<EOF
[mysql]
host=${DB_HOST:-127.0.0.1}
port=${DB_PORT:-3306}
user=$DB_USERNAME
password=$DB_PASSWORD
EOF
# Set secure permissions on the config file
chmod 600 "$mysql_cnf_file"
if mysql --defaults-extra-file="$mysql_cnf_file" -e "SELECT 1;" >/dev/null 2>&1; then
print_status "SUCCESS" "Database connection successful"
rm -f "$mysql_cnf_file"
return 0
else
print_status "ERROR" "Database connection failed"
rm -f "$mysql_cnf_file"
return 1
fi
}
# Function to set up backup directories
setup_directories() {
print_status "INFO" "Setting up backup directories..."
local backup_dirs=(
"$PROJECT_ROOT/backups"
"$PROJECT_ROOT/backups/database/daily"
"$PROJECT_ROOT/backups/database/weekly"
"$PROJECT_ROOT/backups/database/monthly"
"$PROJECT_ROOT/backups/database/pre-restore"
"$PROJECT_ROOT/backups/storage/daily"
"$PROJECT_ROOT/backups/storage/weekly"
"$PROJECT_ROOT/backups/storage/monthly"
"$PROJECT_ROOT/backups/storage/snapshots"
"$PROJECT_ROOT/backups/storage/pre-restore"
"$PROJECT_ROOT/backups/logs"
)
for dir in "${backup_dirs[@]}"; do
if [ ! -d "$dir" ]; then
mkdir -p "$dir"
print_status "INFO" "Created directory: $dir"
fi
done
# Set appropriate permissions
chmod 755 "$PROJECT_ROOT/backups"
chmod -R 750 "$PROJECT_ROOT/backups"/{database,storage}
chmod 755 "$PROJECT_ROOT/backups/logs"
print_status "SUCCESS" "Backup directories created successfully"
}
# Function to make scripts executable
setup_script_permissions() {
print_status "INFO" "Setting up script permissions..."
local scripts=(
"$SCRIPT_DIR/backup-database.sh"
"$SCRIPT_DIR/backup-storage.sh"
"$SCRIPT_DIR/backup-all.sh"
"$SCRIPT_DIR/restore-database.sh"
"$SCRIPT_DIR/restore-storage.sh"
"$SCRIPT_DIR/cleanup-backups.sh"
)
for script in "${scripts[@]}"; do
if [ -f "$script" ]; then
chmod +x "$script"
print_status "INFO" "Made executable: $(basename "$script")"
else
print_status "WARNING" "Script not found: $(basename "$script")"
fi
done
print_status "SUCCESS" "Script permissions configured"
}
# Function to run initial test backup
run_test_backup() {
print_status "INFO" "Running initial test backup..."
# Test database backup
if [ -x "$SCRIPT_DIR/backup-database.sh" ]; then
print_status "INFO" "Testing database backup..."
if "$SCRIPT_DIR/backup-database.sh" daily; then
print_status "SUCCESS" "Database backup test successful"
else
print_status "ERROR" "Database backup test failed"
return 1
fi
fi
# Test storage backup
if [ -x "$SCRIPT_DIR/backup-storage.sh" ]; then
print_status "INFO" "Testing storage backup..."
if "$SCRIPT_DIR/backup-storage.sh" daily; then
print_status "SUCCESS" "Storage backup test successful"
else
print_status "ERROR" "Storage backup test failed"
return 1
fi
fi
return 0
}
# Function to show next steps
show_next_steps() {
print_status "SUCCESS" "Backup system setup completed successfully!"
echo ""
print_status "INFO" "Next steps:"
echo " 1. Review the backup configuration in BACKUP_GUIDE.md"
echo " 2. Set up automated backups using cron:"
echo " sed 's#^PROJECT_DIR=.*#PROJECT_DIR=$PROJECT_ROOT#' scripts/cron-backup.conf | sudo tee /etc/cron.d/timebank-backup >/dev/null"
echo " sudo nano /etc/cron.d/timebank-backup # Check the path, set MAILTO"
echo " 3. Test the complete backup system:"
echo " ./scripts/backup-all.sh daily --verify"
echo " 4. Test restore procedures:"
echo " ./scripts/restore-database.sh --list-backups"
echo " 5. Set up monitoring and notifications"
echo ""
print_status "INFO" "Available commands:"
echo " ./scripts/backup-all.sh daily # Daily backup"
echo " ./scripts/backup-all.sh weekly # Weekly backup"
echo " ./scripts/backup-all.sh monthly # Monthly backup"
echo " ./scripts/restore-database.sh --latest # Restore database"
echo " ./scripts/restore-storage.sh --latest # Restore storage"
echo " ./scripts/cleanup-backups.sh --dry-run # Check cleanup"
echo ""
}
# Function to display backup status
show_backup_status() {
print_status "INFO" "Current backup status:"
local backup_dir="$PROJECT_ROOT/backups"
if [ -d "$backup_dir" ]; then
# Count backups
local db_backups=$(find "$backup_dir/database" -name "*.sql.gz" 2>/dev/null | wc -l)
local storage_backups=$(find "$backup_dir/storage" -name "*.tar.gz" 2>/dev/null | wc -l)
echo " Database backups: $db_backups"
echo " Storage backups: $storage_backups"
if [ -d "$backup_dir" ]; then
local total_size=$(du -sh "$backup_dir" 2>/dev/null | cut -f1)
echo " Total backup size: $total_size"
fi
# Show recent backups
local recent_db=$(find "$backup_dir/database" -name "*.sql.gz" -mtime -1 2>/dev/null | head -n 1)
local recent_storage=$(find "$backup_dir/storage" -name "*.tar.gz" -mtime -1 2>/dev/null | head -n 1)
if [ -n "$recent_db" ]; then
echo " Latest database backup: $(basename "$recent_db") ($(date -r "$recent_db" '+%Y-%m-%d %H:%M:%S'))"
fi
if [ -n "$recent_storage" ]; then
echo " Latest storage backup: $(basename "$recent_storage") ($(date -r "$recent_storage" '+%Y-%m-%d %H:%M:%S'))"
fi
else
echo " No backups found"
fi
echo ""
}
# Main execution
main() {
echo "============================================"
echo " Laravel Timebank Backup System Setup "
echo "============================================"
echo ""
# Check if already set up
if [ -d "$PROJECT_ROOT/backups" ] && [ -f "$PROJECT_ROOT/backups/backup.log" ]; then
print_status "INFO" "Backup system appears to already be configured"
show_backup_status
read -p "Do you want to re-run the setup? [y/N]: " -n 1 -r
echo ""
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
print_status "INFO" "Setup cancelled"
exit 0
fi
echo ""
fi
# Run setup steps
if ! check_prerequisites; then
exit 1
fi
if ! check_environment; then
print_status "INFO" "Please configure your .env file with proper database credentials"
exit 1
fi
if ! test_database_connection; then
print_status "INFO" "Please check your database configuration and connection"
exit 1
fi
setup_directories
setup_script_permissions
# Ask if user wants to run test backup
echo ""
read -p "Do you want to run an initial test backup? [Y/n]: " -n 1 -r
echo ""
if [[ ! $REPLY =~ ^[Nn]$ ]]; then
if run_test_backup; then
print_status "SUCCESS" "Test backup completed successfully"
else
print_status "WARNING" "Test backup had issues, but setup is complete"
fi
fi
echo ""
show_next_steps
}
# Check if running from correct directory
if [ ! -f "$PROJECT_ROOT/composer.json" ] || [ ! -f "$PROJECT_ROOT/.env.example" ]; then
print_status "ERROR" "This script must be run from the Laravel project root directory"
exit 1
fi
# Run main function
main
scripts/sync-translation-keys.php
Executable
+51
View File
@@ -0,0 +1,51 @@
<?php
/**
* Sync all language files to have the same keys as en.json
*/
echo "=== SYNCING TRANSLATION KEYS ===\n\n";
$enFile = 'resources/lang/en.json';
$en = json_decode(file_get_contents($enFile), true);
$enCount = count($en);
echo "Source (en.json): {$enCount} keys\n\n";
$locales = ['nl', 'de', 'es', 'fr'];
foreach ($locales as $locale) {
$file = "resources/lang/{$locale}.json";
$data = json_decode(file_get_contents($file), true);
$before = count($data);
// Add missing keys from en.json
$added = 0;
foreach ($en as $key => $value) {
if (!isset($data[$key])) {
$data[$key] = $key; // Use English as placeholder
$added++;
}
}
// Remove keys not in en.json
$removed = 0;
foreach (array_keys($data) as $key) {
if (!isset($en[$key])) {
unset($data[$key]);
$removed++;
}
}
$after = count($data);
// Sort alphabetically
ksort($data);
// Save
file_put_contents($file, json_encode($data, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES) . PHP_EOL);
echo "{$locale}.json: {$before} → {$after} keys (+{$added} -{$removed})\n";
}
echo "\n✓ All language files synced!\n";
scripts/test-all-emails.php
Executable
+237
View File
@@ -0,0 +1,237 @@
#!/usr/bin/env php
<?php
/**
* Test script to dispatch all transactional emails in all locales
*
* This script creates test scenarios for each email type and dispatches
* them to a test email address in all available locales.
*/
require __DIR__.'/vendor/autoload.php';
$app = require_once __DIR__.'/bootstrap/app.php';
$kernel = $app->make(Illuminate\Contracts\Console\Kernel::class);
$kernel->bootstrap();
use App\Models\User;
use App\Models\Organization;
use App\Models\Transaction;
use App\Models\Post;
use App\Models\Reaction;
use App\Models\Tag;
use App\Mail\TransferReceived;
use App\Mail\ReactionCreatedMail;
use App\Mail\ReservationCreatedMail;
use App\Mail\ReservationUpdateMail;
use App\Mail\ReservationCancelledMail;
use App\Mail\ProfileEditedByAdminMail;
use App\Mail\ProfileLinkChangedMail;
use App\Mail\TagAddedMail;
use App\Mail\UserDeletedMail;
use App\Mail\VerifyProfileEmailMailable;
use App\Mail\NewMessageMail;
use Illuminate\Support\Facades\Mail;
echo "=" . str_repeat("=", 78) . "\n";
echo "Testing All Transactional Emails\n";
echo "=" . str_repeat("=", 78) . "\n\n";
// Configuration
$testEmail = 'test@example.com'; // Change this to your test email
$locales = ['en', 'nl', 'de', 'es', 'fr'];
// Find or create test user
$testUser = User::where('email', 'test-user@timebank.local')->first();
if (!$testUser) {
echo "ERROR: Test user not found. Please create a user with email 'test-user@timebank.local'\n";
echo "Attempting to use first available user instead...\n";
$testUser = User::whereNull('deleted_at')->first();
if (!$testUser) {
echo "ERROR: No users found in database!\n";
exit(1);
}
}
echo "Using test user: {$testUser->name} ({$testUser->email})\n";
echo "Emails will be queued for: {$testEmail}\n\n";
$emailsSent = 0;
$errors = [];
// Helper function to send email
function sendTestEmail($mailClass, $emailName, $locale, $testEmail, &$emailsSent, &$errors) {
try {
Mail::to($testEmail)->queue($mailClass);
echo " ✓ {$locale}: Queued successfully\n";
$emailsSent++;
} catch (\Exception $e) {
$error = " ✗ {$locale}: {$e->getMessage()}";
echo $error . "\n";
$errors[] = $error;
}
}
// 1. Transfer/Payment Received Email
echo "\n1. Testing Transfer Received Email\n";
echo str_repeat("-", 80) . "\n";
$transaction = Transaction::with(['accountFrom.accountable', 'accountTo.accountable'])
->whereHas('accountFrom')
->whereHas('accountTo')
->first();
if ($transaction) {
foreach ($locales as $locale) {
$mail = new TransferReceived($transaction, $locale);
sendTestEmail($mail, 'TransferReceived', $locale, $testEmail, $emailsSent, $errors);
}
} else {
echo " ⚠ Skipped: No transactions found\n";
}
// 2. Reaction Emails (Star Received)
echo "\n2. Testing Star Received Email\n";
echo str_repeat("-", 80) . "\n";
$reaction = Reaction::where('reaction_type', 'star')
->with(['post.profile', 'profile'])
->first();
if ($reaction) {
foreach ($locales as $locale) {
$mail = new ReactionCreatedMail($reaction->post, $reaction, $locale);
sendTestEmail($mail, 'StarReceived', $locale, $testEmail, $emailsSent, $errors);
}
} else {
echo " ⚠ Skipped: No star reactions found\n";
}
// 3. Reservation Created Email
echo "\n3. Testing Reservation Created Email\n";
echo str_repeat("-", 80) . "\n";
$reservation = Reaction::where('reaction_type', 'reservation')
->with(['post.profile', 'profile'])
->first();
if ($reservation) {
foreach ($locales as $locale) {
$mail = new ReservationCreatedMail($reservation->post, $reservation, $locale);
sendTestEmail($mail, 'ReservationCreated', $locale, $testEmail, $emailsSent, $errors);
}
} else {
echo " ⚠ Skipped: No reservations found\n";
}
// 4. Reservation Updated Email
echo "\n4. Testing Reservation Updated Email\n";
echo str_repeat("-", 80) . "\n";
if ($reservation) {
foreach ($locales as $locale) {
$mail = new ReservationUpdateMail($reservation->post, $reservation, $locale);
sendTestEmail($mail, 'ReservationUpdate', $locale, $testEmail, $emailsSent, $errors);
}
} else {
echo " ⚠ Skipped: No reservations found\n";
}
// 5. Reservation Cancelled Email
echo "\n5. Testing Reservation Cancelled Email\n";
echo str_repeat("-", 80) . "\n";
if ($reservation) {
foreach ($locales as $locale) {
$mail = new ReservationCancelledMail($reservation->post, $reservation, $locale);
sendTestEmail($mail, 'ReservationCancelled', $locale, $testEmail, $emailsSent, $errors);
}
} else {
echo " ⚠ Skipped: No reservations found\n";
}
// 6. Profile Edited by Admin Email
echo "\n6. Testing Profile Edited by Admin Email\n";
echo str_repeat("-", 80) . "\n";
foreach ($locales as $locale) {
$changes = [
'name' => ['old' => 'Old Name', 'new' => 'New Name'],
'email' => ['old' => 'old@example.com', 'new' => 'new@example.com'],
];
$mail = new ProfileEditedByAdminMail($testUser, $changes, $locale);
sendTestEmail($mail, 'ProfileEditedByAdmin', $locale, $testEmail, $emailsSent, $errors);
}
// 7. Profile Link Changed Email
echo "\n7. Testing Profile Link Changed Email\n";
echo str_repeat("-", 80) . "\n";
foreach ($locales as $locale) {
$mail = new ProfileLinkChangedMail($testUser, 'website', 'https://old-site.com', 'https://new-site.com', $locale);
sendTestEmail($mail, 'ProfileLinkChanged', $locale, $testEmail, $emailsSent, $errors);
}
// 8. Tag Added Email
echo "\n8. Testing Tag Added Email\n";
echo str_repeat("-", 80) . "\n";
$tag = Tag::first();
if ($tag) {
foreach ($locales as $locale) {
$mail = new TagAddedMail($testUser, $tag, $locale);
sendTestEmail($mail, 'TagAdded', $locale, $testEmail, $emailsSent, $errors);
}
} else {
echo " ⚠ Skipped: No tags found\n";
}
// 9. User Deleted Email
echo "\n9. Testing User Deleted Email\n";
echo str_repeat("-", 80) . "\n";
foreach ($locales as $locale) {
$emailData = [
'time' => now()->translatedFormat('j F Y, H:i'),
'deletedUser' => (object)[
'name' => $testUser->name,
'full_name' => $testUser->full_name ?? $testUser->name,
'lang_preference' => $locale,
],
'mail' => $testEmail,
'balanceHandlingOption' => 'delete',
'totalBalance' => 500,
'donationAccountId' => null,
'donationAccountName' => null,
'donationOrganizationName' => null,
];
$mail = new UserDeletedMail($emailData);
sendTestEmail($mail, 'UserDeleted', $locale, $testEmail, $emailsSent, $errors);
}
// 10. Email Verification
echo "\n10. Testing Email Verification Email\n";
echo str_repeat("-", 80) . "\n";
foreach ($locales as $locale) {
$verificationUrl = url('/verify-email/' . base64_encode($testEmail));
$mail = new VerifyProfileEmailMailable($testEmail, $verificationUrl, $locale);
sendTestEmail($mail, 'VerifyEmail', $locale, $testEmail, $emailsSent, $errors);
}
// Summary
echo "\n" . str_repeat("=", 80) . "\n";
echo "Testing Complete!\n";
echo str_repeat("=", 80) . "\n";
echo "Total emails queued: {$emailsSent}\n";
if (count($errors) > 0) {
echo "\nErrors encountered: " . count($errors) . "\n";
foreach ($errors as $error) {
echo $error . "\n";
}
}
echo "\nProcessing queue...\n";
echo "Run: php artisan queue:work --stop-when-empty\n";
echo "\n";
scripts/test-all-warnings.sh
Executable
+51
View File
@@ -0,0 +1,51 @@
#!/bin/bash
USER_ID=${1:-102}
echo "=== Testing All 3 Warning Emails for User ID: $USER_ID ==="
echo ""
# Warning 1: Set inactive_at to 2 minutes ago
echo "=== Test 1: Warning 1 (inactive for 2 minutes) ==="
php artisan tinker --execute="
\$user = App\Models\User::find($USER_ID);
\$user->inactive_at = now()->subMinutes(2);
\$user->save();
echo 'Set inactive_at to 2 minutes ago' . PHP_EOL;
exit;
"
php artisan profiles:process-inactive
sleep 2
echo ""
# Warning 2: Set inactive_at to 3 minutes ago
echo "=== Test 2: Warning 2 (inactive for 3 minutes) ==="
php artisan tinker --execute="
\$user = App\Models\User::find($USER_ID);
\$user->inactive_at = now()->subMinutes(3);
\$user->save();
echo 'Set inactive_at to 3 minutes ago' . PHP_EOL;
exit;
"
php artisan profiles:process-inactive
sleep 2
echo ""
# Warning Final: Set inactive_at to 4 minutes ago
echo "=== Test 3: Warning Final (inactive for 4 minutes) ==="
php artisan tinker --execute="
\$user = App\Models\User::find($USER_ID);
\$user->inactive_at = now()->subMinutes(4);
\$user->save();
echo 'Set inactive_at to 4 minutes ago' . PHP_EOL;
exit;
"
php artisan profiles:process-inactive
sleep 2
echo ""
echo "=== All 3 warning emails sent! ==="
echo "Processing queue..."
php artisan queue:work --stop-when-empty --timeout=30
echo ""
echo "Check Mailpit for all 3 warning emails sent to user $USER_ID"
scripts/test-balance-visibility.php
Executable
+46
View File
@@ -0,0 +1,46 @@
<?php
require __DIR__.'/vendor/autoload.php';
$app = require_once __DIR__.'/bootstrap/app.php';
$app->make('Illuminate\Contracts\Console\Kernel')->bootstrap();
use App\Models\User;
use App\Models\Account;
// Set TIMEBANK_CONFIG to timebank-default
putenv('TIMEBANK_CONFIG=timebank-default');
echo "Testing balance visibility with TIMEBANK_CONFIG=timebank-default\n";
echo "======================================================================\n\n";
// Test 1: Check config values
echo "1. Configuration values:\n";
echo " - balance_public (user): " . (timebank_config('account_info.user.balance_public') ? 'true' : 'false') . "\n";
echo " - sumBalances_public (user): " . (timebank_config('account_info.user.sumBalances_public') ? 'true' : 'false') . "\n\n";
// Test 2: Get a test user
$testUser = User::where('name', '!=', 'Super User')->first();
if (!$testUser) {
echo "No test user found!\n";
exit(1);
}
echo "2. Test user: {$testUser->name} (ID: {$testUser->id})\n\n";
// Test 3: Get account totals without authentication
echo "3. Getting account totals (not logged in):\n";
$account = new Account();
$totals = $account->getAccountsTotals(get_class($testUser), $testUser->id, 365);
echo " - sumBalances: ";
if ($totals['sumBalances'] === null) {
echo "NULL (hidden as expected)\n";
} else {
echo $totals['sumBalances'] . " (VISIBLE - this is the bug!)\n";
}
echo "\n4. Testing getCanManageAccounts():\n";
echo " - Result: " . ($account->getCanManageAccounts() ? 'true' : 'false') . "\n";
echo "\nDone!\n";
scripts/test-exact-search.php
Executable
+172
View File
@@ -0,0 +1,172 @@
<?php
require __DIR__.'/vendor/autoload.php';
$app = require_once __DIR__.'/bootstrap/app.php';
$app->make(\Illuminate\Contracts\Console\Kernel::class)->bootstrap();
echo "=== EXACT MAINSEARCHBAR SEARCH TEST ===" . PHP_EOL . PHP_EOL;
$searchTerm = 'event';
$locale = 'en';
$currentTime = now()->toISOString();
// Build exact query from MainSearchBar
use ONGR\ElasticsearchDSL\Query\Compound\BoolQuery;
use ONGR\ElasticsearchDSL\Query\FullText\MultiMatchQuery;
$postsBoolQuery = new BoolQuery();
// Class name filter
$postsBoolQuery->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\TermQuery('__class_name', 'App\Models\Post'),
BoolQuery::MUST
);
// Search fields
$postSearchFields = [
'post_translations.title_' . $locale . '^3',
'post_translations.content_' . $locale . '^1',
'post_translations.excerpt_' . $locale . '^2',
];
$postMultiMatchQuery = new MultiMatchQuery($postSearchFields, $searchTerm);
$postMultiMatchQuery->addParameter('boost', 4);
$postsBoolQuery->add($postMultiMatchQuery, BoolQuery::MUST);
// Publication filters (CORRECTED VERSION)
// From date: must exist AND be in the past (null means NOT published)
$postsBoolQuery->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\ExistsQuery(
"post_translations.from_{$locale}"
),
BoolQuery::MUST
);
$postsBoolQuery->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\RangeQuery(
"post_translations.from_{$locale}",
['lte' => $currentTime]
),
BoolQuery::MUST
);
// Till date: (NOT exists) OR (exists AND in future) = never expires OR not yet expired
$tillFilter = new BoolQuery();
// Option 1: field doesn't exist (null)
$tillNotExists = new BoolQuery();
$tillNotExists->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\ExistsQuery(
"post_translations.till_{$locale}"
),
BoolQuery::MUST_NOT
);
$tillFilter->add($tillNotExists, BoolQuery::SHOULD);
// Option 2: field exists and is in the future
$tillInFuture = new BoolQuery();
$tillInFuture->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\ExistsQuery(
"post_translations.till_{$locale}"
),
BoolQuery::MUST
);
$tillInFuture->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\RangeQuery(
"post_translations.till_{$locale}",
['gte' => $currentTime]
),
BoolQuery::MUST
);
$tillFilter->add($tillInFuture, BoolQuery::SHOULD);
$postsBoolQuery->add($tillFilter, BoolQuery::MUST);
// Deleted date: (NOT exists) OR (exists AND in future) = not deleted OR scheduled for future
$deletionFilter = new BoolQuery();
// Option 1: field doesn't exist (null)
$deletionNotExists = new BoolQuery();
$deletionNotExists->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\ExistsQuery(
"post_translations.deleted_at_{$locale}"
),
BoolQuery::MUST_NOT
);
$deletionFilter->add($deletionNotExists, BoolQuery::SHOULD);
// Option 2: field exists and is in the future
$deletionInFuture = new BoolQuery();
$deletionInFuture->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\ExistsQuery(
"post_translations.deleted_at_{$locale}"
),
BoolQuery::MUST
);
$deletionInFuture->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\RangeQuery(
"post_translations.deleted_at_{$locale}",
['gt' => $currentTime]
),
BoolQuery::MUST
);
$deletionFilter->add($deletionInFuture, BoolQuery::SHOULD);
$postsBoolQuery->add($deletionFilter, BoolQuery::MUST);
// Category filter
$categoryIds = timebank_config('main_search_bar.category_ids_posts');
if (!empty($categoryIds)) {
$categoryBoolQuery = new BoolQuery();
foreach ($categoryIds as $categoryId) {
$categoryBoolQuery->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\TermQuery('category_id', $categoryId),
BoolQuery::SHOULD
);
}
$postsBoolQuery->add($categoryBoolQuery, BoolQuery::MUST);
}
// Execute search
$client = app(Elastic\Elasticsearch\ClientBuilder::class)->build();
$searchBody = ['query' => $postsBoolQuery->toArray()];
echo "Searching for: '{$searchTerm}'" . PHP_EOL;
echo "Locale: {$locale}" . PHP_EOL;
echo "Current time: {$currentTime}" . PHP_EOL;
echo "Category filter: " . json_encode($categoryIds) . PHP_EOL;
echo PHP_EOL;
echo "Query structure:" . PHP_EOL;
echo json_encode($searchBody, JSON_PRETTY_PRINT) . PHP_EOL;
echo PHP_EOL;
try {
$response = $client->search([
'index' => 'posts_index',
'body' => $searchBody
])->asArray();
$totalHits = $response['hits']['total']['value'] ?? 0;
echo "Total hits: {$totalHits}" . PHP_EOL . PHP_EOL;
if (!empty($response['hits']['hits'])) {
echo "Results:" . PHP_EOL;
echo str_repeat('-', 100) . PHP_EOL;
foreach ($response['hits']['hits'] as $hit) {
$postId = $hit['_source']['id'] ?? 'N/A';
$categoryId = $hit['_source']['category_id'] ?? 'N/A';
$title = $hit['_source']['post_translations']['title_en'] ?? 'N/A';
$from = $hit['_source']['post_translations']['from_en'] ?? 'NULL';
$till = $hit['_source']['post_translations']['till_en'] ?? 'NULL';
$score = $hit['_score'] ?? 'N/A';
echo "ID: {$postId} | Cat: {$categoryId} | Score: {$score}" . PHP_EOL;
echo " Title: " . substr($title, 0, 60) . PHP_EOL;
echo " from_en: {$from} | till_en: {$till}" . PHP_EOL;
echo PHP_EOL;
}
}
} catch (\Exception $e) {
echo "Error: " . $e->getMessage() . PHP_EOL;
echo "Stack trace:" . PHP_EOL;
echo $e->getTraceAsString() . PHP_EOL;
}
echo "=== TEST COMPLETE ===" . PHP_EOL;
scripts/test-full-search-flow.php
Executable
+236
View File
@@ -0,0 +1,236 @@
<?php
require __DIR__.'/vendor/autoload.php';
$app = require_once __DIR__.'/bootstrap/app.php';
$app->make(\Illuminate\Contracts\Console\Kernel::class)->bootstrap();
// Simulate the EXACT flow that MainSearchBar uses
echo "=== TESTING FULL MAINSEARCHBAR FLOW ===" . PHP_EOL . PHP_EOL;
$searchTerm = 'event';
$locale = 'en';
// Set app locale
app()->setLocale($locale);
// Clean search term (same as MainSearchBar)
$search = preg_replace('/[^a-zA-Z0-9\s]/', '', $searchTerm);
$search = rtrim($search);
$cleanSearch = trim(str_replace('*', '', $search));
echo "Original term: '{$searchTerm}'" . PHP_EOL;
echo "Cleaned term: '{$cleanSearch}'" . PHP_EOL;
echo "Locale: {$locale}" . PHP_EOL;
echo PHP_EOL;
// Create the exact query from MainSearchBar
$currentTime = now()->toISOString();
use ONGR\ElasticsearchDSL\Query\Compound\BoolQuery;
use ONGR\ElasticsearchDSL\Query\FullText\MultiMatchQuery;
use Matchish\ScoutElasticSearch\MixedSearch;
use ONGR\ElasticsearchDSL\Search;
use Elastic\Elasticsearch\Client;
$mainBoolQuery = new BoolQuery();
// Add posts search query
$postsBoolQuery = new BoolQuery();
$postsBoolQuery->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\TermQuery('__class_name', 'App\Models\Post'),
BoolQuery::MUST
);
$postSearchFields = [
'post_translations.title_' . $locale . '^3',
'post_translations.content_' . $locale . '^1',
'post_translations.excerpt_' . $locale . '^2',
];
$postMultiMatchQuery = new MultiMatchQuery($postSearchFields, $cleanSearch);
$postMultiMatchQuery->addParameter('boost', 4);
$postsBoolQuery->add($postMultiMatchQuery, BoolQuery::MUST);
// Publication filters
$postsBoolQuery->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\ExistsQuery(
"post_translations.from_{$locale}"
),
BoolQuery::MUST
);
$postsBoolQuery->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\RangeQuery(
"post_translations.from_{$locale}",
['lte' => $currentTime]
),
BoolQuery::MUST
);
$tillFilter = new BoolQuery();
$tillNotExists = new BoolQuery();
$tillNotExists->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\ExistsQuery(
"post_translations.till_{$locale}"
),
BoolQuery::MUST_NOT
);
$tillFilter->add($tillNotExists, BoolQuery::SHOULD);
$tillInFuture = new BoolQuery();
$tillInFuture->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\ExistsQuery(
"post_translations.till_{$locale}"
),
BoolQuery::MUST
);
$tillInFuture->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\RangeQuery(
"post_translations.till_{$locale}",
['gte' => $currentTime]
),
BoolQuery::MUST
);
$tillFilter->add($tillInFuture, BoolQuery::SHOULD);
$postsBoolQuery->add($tillFilter, BoolQuery::MUST);
$deletionFilter = new BoolQuery();
$deletionNotExists = new BoolQuery();
$deletionNotExists->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\ExistsQuery(
"post_translations.deleted_at_{$locale}"
),
BoolQuery::MUST_NOT
);
$deletionFilter->add($deletionNotExists, BoolQuery::SHOULD);
$deletionInFuture = new BoolQuery();
$deletionInFuture->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\ExistsQuery(
"post_translations.deleted_at_{$locale}"
),
BoolQuery::MUST
);
$deletionInFuture->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\RangeQuery(
"post_translations.deleted_at_{$locale}",
['gt' => $currentTime]
),
BoolQuery::MUST
);
$deletionFilter->add($deletionInFuture, BoolQuery::SHOULD);
$postsBoolQuery->add($deletionFilter, BoolQuery::MUST);
// Category filter
$categoryIds = timebank_config('main_search_bar.category_ids_posts');
if (!empty($categoryIds)) {
$categoryBoolQuery = new BoolQuery();
foreach ($categoryIds as $categoryId) {
$categoryBoolQuery->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\TermQuery('category_id', $categoryId),
BoolQuery::SHOULD
);
}
$postsBoolQuery->add($categoryBoolQuery, BoolQuery::MUST);
}
$mainBoolQuery->add($postsBoolQuery, BoolQuery::SHOULD);
// Execute search via MixedSearch
try {
$rawResponse = MixedSearch::search($cleanSearch, function (Client $client, Search $body) use ($mainBoolQuery) {
$body->addQuery($mainBoolQuery);
$body->setSize(50);
return $client->search([
'index' => implode(',', timebank_config('main_search_bar.model_indices', [])),
'body' => $body->toArray(),
])->asArray();
})->raw();
echo "Total hits: " . ($rawResponse['hits']['total']['value'] ?? 0) . PHP_EOL;
echo "Indices searched: " . json_encode(timebank_config('main_search_bar.model_indices')) . PHP_EOL;
echo PHP_EOL;
if (!empty($rawResponse['hits']['hits'])) {
echo "Raw search results:" . PHP_EOL;
echo str_repeat('-', 100) . PHP_EOL;
foreach ($rawResponse['hits']['hits'] as $hit) {
$modelClass = $hit['_source']['__class_name'] ?? 'N/A';
$id = $hit['_source']['id'] ?? 'N/A';
$score = $hit['_score'] ?? 'N/A';
echo "Model: {$modelClass} | ID: {$id} | Score: {$score}" . PHP_EOL;
if ($modelClass === 'App\Models\Post') {
$title = $hit['_source']['post_translations']['title_en'] ?? 'N/A';
$categoryId = $hit['_source']['category_id'] ?? 'N/A';
echo " Title: " . substr($title, 0, 60) . PHP_EOL;
echo " Category: {$categoryId}" . PHP_EOL;
} elseif (in_array($modelClass, ['App\Models\User', 'App\Models\Organization', 'App\Models\Bank'])) {
$name = $hit['_source']['name'] ?? 'N/A';
echo " Name: {$name}" . PHP_EOL;
}
echo PHP_EOL;
}
// Now process through processPostCard to see if they get filtered
echo PHP_EOL . "Processing results through processPostCard logic:" . PHP_EOL;
echo str_repeat('-', 100) . PHP_EOL;
foreach ($rawResponse['hits']['hits'] as $hit) {
$modelClass = $hit['_source']['__class_name'] ?? null;
$modelId = $hit['_source']['id'] ?? null;
if ($modelClass === 'App\Models\Post' && $modelId) {
$post = App\Models\Post::with(['translations', 'category'])->find($modelId);
if ($post) {
$translation = $post->translations()->where('locale', $locale)->first();
echo "Post ID {$modelId}:" . PHP_EOL;
if (!$translation) {
echo " ✗ FILTERED OUT: No translation for locale '{$locale}'" . PHP_EOL;
continue;
}
$currentTime = now();
$isPublished = true;
$reason = 'Visible';
if (!$translation->from) {
$isPublished = false;
$reason = "No publication date (from is null)";
} elseif ($currentTime->lt($translation->from)) {
$isPublished = false;
$reason = "Not yet published (from: {$translation->from})";
}
if ($translation->till && $currentTime->gt($translation->till)) {
$isPublished = false;
$reason = "Publication ended (till: {$translation->till})";
}
if ($translation->deleted_at && $currentTime->gte($translation->deleted_at)) {
$isPublished = false;
$reason = "Scheduled deletion";
}
if ($isPublished) {
echo " ✓ PASSED: {$reason}" . PHP_EOL;
echo " Title: {$translation->title}" . PHP_EOL;
} else {
echo " ✗ FILTERED OUT: {$reason}" . PHP_EOL;
}
}
}
}
} else {
echo "No results found!" . PHP_EOL;
}
} catch (\Exception $e) {
echo "ERROR: " . $e->getMessage() . PHP_EOL;
echo "Stack trace:" . PHP_EOL;
echo $e->getTraceAsString() . PHP_EOL;
}
echo PHP_EOL . "=== TEST COMPLETE ===" . PHP_EOL;
scripts/test-inactive-warning-emails.sh
Executable
+27
View File
@@ -0,0 +1,27 @@
#!/bin/bash
# Quick script to test all 3 inactive profile warning emails
USER_ID=${1:-102}
echo "================================================"
echo " Testing Inactive profile warning Emails"
echo "================================================"
echo ""
echo "Sending all 3 warning emails to user #$USER_ID"
echo ""
echo "📧 Warning 1 (2 weeks remaining)..."
php artisan email:send-test --type=inactive-warning-1 --receiver=user --id=$USER_ID
echo ""
echo "📧 Warning 2 (1 week remaining)..."
php artisan email:send-test --type=inactive-warning-2 --receiver=user --id=$USER_ID
echo ""
echo "📧 Final Warning (24 hours remaining)..."
php artisan email:send-test --type=inactive-warning-final --receiver=user --id=$USER_ID
echo ""
echo "================================================"
echo " ✅ All warning emails sent!"
echo "================================================"
scripts/test-post-search.php
Executable
+128
View File
@@ -0,0 +1,128 @@
<?php
require __DIR__.'/vendor/autoload.php';
$app = require_once __DIR__.'/bootstrap/app.php';
$app->make(\Illuminate\Contracts\Console\Kernel::class)->bootstrap();
echo "=== TESTING POST SEARCH FLOW ===" . PHP_EOL . PHP_EOL;
// Test search term
$searchTerm = 'event';
$locale = 'en';
// Step 1: Check how many posts match in Elasticsearch
echo "Step 1: Direct Elasticsearch search for 'event'" . PHP_EOL;
echo str_repeat('-', 80) . PHP_EOL;
$esResults = App\Models\Post::search($searchTerm)->raw();
echo "Total ES hits: " . ($esResults['hits']['total']['value'] ?? 0) . PHP_EOL;
if (!empty($esResults['hits']['hits'])) {
echo "Sample results:" . PHP_EOL;
foreach (array_slice($esResults['hits']['hits'], 0, 5) as $hit) {
$postId = $hit['_source']['id'] ?? 'N/A';
$categoryId = $hit['_source']['category_id'] ?? 'N/A';
$title = $hit['_source']['post_translations']['title_en'] ?? 'N/A';
echo " - ID: {$postId}, Cat: {$categoryId}, Title: " . substr($title, 0, 40) . PHP_EOL;
}
}
echo PHP_EOL;
// Step 2: Test with category filter
echo "Step 2: ES search with category filter [4,5,6,7,8,113]" . PHP_EOL;
echo str_repeat('-', 80) . PHP_EOL;
$categoryIds = timebank_config('main_search_bar.category_ids_posts');
echo "Allowed categories: " . json_encode($categoryIds) . PHP_EOL;
// Build filtered query
use ONGR\ElasticsearchDSL\Query\Compound\BoolQuery;
use ONGR\ElasticsearchDSL\Query\FullText\MultiMatchQuery;
$postsBoolQuery = new BoolQuery();
$postsBoolQuery->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\TermQuery('__class_name', 'App\Models\Post'),
BoolQuery::MUST
);
$postSearchFields = ['post_translations.title_' . $locale . '^2'];
$postMultiMatchQuery = new MultiMatchQuery($postSearchFields, $searchTerm);
$postsBoolQuery->add($postMultiMatchQuery, BoolQuery::MUST);
// Add category filter
if (!empty($categoryIds)) {
$categoryBoolQuery = new BoolQuery();
foreach ($categoryIds as $categoryId) {
$categoryBoolQuery->add(
new \ONGR\ElasticsearchDSL\Query\TermLevel\TermQuery('category_id', $categoryId),
BoolQuery::SHOULD
);
}
$postsBoolQuery->add($categoryBoolQuery, BoolQuery::MUST);
}
// Execute filtered search via Elasticsearch client
$client = app(Elastic\Elasticsearch\ClientBuilder::class)->build();
$searchBody = ['query' => $postsBoolQuery->toArray()];
try {
$response = $client->search([
'index' => 'posts_index',
'body' => $searchBody
])->asArray();
echo "Total filtered hits: " . ($response['hits']['total']['value'] ?? 0) . PHP_EOL;
if (!empty($response['hits']['hits'])) {
echo "Filtered results:" . PHP_EOL;
foreach ($response['hits']['hits'] as $hit) {
$postId = $hit['_source']['id'] ?? 'N/A';
$categoryId = $hit['_source']['category_id'] ?? 'N/A';
$title = $hit['_source']['post_translations']['title_en'] ?? 'N/A';
echo " - ID: {$postId}, Cat: {$categoryId}, Title: " . substr($title, 0, 40) . PHP_EOL;
}
}
} catch (\Exception $e) {
echo "Error: " . $e->getMessage() . PHP_EOL;
}
echo PHP_EOL;
// Step 3: Process through MainSearchBar logic
echo "Step 3: Simulate processPostCard() for each result" . PHP_EOL;
echo str_repeat('-', 80) . PHP_EOL;
$posts = App\Models\Post::whereIn('category_id', $categoryIds)->get();
foreach ($posts as $post) {
$translation = $post->translations()->where('locale', $locale)->first();
if (!$translation) {
echo "Post ID {$post->id}: NO TRANSLATION" . PHP_EOL;
continue;
}
$currentTime = now();
$isPublished = true;
$reason = 'Visible';
if ($translation->from && $currentTime->lt($translation->from)) {
$isPublished = false;
$reason = "Not yet published (from: {$translation->from})";
}
if ($translation->till && $currentTime->gt($translation->till)) {
$isPublished = false;
$reason = "Publication ended (till: {$translation->till})";
}
if ($translation->deleted_at && $currentTime->gte($translation->deleted_at)) {
$isPublished = false;
$reason = "Scheduled deletion";
}
$status = $isPublished ? '✓ PASS' : '✗ FAIL';
echo "Post ID {$post->id} (Cat: {$post->category_id}): {$status} - {$reason}" . PHP_EOL;
}
echo PHP_EOL . "=== TEST COMPLETE ===" . PHP_EOL;
scripts/test-transaction-immutability.sh
Executable
+314
View File
@@ -0,0 +1,314 @@
#!/bin/bash
# Script to test transaction immutability on active database (from .env)
# This script safely tests database permissions without altering real data
#
# Tests:
# 1. Can we INSERT into transactions? (should be allowed)
# 2. Can we UPDATE transactions? (should be DENIED)
# 3. Can we DELETE transactions? (should be DENIED)
#
# Safety: Uses database transactions with ROLLBACK to prevent any data changes
set -e # Exit on error
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
echo -e "${BLUE}========================================${NC}"
echo -e "${BLUE}Transaction Immutability Test${NC}"
echo -e "${BLUE}========================================${NC}"
echo ""
# Get database credentials from .env
if [ ! -f .env ]; then
echo -e "${RED}Error: .env file not found${NC}"
exit 1
fi
DB_DATABASE=$(grep "^DB_DATABASE=" .env | cut -d '=' -f2 | sed 's/#.*//' | tr -d '"' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | sed 's/\r$//')
DB_USERNAME=$(grep "^DB_USERNAME=" .env | cut -d '=' -f2 | sed 's/#.*//' | tr -d '"' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | sed 's/\r$//')
DB_PASSWORD=$(grep "^DB_PASSWORD=" .env | cut -d '=' -f2- | sed 's/#.*//' | sed 's/^"//' | sed 's/"$//' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | sed 's/\r$//')
DB_HOST=$(grep "^DB_HOST=" .env | cut -d '=' -f2 | sed 's/#.*//' | tr -d '"' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | sed 's/\r$//')
# Default to localhost if not set
if [ -z "$DB_HOST" ]; then
DB_HOST="localhost"
fi
echo -e "${BLUE}Database:${NC} $DB_DATABASE"
echo -e "${BLUE}User:${NC} $DB_USERNAME"
echo -e "${BLUE}Host:${NC} $DB_HOST"
echo ""
# Check if mysql command is available
if ! command -v mysql &> /dev/null; then
echo -e "${RED}Error: mysql command not found${NC}"
echo "Please install mysql-client"
exit 1
fi
# Test database connection
echo -e "${YELLOW}Testing database connection...${NC}"
if ! MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" -e "SELECT 1;" 2>/dev/null; then
echo -e "${RED}Error: Cannot connect to database${NC}"
echo -e "${YELLOW}Debug info:${NC}"
echo -e " Database: $DB_DATABASE"
echo -e " Username: $DB_USERNAME"
echo -e " Host: $DB_HOST"
exit 1
fi
echo -e "${GREEN}✓ Database connection successful${NC}"
echo ""
# Get sample account IDs for testing
echo -e "${YELLOW}Getting sample account IDs...${NC}"
ACCOUNT_IDS=$(MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" -N -e "SELECT id FROM accounts LIMIT 2;")
FROM_ACCOUNT=$(echo "$ACCOUNT_IDS" | head -n 1)
TO_ACCOUNT=$(echo "$ACCOUNT_IDS" | tail -n 1)
if [ -z "$FROM_ACCOUNT" ] || [ -z "$TO_ACCOUNT" ]; then
echo -e "${RED}Error: Could not find sample accounts${NC}"
exit 1
fi
echo -e "${GREEN}✓ Using account IDs: $FROM_ACCOUNT → $TO_ACCOUNT${NC}"
echo ""
# Create a temporary test transaction ID variable
TEST_TRANSACTION_ID=""
# ====================
# TEST 1: INSERT (should be ALLOWED)
# ====================
echo -e "${BLUE}========================================${NC}"
echo -e "${BLUE}TEST 1: INSERT Permission${NC}"
echo -e "${BLUE}========================================${NC}"
MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" <<EOF
START TRANSACTION;
-- Try to insert a test transaction
INSERT INTO transactions (
from_account_id,
to_account_id,
amount,
description,
transaction_type_id,
created_at,
updated_at
) VALUES (
$FROM_ACCOUNT,
$TO_ACCOUNT,
1,
'IMMUTABILITY TEST - WILL BE ROLLED BACK',
1,
NOW(),
NOW()
);
-- Get the ID of the inserted test transaction
SELECT @test_id := LAST_INSERT_ID();
-- Display the test transaction
SELECT id, from_account_id, to_account_id, amount, description
FROM transactions
WHERE id = @test_id;
-- ROLLBACK to prevent any data changes
ROLLBACK;
SELECT 'Transaction ROLLED BACK - no data was changed' AS status;
EOF
INSERT_RESULT=$?
if [ $INSERT_RESULT -eq 0 ]; then
echo -e "${GREEN}✓ INSERT permission: ALLOWED${NC}"
echo -e "${GREEN} Database user CAN create new transactions${NC}"
else
echo -e "${RED}✗ INSERT permission: DENIED${NC}"
echo -e "${RED} Database user CANNOT create transactions${NC}"
fi
echo ""
# ====================
# TEST 2: UPDATE (should be DENIED)
# ====================
echo -e "${BLUE}========================================${NC}"
echo -e "${BLUE}TEST 2: UPDATE Permission${NC}"
echo -e "${BLUE}========================================${NC}"
# Get an existing transaction ID
EXISTING_ID=$(MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" -N -e "SELECT id FROM transactions ORDER BY id DESC LIMIT 1;")
if [ -z "$EXISTING_ID" ]; then
echo -e "${YELLOW}No existing transactions found, skipping UPDATE test${NC}"
UPDATE_RESULT=1
else
echo -e "Testing UPDATE on transaction ID: $EXISTING_ID"
# Get original amount
ORIGINAL_AMOUNT=$(MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" -N -e "SELECT amount FROM transactions WHERE id = $EXISTING_ID;")
echo -e "Original amount: $ORIGINAL_AMOUNT"
# Try to update (wrapped in transaction for safety)
MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" <<EOF 2>&1 | tee /tmp/update_test.log
START TRANSACTION;
-- Try to update an existing transaction
UPDATE transactions
SET amount = 99999, description = 'IMMUTABILITY TEST - SHOULD BE BLOCKED'
WHERE id = $EXISTING_ID;
-- Check if update succeeded
SELECT amount, description
FROM transactions
WHERE id = $EXISTING_ID;
-- ROLLBACK for safety
ROLLBACK;
EOF
UPDATE_RESULT=${PIPESTATUS[0]}
# Verify the transaction was not modified
CURRENT_AMOUNT=$(MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" -N -e "SELECT amount FROM transactions WHERE id = $EXISTING_ID;")
if [ "$CURRENT_AMOUNT" != "$ORIGINAL_AMOUNT" ]; then
echo -e "${RED}✗ UPDATE permission: ALLOWED (CRITICAL SECURITY ISSUE)${NC}"
echo -e "${RED} Transaction amount was changed from $ORIGINAL_AMOUNT to $CURRENT_AMOUNT${NC}"
echo -e "${RED} ⚠️ TRANSACTIONS ARE MUTABLE - THIS IS A CRITICAL FINANCIAL SECURITY ISSUE${NC}"
elif grep -q "denied" /tmp/update_test.log 2>/dev/null; then
echo -e "${GREEN}✓ UPDATE permission: DENIED${NC}"
echo -e "${GREEN} Database user CANNOT modify transactions (secure)${NC}"
else
# Update command succeeded but amount unchanged (transaction rolled back)
echo -e "${YELLOW}⚠ UPDATE permission: ALLOWED but transaction rolled back${NC}"
echo -e "${YELLOW} Database user HAS UPDATE permission (potential security issue)${NC}"
echo -e "${YELLOW} Data was not changed due to ROLLBACK${NC}"
fi
fi
echo ""
# ====================
# TEST 3: DELETE (should be DENIED)
# ====================
echo -e "${BLUE}========================================${NC}"
echo -e "${BLUE}TEST 3: DELETE Permission${NC}"
echo -e "${BLUE}========================================${NC}"
if [ -z "$EXISTING_ID" ]; then
echo -e "${YELLOW}No existing transactions found, skipping DELETE test${NC}"
DELETE_RESULT=1
else
echo -e "Testing DELETE on transaction ID: $EXISTING_ID"
# Verify transaction exists
EXISTS_BEFORE=$(MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" -N -e "SELECT COUNT(*) FROM transactions WHERE id = $EXISTING_ID;")
echo -e "Transaction exists: $EXISTS_BEFORE"
# Try to delete (wrapped in transaction for safety)
MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" <<EOF 2>&1 | tee /tmp/delete_test.log
START TRANSACTION;
-- Try to delete an existing transaction
DELETE FROM transactions WHERE id = $EXISTING_ID;
-- Check if delete succeeded
SELECT COUNT(*) as remaining
FROM transactions
WHERE id = $EXISTING_ID;
-- ROLLBACK for safety
ROLLBACK;
EOF
DELETE_RESULT=${PIPESTATUS[0]}
# Verify the transaction still exists
EXISTS_AFTER=$(MYSQL_PWD="$DB_PASSWORD" mysql -h"$DB_HOST" -u"$DB_USERNAME" "$DB_DATABASE" -N -e "SELECT COUNT(*) FROM transactions WHERE id = $EXISTING_ID;")
if [ "$EXISTS_AFTER" != "$EXISTS_BEFORE" ]; then
echo -e "${RED}✗ DELETE permission: ALLOWED (CRITICAL SECURITY ISSUE)${NC}"
echo -e "${RED} Transaction was deleted${NC}"
echo -e "${RED} ⚠️ TRANSACTIONS CAN BE DELETED - THIS IS A CRITICAL FINANCIAL SECURITY ISSUE${NC}"
elif grep -q "denied" /tmp/delete_test.log 2>/dev/null; then
echo -e "${GREEN}✓ DELETE permission: DENIED${NC}"
echo -e "${GREEN} Database user CANNOT delete transactions (secure)${NC}"
else
# Delete command succeeded but transaction still exists (rolled back)
echo -e "${YELLOW}⚠ DELETE permission: ALLOWED but transaction rolled back${NC}"
echo -e "${YELLOW} Database user HAS DELETE permission (potential security issue)${NC}"
echo -e "${YELLOW} Data was not changed due to ROLLBACK${NC}"
fi
fi
echo ""
# ====================
# FINAL SUMMARY
# ====================
echo -e "${BLUE}========================================${NC}"
echo -e "${BLUE}SUMMARY${NC}"
echo -e "${BLUE}========================================${NC}"
echo ""
ISSUES_FOUND=0
if [ $INSERT_RESULT -eq 0 ]; then
echo -e "${GREEN}✓ INSERT: ALLOWED (expected)${NC}"
else
echo -e "${RED}✗ INSERT: DENIED (unexpected - should be allowed)${NC}"
ISSUES_FOUND=$((ISSUES_FOUND + 1))
fi
# Check UPDATE test results
if grep -q "denied" /tmp/update_test.log 2>/dev/null; then
echo -e "${GREEN}✓ UPDATE: DENIED (expected - secure)${NC}"
elif [ $UPDATE_RESULT -ne 0 ]; then
echo -e "${GREEN}✓ UPDATE: DENIED (expected - secure)${NC}"
else
echo -e "${RED}✗ UPDATE: ALLOWED (security issue)${NC}"
ISSUES_FOUND=$((ISSUES_FOUND + 1))
fi
# Check DELETE test results
if grep -q "denied" /tmp/delete_test.log 2>/dev/null; then
echo -e "${GREEN}✓ DELETE: DENIED (expected - secure)${NC}"
elif [ $DELETE_RESULT -ne 0 ]; then
echo -e "${GREEN}✓ DELETE: DENIED (expected - secure)${NC}"
else
echo -e "${RED}✗ DELETE: ALLOWED (security issue)${NC}"
ISSUES_FOUND=$((ISSUES_FOUND + 1))
fi
echo ""
if [ $ISSUES_FOUND -eq 0 ]; then
echo -e "${GREEN}========================================${NC}"
echo -e "${GREEN}✓ ALL TESTS PASSED${NC}"
echo -e "${GREEN}Transaction immutability is properly enforced${NC}"
echo -e "${GREEN}========================================${NC}"
else
echo -e "${RED}========================================${NC}"
echo -e "${RED}✗ $ISSUES_FOUND SECURITY ISSUE(S) FOUND${NC}"
echo -e "${RED}Transaction immutability is NOT properly enforced${NC}"
echo -e "${RED}========================================${NC}"
echo ""
echo -e "${YELLOW}Recommended Fix:${NC}"
echo -e "${YELLOW}REVOKE UPDATE, DELETE ON $DB_DATABASE.transactions FROM '$DB_USERNAME'@'$DB_HOST';${NC}"
echo -e "${YELLOW}FLUSH PRIVILEGES;${NC}"
fi
echo ""
echo -e "${BLUE}Note: All tests used database transactions with ROLLBACK${NC}"
echo -e "${BLUE}No actual data was modified in the database${NC}"
# Cleanup
rm -f /tmp/update_test.log /tmp/delete_test.log
exit $ISSUES_FOUND
scripts/test-transactional-emails-info.txt
+82
View File
@@ -0,0 +1,82 @@
===============================================
IMPORTANT: How to View Test Emails
===============================================
Your application is configured to send emails to **Mailpit** (a local mail testing tool),
NOT to real email addresses.
MAILPIT WEB INTERFACE:
----------------------
Open your browser and navigate to:
http://localhost:8025
(Replace 'localhost' with your server IP if accessing remotely)
All emails sent by the test script will appear in this interface.
ALTERNATIVE: Send to Real Email
--------------------------------
If you want emails delivered to a real inbox, you need to update your .env file:
1. Open .env file
2. Update these settings with real SMTP credentials:
MAIL_MAILER=smtp
MAIL_HOST=your-smtp-server.com
MAIL_PORT=587
MAIL_USERNAME=your-email@example.com
MAIL_PASSWORD=your-password
MAIL_ENCRYPTION=tls
MAIL_FROM_ADDRESS=noreply@your-domain.com
3. Run: php artisan config:clear
4. Run the test script again
CURRENT CONFIGURATION:
----------------------
MAIL_MAILER: smtp
MAIL_HOST: localhost
MAIL_PORT: 1025 (Mailpit SMTP port)
Mailpit Web Interface: http://localhost:8025
EMAIL TYPES TESTED:
-------------------
The test script sends 20 different types of transactional emails in 5 languages (100 total):
1. Transfer Received - Notification when a payment is received
2. Profile Edited by Admin - Notification when admin edits a profile
3. Profile Link Changed - Notification when profile URL is changed
4a. User Deleted (Manual) - Manual deletion confirmation
4b. User Deleted (Auto-deletion) - Auto-deletion notification
5. Email Verification - Email address verification
6. Inactive Profile Warning 1 - First warning about inactivity
7. Inactive Profile Warning 2 - Second warning about inactivity
8. Inactive Profile Warning Final - Final warning before deletion
9. New Message (Chat) - New chat message notification
10. Reaction Created (Star) - Star/favorite notification
11. Tag Added - Tag/keyword added notification
12. Reservation Created - Event reservation confirmation
13. Reservation Cancelled - Event reservation cancellation
14. Reservation Update - Event update from organizer
15a. Contact Form - General Contact - General contact form message to admin
15b. Contact Form - Report Issue - Issue report to admin (community violations, etc.)
15c. Contact Form - Report Error - Technical error report to admin (with page URL)
15d. Contact Form - Delete Profile - Profile deletion request to admin
16a. Contact Form Copy - General Contact - Confirmation copy to submitter
16b. Contact Form Copy - Report Issue - Confirmation copy to issue reporter
16c. Contact Form Copy - Report Error - Confirmation copy to error reporter
16d. Contact Form Copy - Delete Profile - Confirmation copy to deletion requester
LANGUAGE SELECTION:
-------------------
When running the test script, you can choose to test:
- Single language (en, nl, de, es, or fr)
- All 5 languages at once
This helps verify that all translations are working correctly
scripts/test-transactional-emails.sh
Executable
+1049
View File
File diff suppressed because it is too large. Load diff
scripts/upgrade.sh
Executable
+571
View File
@@ -0,0 +1,571 @@
#!/bin/bash
#
# Upgrade this installation to a newer commit (latest main by default), from any older commit of main,
# with a verified backup and a rollback. Written for the Laravel 10 -> 11 upgrade (October 2026), usable
# for later ones.
#
# Run it from the application root, inside tmux or screen, using the NEWEST version of this script:
#
# git fetch origin
# git show origin/main:scripts/upgrade.sh > /tmp/timebank-upgrade.sh (or origin/laravel-11:...)
# bash /tmp/timebank-upgrade.sh --dry-run [--target origin/laravel-11] checks and plan, changes nothing
# bash /tmp/timebank-upgrade.sh [--target origin/laravel-11] the upgrade, step by step
# bash storage/app/upgrades/<stamp>/upgrade.sh --rollback back to the previous version
# bash /tmp/timebank-upgrade.sh --leave-branch --target origin/laravel-11 leave a platform branch (see below)
#
# --target defaults to origin/main. Upgrading servers one by one from origin/laravel-11 (a "staged rollout")
# keeps main on the old version, so routine ./deploy.sh runs on servers that are not upgraded yet do not
# upgrade them without this script's checks. Fast-forward main only when all servers are done or ready.
# Do not rewrite the target branch (rebase / force-push) after a server has upgraded from it.
#
# Extra arguments after "--" go to deploy.sh (default: -e server).
#
# --leave-branch: for an installation on a branch of its own (e.g. 'uuro') whose customisations now live in its
# platform folder (platform/, outside git). The update is then not a fast-forward: the installation moves to local
# branch main at the target, and the commits of its branch are no longer in it (a tag pre-leave-<branch>-<stamp>
# keeps them). Tracked files that only that branch has (e.g. config/<name>.php) are deleted by the move, so the
# platform folder must hold them first. The rollback returns to the branch and its files.
# UPGRADE_DEPLOY_SCRIPT=<path> runs another script instead of deploy.sh: only for rehearsals.
#
# Upgrade steps (the script waits for you at each one):
# 1. checks branch, clean tree, fast-forward, PHP version and extensions the target needs (CLI and
# web server), disk space, tmux/screen
# 2. plan current -> target commit and Laravel version, new migrations (adds tables / changes tables)
# 3. backup database backup, verified (complete dump), plus a snapshot of vendor/, public/build and the
# other files deploy.sh makes on the server, for an offline rollback
# 4. YOU switch on maintenance mode (with a bypass secret to test)
# 5. deploy git merge --ff-only <target>, then the NEW deploy.sh; stops here if deploy.sh fails
# 6. check Laravel version
# 7. YOU restart Reverb and reload the web server / PHP-FPM
# 8. go / no-go test through the bypass URL while still in maintenance; roll back now (database restore
# allowed: nothing live yet) or YOU switch maintenance off
# Everything is logged in storage/app/upgrades/<stamp>/, with the state the rollback needs.
set -u
TARGET="origin/main"
MODE="upgrade"
DRY_RUN=0
LEAVE_BRANCH=0
ROLLBACK_STAMP=""
DEPLOY_ARGS=()
while [ $# -gt 0 ]; do
case "$1" in
--dry-run) DRY_RUN=1 ;;
--leave-branch) LEAVE_BRANCH=1 ;;
--target) TARGET="${2:?--target needs a ref, like origin/laravel-11}"; shift ;;
--rollback) MODE="rollback"; if [ -n "${2:-}" ] && [[ "$2" != -* ]]; then ROLLBACK_STAMP="$2"; shift; fi ;;
--) shift; DEPLOY_ARGS=("$@"); break ;;
*) echo "Unknown option: $1" >&2; exit 2 ;;
esac
shift
done
[ ${#DEPLOY_ARGS[@]} -eq 0 ] && DEPLOY_ARGS=(-e server)
GREEN='\033[0;32m'; YELLOW='\033[1;33m'; RED='\033[0;31m'; BLUE='\033[0;34m'; BOLD='\033[1m'; NC='\033[0m'
step() { echo -e "\n${BLUE}${BOLD}== $1${NC}"; }
ok() { echo -e "${GREEN}$1${NC}"; }
warn() { echo -e "${YELLOW}$1${NC}"; }
stop() { echo -e "${RED}$1${NC}" >&2; exit 1; }
confirm() { local a; read -r -p "$1 (y/N) " a; [[ "$a" =~ ^[Yy]$ ]]; }
wait_for() { read -r -p "$(echo -e "${BOLD}$1${NC} Press Enter when done (Ctrl+C stops) ")" _; }
STATE_ROOT="storage/app/upgrades"
SITE_PHP=""
lock_laravel_version() {
php -r '$l = json_decode(stream_get_contents(STDIN), true); foreach (($l["packages"] ?? []) as $p) if ($p["name"] === "laravel/framework") { echo ltrim($p["version"], "v"); exit; }'
}
# The php binary deploy.sh uses for composer (php8.3 when present)
deploy_php() { command -v php8.3 >/dev/null 2>&1 && echo php8.3 || echo php; }
state_set() { # key value -> state.env
local file="$STATE_DIR/state.env"
touch "$file"
grep -v "^$1=" "$file" > "$file.tmp" 2>/dev/null; mv "$file.tmp" "$file"
printf '%s=%q\n' "$1" "$2" >> "$file"
}
# A gzipped mysqldump that is complete: valid gzip and the trailer mysqldump writes last
verify_backup() {
local f="$1"
[ -s "$f" ] || { warn "Backup file missing or empty: $f"; return 1; }
gzip -t "$f" 2>/dev/null || { warn "Backup is not a valid gzip file: $f"; return 1; }
gzip -dc "$f" | tail -c 400 | grep -q "Dump completed" || { warn "Backup is incomplete (no 'Dump completed' at the end): $f"; return 1; }
ok "Backup verified: $f ($(du -h "$f" | cut -f1))"
}
# Value of a .env key, parsed the way Laravel parses it (quotes, inline comments, escapes)
env_value() {
php -r 'require "vendor/autoload.php"; $e = Dotenv\Dotenv::parse(file_get_contents(".env")); echo $e[$argv[1]] ?? "";' "$1" 2>/dev/null
}
# Restore a verified backup. The dump drops and recreates the database, which the restricted application
# user may not do, so this connects as the user deploy.sh uses for migrations (DB_DEPLOY_USERNAME and
# DB_DEPLOY_PASSWORD in .env) or asks for one. Self-contained on purpose: after the code is reset, the
# installation's own scripts/restore-database.sh is the old version.
restore_database() {
local file="$1" user password host port cnf status
verify_backup "$file" || return 1
user=$(env_value DB_DEPLOY_USERNAME); password=$(env_value DB_DEPLOY_PASSWORD)
if [ -z "$user" ]; then
read -r -p "MySQL user that may drop and create the database [root]: " user
user="${user:-root}"
fi
if [ -z "$password" ]; then
read -r -s -p "MySQL password for $user: " password
echo ""
fi
host=$(env_value DB_HOST); port=$(env_value DB_PORT)
cnf=$(mktemp); chmod 600 "$cnf"
password="${password//\\/\\\\}"; password="${password//\"/\\\"}"
printf '[client]\nhost=%s\nport=%s\nuser=%s\npassword="%s"\n' "${host:-127.0.0.1}" "${port:-3306}" "$user" "$password" > "$cnf"
echo " Restoring $(basename "$file") as MySQL user $user ..."
set -o pipefail
gzip -dc "$file" | mysql --defaults-extra-file="$cnf"; status=$?
set +o pipefail
rm -f "$cnf"
return $status
}
restart_hints() {
if command -v systemctl >/dev/null 2>&1; then
for unit in $(systemctl list-units --all --type=service --plain --no-legend 2>/dev/null | awk '{print $1}' | grep -iE 'reverb|timebank|apache2|httpd|nginx|php[0-9.]*-fpm'); do
case "$unit" in
*reverb*|*timebank*) echo " sudo systemctl restart $unit" ;;
php*-fpm*) if [ -z "${SITE_PHP:-}" ] || [[ "$unit" == php${SITE_PHP}-fpm* ]]; then
systemctl is-active --quiet "$unit" && echo " sudo systemctl reload $unit"; fi ;;
*) systemctl is-active --quiet "$unit" && echo " sudo systemctl reload $unit" ;;
esac
done
fi
if command -v supervisorctl >/dev/null 2>&1; then
for program in $(sudo -n supervisorctl status 2>/dev/null | awk '{print $1}' | grep -iE 'reverb|timebank|queue'); do
echo " sudo supervisorctl restart $program"
done
fi
}
maintenance_on_prompt() {
local secret="$1"
echo " In another terminal, in $(pwd), run:"
echo " php artisan down --retry=60 --secret=$secret"
echo " Visitors get the maintenance page; you can still test the site by opening <site-url>/$secret once."
wait_for "Switch on maintenance mode."
if [ -f storage/framework/down ]; then ok "Maintenance mode is on."; else
warn "storage/framework/down not found: maintenance mode seems to be off."
confirm "Continue without maintenance mode?" || exit 1
fi
}
[ -f artisan ] && [ -f composer.json ] && [ -d .git ] || stop "Run this from the application root (the folder with artisan, composer.json and .git)."
# ===========================================================================================
if [ "$MODE" = "rollback" ]; then
if [ -z "$ROLLBACK_STAMP" ]; then
ROLLBACK_STAMP=$(ls -1 "$STATE_ROOT" 2>/dev/null | grep -E '^[0-9]{8}-[0-9]{6}$' | sort | tail -1)
fi
STATE_DIR="$STATE_ROOT/$ROLLBACK_STAMP"
[ -n "$ROLLBACK_STAMP" ] && [ -f "$STATE_DIR/state.env" ] || stop "No upgrade state found in $STATE_ROOT (give the stamp: --rollback 20261012-170000)."
# shellcheck disable=SC1090
source "$STATE_DIR/state.env"
exec > >(tee -a "$STATE_DIR/rollback.log") 2>&1
# A site that is out of maintenance mode now has been live on the new version, whatever the state says
# (e.g. 'php artisan up' run by hand after the upgrade script ended)
if [ "${WENT_LIVE:-0}" != 1 ] && [ ! -f storage/framework/down ] && [ -n "${TARGET_COMMIT:-}" ]; then
WENT_LIVE=1
state_set WENT_LIVE 1
fi
step "Rollback of upgrade $ROLLBACK_STAMP"
echo " previous version: $(git log -1 --format='%h %s' "$PREV_COMMIT" 2>/dev/null)"
echo " upgraded to: $(git log -1 --format='%h %s' "${TARGET_COMMIT:-$PREV_COMMIT}" 2>/dev/null)"
echo " now running: $(git log -1 --format='%h %s')"
echo " went live: ${WENT_LIVE:-0} database backup: ${BACKUP_FILE:-none} vendor snapshot: ${VENDOR_SNAPSHOT:-none}"
[ "${TABLE_CHANGES:-0}" -gt 0 ] && warn " ${TABLE_CHANGES} of the new migrations change existing tables: the old code may not work on the new schema without the database restore."
confirm "Roll back to $(git rev-parse --short "$PREV_COMMIT")?" || exit 1
restore_db=0
if [ -n "${BACKUP_FILE:-}" ]; then
if [ "${WENT_LIVE:-0}" = 1 ]; then
warn "The site has been live on the new version. Restoring the backup deletes everything written since"
warn "the backup: payments (transactions), messages, votes, new profiles. Usually roll back the CODE only."
read -r -p "Type RESTORE to restore the database anyway, or press Enter for code only: " answer
[ "$answer" = "RESTORE" ] && restore_db=1
else
confirm "Restore the database backup too (safe: the new version was never live)?" && restore_db=1
fi
fi
step "Maintenance mode"
maintenance_on_prompt "$(head -c 12 /dev/urandom | od -An -tx1 | tr -d ' \n')"
step "Code"
if [ -n "${PREV_BRANCH:-}" ] && [ "${LEFT_BRANCH:-0}" = 1 ]; then
# The upgrade left branch PREV_BRANCH for main: go back to it, and put main back as it was
git checkout -q -f "$PREV_BRANCH" || stop "git checkout $PREV_BRANCH failed."
git reset --hard -q "$PREV_COMMIT" || stop "git reset failed."
if [ -n "${PREV_MAIN:-}" ]; then
git branch -f main "$PREV_MAIN" >/dev/null || warn "Could not put branch main back at $PREV_MAIN."
else
git branch -D main >/dev/null 2>&1 || true
fi
ok "Back on branch $PREV_BRANCH at $(git log -1 --format='%h %s')"
else
git reset --hard -q "$PREV_COMMIT" || stop "git reset failed."
ok "Code back at $(git log -1 --format='%h %s')"
fi
step "Packages"
if [ -n "${VENDOR_SNAPSHOT:-}" ] && [ -f "$VENDOR_SNAPSHOT" ]; then
tar -tzf "$VENDOR_SNAPSHOT" | grep "^public/build/" >/dev/null && rm -rf public/build
rm -rf vendor && tar -xzf "$VENDOR_SNAPSHOT" || stop "Restoring the vendor snapshot failed; run: composer install --no-dev --optimize-autoloader"
ok "vendor/, bootstrap/cache and this server's public/build, package-lock.json and emoji picker restored from the snapshot (no download needed)."
else
warn "No vendor snapshot: installing the old packages with composer (needs the network)."
$(deploy_php) "$(command -v composer)" install --no-dev --no-interaction --prefer-dist --optimize-autoloader --ignore-platform-req=php || stop "composer install failed."
fi
php artisan optimize:clear >/dev/null 2>&1 && ok "Caches cleared." || warn "php artisan optimize:clear failed: check before going live."
# git reset --hard put back the symlink git tracks, which points into a developer's folder
php artisan storage:link --force >/dev/null 2>&1 && ok "public/storage linked to this server's storage." \
|| warn "php artisan storage:link --force failed: uploaded images do not show until it runs."
if [ "$restore_db" = 1 ]; then
step "Database"
if restore_database "$BACKUP_FILE"; then
ok "Database restored from $(basename "$BACKUP_FILE")."
else
warn "Database restore failed. Restore it yourself with a MySQL user that may drop databases:"
echo " gzip -dc '$BACKUP_FILE' | mysql -u root -p"
wait_for "Restore the database."
fi
else
warn "Database not restored: the new tables and columns stay (the old code ignores tables it does not know)."
fi
step "Restart workers, Reverb and the web server"
echo " Long-running processes hold the new code. Restart them, for example:"
restart_hints
wait_for "Restart the queue workers and Reverb, and reload the web server / PHP-FPM."
echo " Search: the index was built by the new version; rebuild it with the old code if search misbehaves:"
echo " bash scripts/re-index-search.sh"
step "Check, then go live"
echo " Test through the bypass URL, then run: php artisan up"
wait_for "Switch off maintenance mode."
[ -f storage/framework/down ] && warn "Still in maintenance mode." || ok "Live again on $(git log -1 --format='%h %s')."
state_set ROLLED_BACK "$(date '+%Y-%m-%d %H:%M:%S')"
warn "deploy.sh pulls main: do not run it on this server until main (or the target branch) is fixed."
exit 0
fi
# ===========================================================================================
step "1. Checks"
branch=$(git rev-parse --abbrev-ref HEAD)
if [ "$LEAVE_BRANCH" = 1 ]; then
[ "$branch" != "main" ] && [ "$branch" != "HEAD" ] || stop "--leave-branch is for an installation on a branch of its own; this one is on '$branch'."
ok "On branch '$branch', which this upgrade leaves (--leave-branch)."
else
[ "$branch" = "main" ] || { warn "This installation is on branch '$branch', not main."; confirm "Continue anyway?" || exit 1; }
fi
# Mode-only changes are ignored (core.fileMode=false): deploy.sh runs chmod -R 775 on storage/ and public/build,
# which makes tracked files executable on every server without changing their content.
# Some tracked files are made again by deploy.sh on every server, so they differ from git there. That is
# expected: they are reset just before the update and deploy.sh makes them again.
# public/build rebuilt with this server's .env (Reverb host and key are compiled in)
# package-lock.json rewritten by npm install
# public/js/vendor/emoji-picker-* copied from node_modules
# public/storage is this server's storage:link symlink; git tracks a developer's path. It is never reset, and
# the rollback links it again. Any other local change stops the upgrade.
generated_paths=(public/build package-lock.json public/js/vendor/emoji-picker-element public/js/vendor/emoji-picker-element-data)
not_checked=()
for path in "${generated_paths[@]}" public/storage; do not_checked+=(":(exclude)$path"); done
generated_dirty=()
for path in "${generated_paths[@]}"; do
[ -n "$(git -c core.fileMode=false status --porcelain -- "$path")" ] && generated_dirty+=("$path")
done
if [ -n "$(git -c core.fileMode=false status --porcelain --untracked-files=no -- . "${not_checked[@]}")" ]; then
git -c core.fileMode=false status --short --untracked-files=no -- . "${not_checked[@]}"
stop "There are local changes to tracked files. Commit, stash or revert them first: the update must be a clean fast-forward."
fi
[ ${#generated_dirty[@]} -gt 0 ] && ok "Made on this server, differ from git: ${generated_dirty[*]}. Reset before the update, made again by deploy.sh."
remote="${TARGET%%/*}"; remote_branch="${TARGET#*/}"
git fetch -q "$remote" "$remote_branch" || stop "git fetch $remote $remote_branch failed."
current=$(git rev-parse HEAD)
target=$(git rev-parse "$TARGET") || stop "Unknown target $TARGET."
[ "$current" = "$target" ] && { ok "Already at $TARGET ($(git log -1 --format='%h %s'))."; exit 0; }
left_commits=0
if ! git merge-base --is-ancestor HEAD "$TARGET"; then
[ "$LEAVE_BRANCH" = 1 ] || stop "This installation has commits that are not on $TARGET: a fast-forward is not possible. An installation on a platform branch of its own moves with --leave-branch, once its customisations are in its platform folder."
left_commits=$(git rev-list --count "$TARGET"..HEAD)
warn "$left_commits commit(s) of branch '$branch' will no longer be in this installation (kept in tag pre-leave-$branch-<stamp>):"
git log --format=' %h %ad %an %s' --date=short "$TARGET"..HEAD | head -40
[ "$left_commits" -gt 40 ] && echo " ..."
if [ -n "$(git rev-parse -q --verify "refs/remotes/$remote/$branch")" ] && [ -n "$(git rev-list "refs/remotes/$remote/$branch"..HEAD)" ]; then
warn "Of these, not pushed to $remote/$branch: $(git rev-list --count "refs/remotes/$remote/$branch"..HEAD). Push them first if they are worth keeping elsewhere."
fi
elif [ "$LEAVE_BRANCH" = 1 ]; then
ok "Branch '$branch' has no commits that $TARGET lacks: the move loses nothing."
fi
if [ "$LEAVE_BRANCH" = 1 ]; then
# Local main must not hold work of its own: the move sets it to the target
if [ -n "$(git rev-parse -q --verify refs/heads/main)" ] && ! git merge-base --is-ancestor refs/heads/main "$TARGET"; then
stop "Local branch main has commits that $TARGET lacks; the move would discard them. Look at: git log $TARGET..main"
fi
# Tracked files the move deletes: everything this installation's platform needs must be in the platform folder
deleted=$(git diff --no-renames --name-only --diff-filter=D HEAD "$TARGET")
if [ -n "$deleted" ]; then
warn "Tracked files the move deletes ($(echo "$deleted" | wc -l)):"
echo "$deleted" | sed 's/^/ /' | head -30
fi
leave_platform_dir=$(env_value PLATFORM_PATH); leave_platform_dir="${leave_platform_dir:-platform}"
[[ "$leave_platform_dir" = /* ]] || leave_platform_dir="$PWD/$leave_platform_dir"
platform_name=$(env_value TIMEBANK_CONFIG)
if echo "$deleted" | grep -qx "config/${platform_name:-timebank-default}.php" && [ ! -f "$leave_platform_dir/config.php" ]; then
stop "The move deletes config/${platform_name}.php, this installation's platform config, and $leave_platform_dir/config.php does not exist. Put the platform's files in its platform folder first."
fi
# Untracked files (not ignored) that the target tracks would stop the checkout
collisions=$(comm -12 <(git ls-files --others --exclude-standard | sort) <(git ls-tree -r --name-only "$TARGET" | sort))
if [ -n "$collisions" ]; then
echo "$collisions" | sed 's/^/ /' | head -20
stop "These untracked files exist in $TARGET too and would block the move. Move them away first."
fi
fi
# PHP version and extensions the target needs, checked with the php deploy.sh uses for composer
tmp=$(mktemp -d)
git show "$TARGET:composer.json" > "$tmp/composer.json" && git show "$TARGET:composer.lock" > "$tmp/composer.lock"
if (cd "$tmp" && $(deploy_php) "$(command -v composer)" check-platform-reqs --lock --no-dev --no-interaction >"$tmp/reqs.txt" 2>&1); then
ok "$(deploy_php) $( $(deploy_php) -r 'echo PHP_VERSION;') meets the PHP version and extensions $TARGET needs."
else
grep -iE "missing|failed" "$tmp/reqs.txt" | head -10
rm -rf "$tmp"
stop "This server's PHP does not meet the requirements of $TARGET (see above)."
fi
required_php=$(php -r '$c = json_decode(file_get_contents($argv[1]), true); preg_match("/(\d+\.\d+)/", $c["require"]["php"] ?? "", $m); echo $m[1] ?? "";' "$tmp/composer.json")
rm -rf "$tmp"
# The web server's PHP can differ from the command line's, and a server can run several PHP versions for
# different sites. Ask this site itself: a randomly named one-line file in public/, removed right away.
app_url=$(grep -E '^APP_URL=' .env 2>/dev/null | tail -1 | cut -d= -f2- | sed -E 's/[[:space:]]+#.*$//; s/^"//; s/"$//; s#/+$##')
probe="php-version-$(head -c 8 /dev/urandom | od -An -tx1 | tr -d ' \n').php"
web_php=""
if [ -n "$app_url" ] && command -v curl >/dev/null 2>&1; then
echo '<?php echo PHP_VERSION;' > "public/$probe"
web_php=$(curl -sk --max-time 15 "$app_url/$probe" | grep -oE '^[0-9]+\.[0-9]+\.[0-9]+$')
rm -f "public/$probe"
fi
if [ -n "$web_php" ]; then
if [ -n "$required_php" ] && php -r "exit(version_compare('$web_php', '$required_php', '<') ? 0 : 1);"; then
stop "This site is served by PHP $web_php; $TARGET needs PHP $required_php or newer. Switch the site's PHP first."
fi
ok "This site is served by PHP $web_php"
SITE_PHP=$(echo "$web_php" | cut -d. -f1,2)
else
warn "Could not ask the site which PHP serves it ($app_url). PHP versions found on this server:"
ls /etc/apache2/mods-enabled/ 2>/dev/null | sed -nE 's/^php([0-9]+\.[0-9]+)\.load$/ Apache module PHP \1/p'
systemctl list-units --type=service --state=running --plain --no-legend 2>/dev/null | awk '{print $1}' | sed -nE 's/^php([0-9]+\.[0-9]+)-fpm\.service$/ PHP-FPM \1 (running)/p'
confirm "Is this site served by PHP ${required_php:-?} or newer?" || stop "Switch the site's PHP first."
fi
for tool in composer npm gzip tar; do command -v "$tool" >/dev/null 2>&1 || stop "$tool not found (deploy.sh / this script need it)."; done
vendor_kb=$(du -sk vendor 2>/dev/null | cut -f1); vendor_kb=${vendor_kb:-0}
free_kb=$(df -Pk . | awk 'NR==2 {print $4}')
need_kb=$(( vendor_kb * 2 + 2 * 1024 * 1024 ))
if [ "$free_kb" -lt "$need_kb" ]; then
stop "Only $((free_kb / 1024)) MB free; about $((need_kb / 1024)) MB needed (backup, vendor snapshot, new packages, build)."
fi
ok "Disk: $((free_kb / 1024)) MB free"
# This installation's platform folder (App\Providers\PlatformServiceProvider) is outside git: the update and a
# rollback leave it as it is. Shown so it is clear which platform files the installation runs on.
platform_dir=$(env_value PLATFORM_PATH); platform_dir="${platform_dir:-platform}"
[[ "$platform_dir" = /* ]] || platform_dir="$PWD/$platform_dir"
if [ -d "$platform_dir" ]; then
ok "Platform folder: $platform_dir ($(cd "$platform_dir" && ls -d * 2>/dev/null | tr '\n' ' '))"
else
ok "No platform folder: this installation runs on config/ and the files in the code."
fi
if [ "$DRY_RUN" = 0 ] && [ -z "${TMUX:-}" ] && [ -z "${STY:-}" ]; then
warn "Not running inside tmux or screen: if the SSH connection drops, the deploy stops halfway."
confirm "Continue anyway?" || exit 1
fi
# ===========================================================================================
step "2. Plan"
laravel_now=$(lock_laravel_version < composer.lock)
laravel_target=$(git show "$TARGET:composer.lock" | lock_laravel_version)
echo " current: $(git log -1 --format='%h %ad %s' --date=short HEAD) Laravel $laravel_now"
echo " target: $(git log -1 --format='%h %ad %s' --date=short "$TARGET") Laravel $laravel_target ($TARGET)"
echo " $(git rev-list --count HEAD.."$TARGET") commits to apply"
[ "$LEAVE_BRANCH" = 1 ] && warn " Leaves branch '$branch' for main ($left_commits of its commits are not in $TARGET)."
[ "${laravel_now%%.*}" != "${laravel_target%%.*}" ] && warn " Laravel major upgrade (${laravel_now%%.*} -> ${laravel_target%%.*})."
new_migrations=0; table_changes=0
migration_files=$(git diff --name-only --diff-filter=A HEAD "$TARGET" -- database/migrations | sort)
# Tables created by this upgrade: changing them later in the same upgrade does not touch existing data
created_tables=" $(for m in $migration_files; do git show "$TARGET:$m"; done | grep -oE "Schema::create\(['\"][^'\"]+" | sed -E "s/Schema::create\(['\"]//" | tr '\n' ' ') "
for m in $migration_files; do
new_migrations=$((new_migrations + 1))
up=$(git show "$TARGET:$m" | sed -n '/function up/,/function down/p')
touched=$(echo "$up" | grep -oE "Schema::(table|drop|dropIfExists|rename)\(['\"][^'\"]+" | sed -E "s/Schema::[a-zA-Z]+\(['\"]//")
risky=0
for t in $touched; do [[ "$created_tables" == *" $t "* ]] || risky=1; done
echo "$up" | grep -qE 'DB::statement|DB::table\(|->update\(|->delete\(|->insert\(' && risky=1
if [ "$risky" = 1 ]; then
table_changes=$((table_changes + 1)); echo " changes existing tables/data: $m"
else
echo " adds tables only: $m"
fi
done
echo " $new_migrations new migrations, $table_changes of them change existing tables or data"
echo " Downtime: composer install, migrations, a full search re-index (search is empty until it is done) and the"
echo " asset build all run in maintenance mode. Queue workers and scheduled tasks (health checks too) pause meanwhile."
if [ "$DRY_RUN" = 1 ]; then
step "Dry run: nothing was changed"
exit 0
fi
confirm "Upgrade this installation to $(git rev-parse --short "$TARGET")?" || exit 1
if [ "$LEAVE_BRANCH" = 1 ]; then
read -r -p "Type the branch name '$branch' to leave it: " typed
[ "$typed" = "$branch" ] || stop "Not confirmed: nothing was changed."
fi
STAMP=$(date '+%Y%m%d-%H%M%S')
STATE_DIR="$STATE_ROOT/$STAMP"
mkdir -p "$STATE_DIR"
cp "$0" "$STATE_DIR/upgrade.sh"
exec > >(tee -a "$STATE_DIR/upgrade.log") 2>&1
state_set PREV_COMMIT "$current"
state_set PREV_BRANCH "$branch"
state_set PREV_MAIN "$(git rev-parse -q --verify refs/heads/main || true)"
state_set LEFT_BRANCH 0
state_set TARGET "$TARGET"
state_set TABLE_CHANGES "$table_changes"
state_set WENT_LIVE 0
state_set SITE_PHP "${SITE_PHP:-}"
ok "State and log: $STATE_DIR (rollback: bash $STATE_DIR/upgrade.sh --rollback $STAMP)"
# ===========================================================================================
step "3. Backup"
backup_file=""
if [ -f scripts/backup-database.sh ] && confirm "Make a database backup now with scripts/backup-database.sh?"; then
marker="$STATE_DIR/before-backup"; touch "$marker"; sleep 1
bash scripts/backup-database.sh pre-restore || warn "backup-database.sh reported an error."
backup_file=$(find backups/database -name '*.sql.gz' -newer "$marker" 2>/dev/null | sort | tail -1)
fi
until [ -n "$backup_file" ] && verify_backup "$backup_file"; do
read -r -p "Path to a complete gzipped mysqldump you made yourself (Ctrl+C stops): " backup_file
done
# backup-database.sh keeps only the last few pre-restore backups: keep this one with the upgrade state
# (a hard link costs no space and survives that rotation; copy if a link is not possible)
kept="$STATE_DIR/$(basename "$backup_file")"
ln "$backup_file" "$kept" 2>/dev/null || cp "$backup_file" "$kept" || stop "Could not keep a copy of the backup in $STATE_DIR."
state_set BACKUP_FILE "$(readlink -f "$kept")"
ok "Backup kept with the upgrade state: $kept"
if [ -d vendor ]; then
# The files deploy.sh made on this server too: a rollback must get this server's own versions back,
# not the committed ones
snapshot_dirs="vendor bootstrap/cache"
for path in "${generated_paths[@]}"; do [ -e "$path" ] && snapshot_dirs="$snapshot_dirs $path"; done
tar -czf "$STATE_DIR/vendor.tar.gz" $snapshot_dirs 2>/dev/null || stop "Could not snapshot vendor/."
state_set VENDOR_SNAPSHOT "$(readlink -f "$STATE_DIR/vendor.tar.gz")"
ok "Snapshot of $snapshot_dirs: $STATE_DIR/vendor.tar.gz ($(du -h "$STATE_DIR/vendor.tar.gz" | cut -f1))"
fi
# ===========================================================================================
step "4. Maintenance mode"
secret=$(head -c 12 /dev/urandom | od -An -tx1 | tr -d ' \n')
state_set BYPASS_SECRET "$secret"
maintenance_on_prompt "$secret"
# ===========================================================================================
step "5. Update code and deploy"
if [ ${#generated_dirty[@]} -gt 0 ]; then
git checkout -q -- "${generated_dirty[@]}" && git clean -fdq -- "${generated_dirty[@]}" \
|| stop "Could not reset ${generated_dirty[*]}. The code is unchanged; switch off maintenance mode (php artisan up)."
fi
if [ "$LEAVE_BRANCH" = 1 ]; then
git tag -f "pre-leave-$branch-$STAMP" HEAD >/dev/null || stop "Could not tag the current commit. The code is unchanged; switch off maintenance mode (php artisan up)."
# public/storage is this server's storage link and differs from git everywhere; a fast-forward leaves it alone,
# a move to another branch does not. Reset it now; it is linked again after the deploy.
git checkout -q -- public/storage 2>/dev/null || true
state_set LEFT_BRANCH 1
git -c core.fileMode=false checkout -q -B main "$TARGET" || stop "git checkout -B main $TARGET failed. Roll back: bash $STATE_DIR/upgrade.sh --rollback $STAMP"
git branch -q --set-upstream-to="$remote/main" main 2>/dev/null || true
ok "Left branch '$branch' (tag pre-leave-$branch-$STAMP); now on main."
else
git -c core.fileMode=false merge --ff-only -q "$TARGET" || stop "git merge --ff-only $TARGET failed. The code is unchanged; switch off maintenance mode (php artisan up)."
fi
state_set TARGET_COMMIT "$(git rev-parse HEAD)"
ok "Code at $(git log -1 --format='%h %s')"
deploy_script="${UPGRADE_DEPLOY_SCRIPT:-deploy.sh}"
echo " Running the new $deploy_script ${DEPLOY_ARGS[*]} (it asks its own questions)."
bash "$deploy_script" "${DEPLOY_ARGS[@]}"
deploy_status=$?
if [ $deploy_status -ne 0 ]; then
warn "deploy.sh stopped with an error (exit $deploy_status). The site stays in maintenance mode."
warn "Either fix the problem and run: bash deploy.sh ${DEPLOY_ARGS[*]} then continue with steps 6-8 by hand,"
warn "or roll back (database restore allowed, nothing went live): bash $STATE_DIR/upgrade.sh --rollback $STAMP"
exit 1
fi
if [ "$LEAVE_BRANCH" = 1 ]; then
# deploy.sh runs storage:link without --force, which keeps the link git checked out (a developer's path)
php artisan storage:link --force >/dev/null 2>&1 && ok "public/storage linked to this server's storage." \
|| warn "php artisan storage:link --force failed: uploaded images do not show until it runs."
fi
# ===========================================================================================
step "6. Version check"
running=$(php artisan --version 2>/dev/null)
echo " ${running:-artisan did not run}"
case "$running" in
"Laravel Framework ${laravel_target%%.*}."*) ok "Running Laravel ${laravel_target%%.*}." ;;
*) warn "Expected Laravel $laravel_target."
confirm "Continue anyway?" || { warn "Roll back: bash $STATE_DIR/upgrade.sh --rollback $STAMP"; exit 1; } ;;
esac
# ===========================================================================================
step "7. Restart Reverb and the web server"
echo " Long-running processes may still hold the old code (deploy.sh restarted the queue workers). For example:"
restart_hints
wait_for "Restart Reverb and reload the web server / PHP-FPM."
# ===========================================================================================
step "8. Go / no-go (still in maintenance mode)"
echo " Test through <site-url>/$secret : log in, switch profile, a payment, a chat message with an image,"
echo " a poll, search; and check storage/logs/laravel-$(date +%F).log for new errors."
read -r -p "Go live (g) or roll back now (r)? " decision
if [[ "$decision" =~ ^[Rr]$ ]]; then
exec bash "$STATE_DIR/upgrade.sh" --rollback "$STAMP"
fi
# Recorded now, not after the check below: 'php artisan up' may well run after Enter, and a later rollback
# must never offer the database restore as safe once real data may have been written.
state_set WENT_LIVE 1
echo " Run: php artisan up"
wait_for "Switch off maintenance mode."
if [ -f storage/framework/down ]; then
warn "storage/framework/down still exists: the site is still in maintenance mode. Run: php artisan up"
else
ok "Live on $(git log -1 --format='%h %s')."
fi
echo " Rollback later (code only by default, it keeps data written since): bash $STATE_DIR/upgrade.sh --rollback $STAMP"
echo " Free the disk later: rm $STATE_DIR/vendor.tar.gz (once the new version has proven itself)"
scripts/verify-translations.php
Executable
+28
View File
@@ -0,0 +1,28 @@
<?php
echo "=== FINAL TRANSLATION VERIFICATION ===\n\n";
$langs = ['en', 'nl', 'de', 'es', 'fr'];
foreach ($langs as $lang) {
$file = "resources/lang/{$lang}.json";
$data = json_decode(file_get_contents($file), true);
$count = count($data);
if ($lang !== 'en') {
$untranslated = 0;
foreach ($data as $key => $value) {
if ($key === $value) {
$untranslated++;
}
}
$translated = $count - $untranslated;
$percentage = round(($translated / $count) * 100, 1);
echo sprintf("%-4s: %4d keys | %4d translated (%5.1f%%) | %3d remaining\n",
strtoupper($lang), $count, $translated, $percentage, $untranslated);
} else {
echo sprintf("%-4s: %4d keys (source)\n", strtoupper($lang), $count);
}
}
echo "\n✓ All languages have been synchronized!\n";