timebank-cc-public/references/RUNBOOK_LARAVEL_10_TO_11.md

# Runbook: Uuro staging server, Laravel 10 to Laravel 11

The checklist for upgrading the Uuro staging server. Work through it top to bottom and tick the boxes. The
background, every question the script asks and the troubleshooting table are in
[UPGRADE_LARAVEL_10_TO_11.md](UPGRADE_LARAVEL_10_TO_11.md).

Timebank.cc runs Laravel 11 since 2026-10-15; Uuro is the last installation on Laravel 10.

| | |
|---|---|
| From | branch `uuro`, Laravel 10.48 |
| To | `origin/laravel-11`, Laravel 11.57, on branch `main` |
| Tool | `scripts/upgrade.sh --leave-branch`, from the target branch, run from `/tmp` |
| Duration | about 30 minutes; the site shows the maintenance page for 15 to 20 of them (search is rebuilt) |
| Rollback | `bash storage/app/upgrades/<stamp>/upgrade.sh --rollback`: back to branch `uuro` and its files |

**Why the branch goes.** Uuro ran a branch of its own with its config, logos, favicons and Open Graph tags. Those now
live in the platform folder `platform/` (outside git, see `MULTI_PLATFORM_INSTANCES.md`, section 3), and the Open
Graph tags and extra favicons are in the code for every platform. So the installation moves to `main`, and later
upgrades are the same as for Timebank.cc. The upgrade cannot fast-forward the branch; `--leave-branch` moves the
installation instead:
- it lists the commits of `uuro` that will no longer be in the installation and keeps them in a tag
  `pre-leave-uuro-<stamp>`; `origin/uuro` stays on Gitea as an archive;
- it lists the tracked files the move deletes, among them `config/uuro.php`, and refuses to move while
  `platform/config.php` is missing;
- it asks you to type `uuro`, then moves the installation to branch `main` at the target;
- the rollback returns to branch `uuro` with all its files.

Rehearsed on a copy of `origin/uuro` on 2026-10-19: the move, the site from the platform folder, the rollback.

Start every terminal session by setting the application root once (the server's own path):

```bash
export APP=/var/www/CHANGE-ME     # the application root on this server
```

Every block below is read-only unless it says otherwise.

---

## T-1 day: preparation

### 1. Pick the moment

- [ ] A quiet hour; the Uuro developer and someone who can test (log in, pay, chat) are available.

### 2. Check the server

As root:

```bash
(
cd "${APP:?set APP first}" || exit 1
echo "== versions";  php artisan --version; php -v | head -1; git branch --show-current
echo "== tmux";      tmux -V 2>/dev/null || echo "tmux MISSING: apt-get install -y tmux"
echo "== disk";      df -h . | tail -1; du -sh vendor
echo "== local changes (a changed public/storage and public/build are normal)"
git -c core.fileMode=false status --short --untracked-files=no
echo "== commits not on Gitea (should be empty)"
git fetch -q origin uuro && git log --oneline origin/uuro..HEAD
echo "== .env keys"
grep -E '^(APP_URL|DEPLOY_ENVIRONMENT|DEPLOY_APP_DIR|DEPLOY_SERVER_TYPE|DEPLOY_WS_URL|TIMEBANK_CONFIG|TIMEBANK_THEME|DB_DEPLOY_USERNAME|SCOUT_QUEUE_TIMEOUT|PLATFORM_PATH)=' .env
echo "DB_DEPLOY_PASSWORD set: $(grep -c '^DB_DEPLOY_PASSWORD=.' .env)"
echo "== apt (look for GPG or EXPKEYSIG errors)"
apt-get update 2>&1 | grep -iE 'err|gpg|expkeysig' || echo "apt ok"
echo "== worker units: folder and PHP must exist"
for u in /etc/systemd/system/*.service; do
  grep -q "artisan" "$u" 2>/dev/null || continue
  wd=$(grep -oP '^WorkingDirectory=\K.*' "$u"); bin=$(grep -oP '^ExecStart=\K\S+' "$u")
  printf '%-45s %-28s %s %s\n' "$(basename "$u")" "$wd" "$([ -d "$wd" ] && echo ok || echo MISSING)" "$bin"
done
)
```

- [ ] Laravel 10.x, branch `uuro`, PHP 8.2 or newer; tmux installed.
- [ ] Free disk space at least twice `vendor/` plus 2 GB.
- [ ] Local changes: only `public/storage` and `public/build` (and files under them). Anything else: find out why first.
- [ ] No commits that are not on Gitea. If there are, push them to `origin/uuro` first: they stay in the archive.
- [ ] `.env`: `TIMEBANK_CONFIG=uuro`, `TIMEBANK_THEME=uuro`, `DEPLOY_ENVIRONMENT=server` with the other `DEPLOY_*`
  keys; `DB_DEPLOY_PASSWORD set: 1` or the MySQL root password at hand; `SCOUT_QUEUE_TIMEOUT=1800` (add it if
  missing). No `PLATFORM_PATH`: the folder is `platform/` in the application root.
- [ ] `apt ok`. An `EXPKEYSIG` for packages.sury.org: fix it first (guide, Troubleshooting, last row).
- [ ] Every worker unit points at `$APP` and an existing PHP (on Timebank.cc four workers pointed at a removed folder
  for months). Fix a unit before the upgrade (backup first), then `systemctl daemon-reload`.

If `git fetch` asks for the Gitea login every time, cache it for an hour:
`git config --global credential.helper 'cache --timeout=3600'`.

### 3. Build the platform folder (creates `platform/`, changes nothing else)

The move replaces or deletes the files of branch `uuro`, so first copy what makes Uuro Uuro into the platform folder.
The old code ignores the folder; it is used from the moment the new code runs.

```bash
(
cd "${APP:?set APP first}" || exit 1
[ ! -e platform ] || { echo "platform/ exists already: not touched"; exit 1; }
mkdir -p platform/public platform/views/logos platform/lang
cp config/uuro.php                                          platform/config.php
cp resources/views/logos/uuro.blade.php                     platform/views/logos/uuro.blade.php
cp storage/app/public/app-images/app_mail_logo.png          platform/public/mail-logo.png
cp storage/app/public/app-images/app_mail_logo_inverted.png platform/public/mail-logo-inverted.png
cp public/images/og-image.png                               platform/public/og-image.png
for f in favicon.ico favicon.svg favicon-96x96.png apple-touch-icon.png web-app-manifest-192x192.png web-app-manifest-512x512.png; do
    [ -f "public/$f" ] && cp "public/$f" "platform/public/$f"
done
chown -R --reference=config platform
find platform -type f | sort
)
```

- [ ] It lists `config.php`, `views/logos/uuro.blade.php` and `public/` with the mail logos, `og-image.png` and the
  favicons.

### 4. Dry run (changes nothing)

```bash
tmux new -s upgrade
```

```bash
(
cd "${APP:?set APP first}" || exit 1
git fetch origin || { echo "git fetch failed"; exit 1; }
git show origin/laravel-11:scripts/upgrade.sh > /tmp/timebank-upgrade.sh
lines=$(wc -l < /tmp/timebank-upgrade.sh)
[ "$lines" -gt 100 ] || { echo "upgrade.sh did not come through ($lines lines)"; exit 1; }
bash /tmp/timebank-upgrade.sh --dry-run --leave-branch --target origin/laravel-11
)
```

- [ ] `On branch 'uuro', which this upgrade leaves`; the list of commits that leave (37 on 2026-10-19, more if the
  branch got new ones); no warning about unpushed commits.
- [ ] `Tracked files the move deletes` includes `config/uuro.php`, and the script does not refuse.
- [ ] PHP accepted for the command line and the website; enough disk space;
  `Platform folder: ... (config.php lang public views)`.
- [ ] `2. Plan`: `Leaves branch 'uuro' for main`, `Laravel major upgrade (10 -> 11)`, 6 new migrations that add
  tables only (polls).
- [ ] Ends with `Dry run: nothing was changed`.

If the dry run stops, fix the cause today, not on the day itself.

---

## Day of the upgrade

### 5. Start

In tmux, the same block as in step 4 but without `--dry-run`, as a single line (the script asks questions):

```bash
cd "$APP" && git fetch origin && git show origin/laravel-11:scripts/upgrade.sh > /tmp/timebank-upgrade.sh && bash /tmp/timebank-upgrade.sh --leave-branch --target origin/laravel-11
```

Open a second tmux window now (`Ctrl-b c`, then `cd "$APP"`). Switch with `Ctrl-b n`. If the SSH connection drops,
log in again and run `tmux attach -t upgrade`.

### 6. Work through the script

- [ ] **2. Plan**: `Upgrade this installation to <commit>?` → `y`; `Type the branch name 'uuro'` → `uuro`.
- [ ] **3. Backup**: `Make a database backup now?` → `y`. Wait for `Backup verified`.
- [ ] **4. Maintenance**: run the printed `php artisan down --retry=60 --secret=<secret>` in window 2 and **write down
  the secret**. Press Enter in window 1.
- [ ] **5. Deploy**: `Left branch 'uuro' (tag pre-leave-uuro-<stamp>); now on main.`, then `deploy.sh` runs:
  - `Would you like to merge new keys now? (y/N)` → `y`; then `yes` for each file. The new keys for Uuro go into
    `platform/config.php` (65 on 2026-10-19); the footer and other lists stay as they are.
  - MySQL user and password: only without `DB_DEPLOY_*` in `.env`.
  - If deploy.sh fails: the site stays in maintenance. See the guide, Troubleshooting.
  - After it: `public/storage linked to this server's storage.`
- [ ] **6. Version check**: `Laravel Framework 11.x`.
- [ ] **7. Restart**: run the printed restart commands in window 2 (check they report `active (running)`), then press
  Enter in window 1.

### 7. Test, while visitors still see the maintenance page

Open `<APP_URL>/<secret>` once in your browser. That browser now bypasses the maintenance page.

- [ ] The site is in the Uuro theme, with Uuro's logo and favicon.
- [ ] Log in; switch to an organisation profile and to a bank profile, and back.
- [ ] Make a small payment, and check it in the transaction list.
- [ ] Send a chat message with an image.
- [ ] Search for a person and for a post (search was rebuilt during the deploy).
- [ ] Open the admin pages: profiles, statistics, mailings.
- [ ] Request a password reset for an admin or bank profile: it works now (on branch `uuro` it failed).
- [ ] The page source has `<meta property="og:image" content=".../platform/og-image.png">`.
- [ ] In window 2:

  ```bash
  php artisan upgrade:verify
  php artisan platform:check
  tail -n 50 storage/logs/laravel-$(date +%F).log
  ```

  `upgrade:verify` reports `platform config: platform/config.php` and no failures; `platform:check` reports
  `Platform config: platform/config.php` and `linked at /platform` (the logo copied by hand gives an `i` line: fine).

Good to know: queue workers do not run in maintenance mode, so like and star counters (updated by queued jobs) lag
behind during this test. A count that looks one off after liking and unliking is that; it corrects itself once the
site is live (seen on Timebank.cc on 2026-10-19).

### 8. Go or no-go

| Go (`g`) if | No-go (`r`) if |
|---|---|
| every test above worked | log-in, payments or profile switching fail |
| the Uuro theme, logo and favicons show | the site shows Timebank.cc's brand |
| no new errors in the log | new errors appear on every page view |

- **Go**: answer `g`, run `php artisan up` in window 2, press Enter. Check the public site once from a private window.
- **No-go**: answer `r`. The rollback starts right away: back to branch `uuro` with `config/uuro.php` and its other
  files; restoring the database is safe now.

Note the stamp the script prints (`storage/app/upgrades/<stamp>/`).

---

## After going live

### 9. The first hour

- [ ] Watch the log: `tail -f storage/logs/laravel-$(date +%F).log`.
- [ ] `php artisan config:merge --all --dry-run` reports nothing new.
- [ ] Review the new keys in `platform/config.php`: they came from the default template, e.g. the email sign-off
  `platform_sign_off_signature` ("happy Timebanking!").
- [ ] Workers and Reverb are running; the queue drains: `php artisan queue:monitor high,messages,default,emails,mailing,low`.

### 10. The next three days: watch the logs

Once a day, read-only:

```bash
( cd "${APP:?set APP first}"; for d in $(seq 0 3); do f=storage/logs/laravel-$(date -d "-$d day" +%F).log; [ -f $f ] && echo "== $f: $(grep -c '\.ERROR' $f) errors" && grep '\.ERROR' $f | cut -c1-160 | sed -E 's/\[[^]]*\] //' | sort | uniq -c | sort -rn | head -5; done )
```

### 11. After three quiet days: clean up and finish

- [ ] Remove the snapshot of `vendor/`; keep the database backup a little longer:

  ```bash
  ( cd "${APP:?set APP first}"; s=storage/app/upgrades/<stamp>/; ls -lh $s; rm -v ${s}vendor.tar.gz )
  ```

- [ ] Put `platform/` in a private git repository of its own: its history and its off-site copy (the server's
  backups stay on the server). No secrets in it; they stay in `.env`.
- [ ] Keep `origin/uuro` on Gitea as an archive.
- [ ] Developers: with Uuro on Laravel 11, every installation is. Fast-forward `main` to `laravel-11`, so routine
  `./deploy.sh -e server` deploys pull the new version again on both servers. From then on `deploy.sh` stops a
  routine deploy that would raise the Laravel major version (e.g. to 12) and points to `scripts/upgrade.sh`.

---

## Rollback

| When | Command | Database |
|---|---|---|
| Before going live | answer `r` at the go/no-go question | restore offered; safe |
| After going live | `bash storage/app/upgrades/<stamp>/upgrade.sh --rollback` | code only, unless you type `RESTORE` |

The rollback returns to branch `uuro` at its old commit, with `config/uuro.php` and the other files of the branch;
the platform folder stays (the old code ignores it). A database restore after going live deletes everything written
since the backup; usually the code-only rollback is right, because the old code ignores new tables.

After a rollback, do not run `./deploy.sh` until the problem is fixed. Send
`storage/app/upgrades/<stamp>/upgrade.log`, `rollback.log` and `state.env` to the developers.

## Record

| | |
|---|---|
| Date and start time | |
| Previous commit (branch `uuro`) | |
| New commit (branch `main`) | |
| Upgrade stamp and tag `pre-leave-uuro-...` | |
| Live at | |
| Issues and how they were solved | |