New Grimoire
This commit is contained in:
traveler
parent
77d589a13d
commit
cc574f8aed
157 changed files with 29420 additions and 0 deletions
567
Vault-Grimoire/Backups/Wiki-Backup.md
Normal file
567
Vault-Grimoire/Backups/Wiki-Backup.md
Normal file
|
|
@ -0,0 +1,567 @@
|
|||
---
|
||||
title: Wikijs Backup
|
||||
description: Backup Wikijs
|
||||
published: true
|
||||
date: 2026-02-23T04:35:32.870Z
|
||||
tags:
|
||||
editor: markdown
|
||||
dateCreated: 2026-02-23T04:35:24.121Z
|
||||
---
|
||||
|
||||
# Wiki.js Backup & Recovery
|
||||
|
||||
**Service:** Wiki.js (Netgrimoire)
|
||||
**Stack:** Docker Compose — Wiki.js + PostgreSQL
|
||||
**Backup Targets:** PostgreSQL database dump, Git content repository, Docker Compose config
|
||||
**Backup Destinations:** Local vault path → Kopia → offsite vaults
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Wiki.js data lives in two separate places that must be backed up independently:
|
||||
|
||||
**PostgreSQL database** — stores page metadata, navigation, user accounts, permissions, page history, assets, and all configuration. This is the critical component for a portable restore. Without it, a new instance has no knowledge of your wiki structure.
|
||||
|
||||
**Git content repository** — stores the actual page content in markdown files, synced from Forgejo. This is already mirrored on the VAULT SSD at `/vault/repos/wiki/`. It is inherently redundant as long as Forgejo is healthy, but is included in backups for completeness and offline portability.
|
||||
|
||||
**Docker Compose config** — the `docker-compose.yml` and `.env` files needed to recreate the stack.
|
||||
|
||||
---
|
||||
|
||||
## What Gets Backed Up
|
||||
|
||||
| Component | Location | Method | Critical? |
|
||||
|---|---|---|---|
|
||||
| PostgreSQL database | Docker volume | `pg_dump` → SQL file | Yes — primary restore target |
|
||||
| Git content repo | `/vault/repos/wiki/` | Already on VAULT SSD | Yes — page content |
|
||||
| Docker Compose files | `/opt/stacks/wikijs/` | rsync copy | Yes — stack config |
|
||||
| Wiki.js data volume | Docker volume | Optional rsync | No — DB + Git covers this |
|
||||
|
||||
---
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### Tier 1 — Daily Dump to Vault Path
|
||||
|
||||
A script runs daily via systemd timer. It produces a portable `pg_dump` SQL file written to `/vault/backups/wiki/`. These local dumps are retained for 14 days.
|
||||
|
||||
**Key choices:**
|
||||
|
||||
- `--format=plain` — plain SQL, portable to any PostgreSQL version and any host
|
||||
- `--no-owner` — strips role ownership, so the dump restores cleanly on a new instance with a different postgres user (critical for Pocket Grimoire restores)
|
||||
- `--no-acl` — strips GRANT/REVOKE statements for the same reason
|
||||
- No application downtime required — PostgreSQL handles consistent dumps natively
|
||||
|
||||
### Tier 2 — Kopia Snapshot to Offsite Vaults
|
||||
|
||||
After the daily dump completes, Kopia snapshots the entire `/vault/backups/wiki/` directory and replicates to your offsite vaults. Kopia deduplication means only changed blocks are transferred after the first run.
|
||||
|
||||
---
|
||||
|
||||
## Setup
|
||||
|
||||
### Step 0 — Confirm Kopia Repository Exists
|
||||
|
||||
If Kopia is not yet initialized on this host, initialize it first. If you already initialized Kopia for Mailcow or another service, skip this step — all services share the same Kopia repository.
|
||||
|
||||
```bash
|
||||
# Check if repository already exists
|
||||
kopia repository status
|
||||
|
||||
# If not initialized, create it against your vault path
|
||||
kopia repository create filesystem --path=/vault/kopia
|
||||
|
||||
# Connect on subsequent logins if disconnected
|
||||
kopia repository connect filesystem --path=/vault/kopia
|
||||
```
|
||||
|
||||
### Step 1 — Create Backup Directories
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /vault/backups/wiki
|
||||
sudo chown $(whoami):$(whoami) /vault/backups/wiki
|
||||
```
|
||||
|
||||
### Step 2 — Create the Backup Script
|
||||
|
||||
```bash
|
||||
sudo nano /usr/local/sbin/wikijs-backup.sh
|
||||
```
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
# wikijs-backup.sh — Daily Wiki.js backup: pg_dump + git repo + config
|
||||
# Writes to /vault/backups/wiki/, then snapshots with Kopia
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# ── Configuration ─────────────────────────────────────────────────────────────
|
||||
BACKUP_DIR="/vault/backups/wiki"
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
CONTAINER_DB="wikijs_db" # Adjust to your actual container name
|
||||
PG_USER="wikijs"
|
||||
PG_DB="wikijs"
|
||||
WIKI_STACK_DIR="/opt/stacks/wikijs" # Location of docker-compose.yml and .env
|
||||
GIT_REPO_DIR="/vault/repos/wiki" # Git content mirror (already on vault SSD)
|
||||
RETAIN_DAYS=14 # Local dump retention
|
||||
|
||||
LOG="/var/log/wikijs-backup.log"
|
||||
touch "$LOG"
|
||||
|
||||
log() { echo "$(date -Is) $*" | tee -a "$LOG"; }
|
||||
|
||||
# ── Step 1: PostgreSQL dump ────────────────────────────────────────────────────
|
||||
log "Starting Wiki.js PostgreSQL dump..."
|
||||
|
||||
docker exec "$CONTAINER_DB" pg_dump \
|
||||
-U "$PG_USER" \
|
||||
"$PG_DB" \
|
||||
--format=plain \
|
||||
--no-owner \
|
||||
--no-acl \
|
||||
> "${BACKUP_DIR}/wikijs-db-${DATE}.sql"
|
||||
|
||||
gzip "${BACKUP_DIR}/wikijs-db-${DATE}.sql"
|
||||
|
||||
log "PostgreSQL dump complete: wikijs-db-${DATE}.sql.gz"
|
||||
|
||||
# ── Step 2: Docker Compose config backup ──────────────────────────────────────
|
||||
log "Backing up Docker Compose config..."
|
||||
|
||||
CONFIG_BACKUP="${BACKUP_DIR}/wikijs-config-${DATE}.tar.gz"
|
||||
|
||||
tar -czf "$CONFIG_BACKUP" \
|
||||
-C "$(dirname "$WIKI_STACK_DIR")" \
|
||||
"$(basename "$WIKI_STACK_DIR")"
|
||||
|
||||
log "Config backup complete: wikijs-config-${DATE}.tar.gz"
|
||||
|
||||
# ── Step 3: Git repo snapshot (content mirror) ────────────────────────────────
|
||||
# The git repo lives on the VAULT SSD and is already versioned.
|
||||
# We record the current HEAD commit for reference.
|
||||
|
||||
if [ -d "${GIT_REPO_DIR}/.git" ]; then
|
||||
GIT_HEAD=$(git -C "$GIT_REPO_DIR" rev-parse HEAD 2>/dev/null || echo "unknown")
|
||||
echo "Git HEAD at backup time: ${GIT_HEAD}" \
|
||||
> "${BACKUP_DIR}/wikijs-git-ref-${DATE}.txt"
|
||||
log "Git content repo HEAD: ${GIT_HEAD}"
|
||||
else
|
||||
log "WARNING: Git repo not found at ${GIT_REPO_DIR} — skipping git ref"
|
||||
fi
|
||||
|
||||
# ── Step 4: Cleanup old local dumps ───────────────────────────────────────────
|
||||
log "Cleaning up dumps older than ${RETAIN_DAYS} days..."
|
||||
|
||||
find "$BACKUP_DIR" -name "wikijs-db-*.sql.gz" -mtime +"$RETAIN_DAYS" -delete
|
||||
find "$BACKUP_DIR" -name "wikijs-config-*.tar.gz" -mtime +"$RETAIN_DAYS" -delete
|
||||
find "$BACKUP_DIR" -name "wikijs-git-ref-*.txt" -mtime +"$RETAIN_DAYS" -delete
|
||||
|
||||
# ── Step 5: Kopia snapshot ────────────────────────────────────────────────────
|
||||
log "Running Kopia snapshot of /vault/backups/wiki/..."
|
||||
|
||||
kopia snapshot create "$BACKUP_DIR" \
|
||||
--tags "service:wikijs,host:$(hostname -s)"
|
||||
|
||||
log "Kopia snapshot complete."
|
||||
|
||||
# ── Done ──────────────────────────────────────────────────────────────────────
|
||||
log "Wiki.js backup finished successfully."
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo chmod +x /usr/local/sbin/wikijs-backup.sh
|
||||
```
|
||||
|
||||
### Step 3 — Create systemd Service and Timer
|
||||
|
||||
```bash
|
||||
sudo nano /etc/systemd/system/wikijs-backup.service
|
||||
```
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Wiki.js daily backup (pg_dump + config + Kopia snapshot)
|
||||
After=docker.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/sbin/wikijs-backup.sh
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo nano /etc/systemd/system/wikijs-backup.timer
|
||||
```
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Run Wiki.js backup daily at 02:00
|
||||
|
||||
[Timer]
|
||||
OnCalendar=*-*-* 02:00:00
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable wikijs-backup.timer
|
||||
sudo systemctl start wikijs-backup.timer
|
||||
|
||||
# Verify
|
||||
systemctl list-timers | grep wikijs
|
||||
```
|
||||
|
||||
### Step 4 — Configure Kopia Retention Policy
|
||||
|
||||
```bash
|
||||
# Set retention policy for wiki backups
|
||||
kopia policy set /vault/backups/wiki \
|
||||
--keep-daily 14 \
|
||||
--keep-weekly 8 \
|
||||
--keep-monthly 12 \
|
||||
--compression zstd
|
||||
|
||||
# Verify policy
|
||||
kopia policy show /vault/backups/wiki
|
||||
```
|
||||
|
||||
### Step 5 — Test the Backup
|
||||
|
||||
```bash
|
||||
# Run manually first time
|
||||
sudo /usr/local/sbin/wikijs-backup.sh
|
||||
|
||||
# Verify output
|
||||
ls -lh /vault/backups/wiki/
|
||||
# Should show: wikijs-db-YYYYMMDD_HHMMSS.sql.gz
|
||||
# wikijs-config-YYYYMMDD_HHMMSS.tar.gz
|
||||
# wikijs-git-ref-YYYYMMDD_HHMMSS.txt
|
||||
|
||||
# Verify Kopia snapshot was created
|
||||
kopia snapshot list /vault/backups/wiki
|
||||
|
||||
# Check backup log
|
||||
tail -n 30 /var/log/wikijs-backup.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verifying Backups
|
||||
|
||||
### Check dump is readable
|
||||
|
||||
```bash
|
||||
# Inspect the SQL dump without extracting
|
||||
zcat /vault/backups/wiki/wikijs-db-YYYYMMDD_HHMMSS.sql.gz | head -50
|
||||
|
||||
# Should show PostgreSQL header, version info, and CREATE TABLE statements
|
||||
```
|
||||
|
||||
### Verify Kopia snapshots
|
||||
|
||||
```bash
|
||||
# List recent snapshots
|
||||
kopia snapshot list /vault/backups/wiki
|
||||
|
||||
# Show snapshot details
|
||||
kopia snapshot list /vault/backups/wiki --all
|
||||
|
||||
# Verify snapshot integrity
|
||||
kopia snapshot verify
|
||||
```
|
||||
|
||||
### Test restore to a temporary database (non-destructive)
|
||||
|
||||
```bash
|
||||
# Start a temporary Postgres container
|
||||
docker run --rm -d \
|
||||
--name wikijs-restore-test \
|
||||
-e POSTGRES_USER=wikijs \
|
||||
-e POSTGRES_PASSWORD=testpassword \
|
||||
-e POSTGRES_DB=wikijs_test \
|
||||
postgres:16-alpine
|
||||
|
||||
# Wait for Postgres to be ready
|
||||
sleep 5
|
||||
|
||||
# Restore dump into test container
|
||||
zcat /vault/backups/wiki/wikijs-db-YYYYMMDD_HHMMSS.sql.gz | \
|
||||
docker exec -i wikijs-restore-test psql -U wikijs -d wikijs_test
|
||||
|
||||
# Verify tables exist
|
||||
docker exec wikijs-restore-test psql -U wikijs -d wikijs_test -c "\dt"
|
||||
|
||||
# Expected output: List of tables (pages, users, pageHistory, assets, etc.)
|
||||
|
||||
# Cleanup test container
|
||||
docker stop wikijs-restore-test
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recovery Procedures
|
||||
|
||||
### Scenario A — Restore to a New Wiki.js Instance (Any Host)
|
||||
|
||||
This covers full disaster recovery to a fresh server, including Pocket Grimoire.
|
||||
|
||||
**Requirements on the destination host:**
|
||||
- Docker and Docker Compose installed
|
||||
- A `docker-compose.yml` and `.env` ready (from backup or Pocket Grimoire stack)
|
||||
- Sufficient disk space
|
||||
|
||||
**Step 1: Locate the backup**
|
||||
|
||||
```bash
|
||||
# On Netgrimoire, find the dump to restore
|
||||
ls -lh /vault/backups/wiki/
|
||||
|
||||
# Or restore from Kopia
|
||||
kopia snapshot list /vault/backups/wiki
|
||||
kopia restore SNAPSHOT_ID /tmp/wiki-restore/
|
||||
ls /tmp/wiki-restore/
|
||||
```
|
||||
|
||||
**Step 2: Copy dump to the destination host**
|
||||
|
||||
```bash
|
||||
# From Netgrimoire, copy to the destination server
|
||||
scp /vault/backups/wiki/wikijs-db-YYYYMMDD_HHMMSS.sql.gz \
|
||||
user@destination-host:/tmp/
|
||||
|
||||
# Or to Pocket Grimoire
|
||||
scp /vault/backups/wiki/wikijs-db-YYYYMMDD_HHMMSS.sql.gz \
|
||||
user@pocket-grimoire.local:/tmp/
|
||||
```
|
||||
|
||||
**Step 3: Start the database container only**
|
||||
|
||||
On the destination host, start just the database — do not start Wiki.js yet:
|
||||
|
||||
```bash
|
||||
cd /srv/pocket-grimoire/stacks/wikijs # Adjust path as needed
|
||||
|
||||
# Start only the database container
|
||||
docker compose up -d db
|
||||
|
||||
# Wait for healthy status
|
||||
docker compose ps
|
||||
# db should show: healthy
|
||||
```
|
||||
|
||||
**Step 4: Restore the dump**
|
||||
|
||||
```bash
|
||||
# Restore the dump into the running database container
|
||||
zcat /tmp/wikijs-db-YYYYMMDD_HHMMSS.sql.gz | \
|
||||
docker exec -i pocketgrimoire_db psql \
|
||||
-U wikijs \
|
||||
-d wikijs
|
||||
|
||||
# Verify tables restored
|
||||
docker exec pocketgrimoire_db psql -U wikijs -d wikijs -c "\dt"
|
||||
```
|
||||
|
||||
**Step 5: Start Wiki.js**
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
|
||||
# Watch startup logs
|
||||
docker logs -f pocketgrimoire_wikijs
|
||||
# Wait for: "HTTP Server started successfully"
|
||||
```
|
||||
|
||||
**Step 6: Verify**
|
||||
|
||||
Open `http://pocket-grimoire.local:3000` and confirm:
|
||||
- Pages load correctly
|
||||
- Navigation structure is intact
|
||||
- User accounts are present (if you had multiple users)
|
||||
|
||||
**Step 7: Re-sync Git content (if needed)**
|
||||
|
||||
The database knows the page structure, but if the Git content repo isn't present on the new host, import it:
|
||||
|
||||
```bash
|
||||
# In Wiki.js admin panel:
|
||||
# Administration → Storage → Git
|
||||
# Click "Force Sync" or "Import Content"
|
||||
|
||||
# Or copy the repo from VAULT SSD
|
||||
rsync -avP /vault/repos/wiki/ /srv/pocket-grimoire/repos/wiki/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Scenario B — Restore on Existing Netgrimoire Instance
|
||||
|
||||
Use this when the Wiki.js database is corrupted but the host is otherwise healthy.
|
||||
|
||||
**Step 1: Stop Wiki.js (leave database running)**
|
||||
|
||||
```bash
|
||||
cd /opt/stacks/wikijs
|
||||
docker compose stop wikijs
|
||||
```
|
||||
|
||||
**Step 2: Drop and recreate the database**
|
||||
|
||||
```bash
|
||||
docker exec -it wikijs_db psql -U postgres -c "DROP DATABASE wikijs;"
|
||||
docker exec -it wikijs_db psql -U postgres -c "CREATE DATABASE wikijs OWNER wikijs;"
|
||||
```
|
||||
|
||||
**Step 3: Restore**
|
||||
|
||||
```bash
|
||||
zcat /vault/backups/wiki/wikijs-db-YYYYMMDD_HHMMSS.sql.gz | \
|
||||
docker exec -i wikijs_db psql -U wikijs -d wikijs
|
||||
```
|
||||
|
||||
**Step 4: Restart Wiki.js**
|
||||
|
||||
```bash
|
||||
docker compose start wikijs
|
||||
docker logs -f wikijs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Scenario C — Restore Config Only
|
||||
|
||||
If the stack config was lost but the database volume is intact:
|
||||
|
||||
```bash
|
||||
# Extract config from backup
|
||||
tar -xzf /vault/backups/wiki/wikijs-config-YYYYMMDD_HHMMSS.tar.gz \
|
||||
-C /opt/stacks/
|
||||
|
||||
# Verify
|
||||
ls /opt/stacks/wikijs/
|
||||
# Should show: docker-compose.yml .env
|
||||
|
||||
# Restart stack
|
||||
cd /opt/stacks/wikijs
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Restore from Kopia (Offsite)
|
||||
|
||||
When local vault files are unavailable, restore the backup directory from Kopia first:
|
||||
|
||||
```bash
|
||||
# List available snapshots
|
||||
kopia snapshot list /vault/backups/wiki
|
||||
|
||||
# Restore snapshot to temp directory
|
||||
kopia restore SNAPSHOT_ID /tmp/wiki-restore/
|
||||
|
||||
# Then proceed with the appropriate scenario above
|
||||
# using files from /tmp/wiki-restore/ instead of /vault/backups/wiki/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pocket Grimoire Specifics
|
||||
|
||||
When restoring to Pocket Grimoire, note the following differences from a full Netgrimoire instance:
|
||||
|
||||
**Container names** differ — use `pocketgrimoire_db` instead of `wikijs_db`.
|
||||
|
||||
**Stack path** is `/srv/pocket-grimoire/stacks/wikijs/` instead of `/opt/stacks/wikijs/`.
|
||||
|
||||
**The database is already initialized** when Pocket Grimoire is first set up. Restoring a Netgrimoire dump overwrites it entirely, which is the intended behavior — Pocket Grimoire becomes a mirror of Netgrimoire's wiki state.
|
||||
|
||||
**Git content repo** is located at `/srv/pocket-grimoire/repos/wiki/` and is populated via the sync script (`pocketgrimoire-sync.sh`). A database restore alone is sufficient if the Git repo is already in place.
|
||||
|
||||
**Recommended restore workflow for Pocket Grimoire:**
|
||||
|
||||
```bash
|
||||
# 1. Copy dump from VAULT SSD (already available on Pocket Grimoire)
|
||||
ls /srv/vaultpg/backups/wiki/
|
||||
|
||||
# 2. Start db container only
|
||||
cd /srv/pocket-grimoire/stacks/wikijs && docker compose up -d db
|
||||
|
||||
# 3. Restore
|
||||
zcat /srv/vaultpg/backups/wiki/wikijs-db-LATEST.sql.gz | \
|
||||
docker exec -i pocketgrimoire_db psql -U wikijs -d wikijs
|
||||
|
||||
# 4. Start full stack
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Because the VAULT SSD is always connected to Pocket Grimoire, no file transfer is needed — the dumps are already there.
|
||||
|
||||
---
|
||||
|
||||
## Monitoring & Alerts
|
||||
|
||||
Add the following to your existing ntfy/monitoring setup to alert on backup failures. Wrap the backup script call in an error trap:
|
||||
|
||||
```bash
|
||||
# Add to wikijs-backup.sh after set -euo pipefail:
|
||||
|
||||
NTFY_URL="https://ntfy.YOUR_DOMAIN/wikijs-backup"
|
||||
|
||||
on_error() {
|
||||
curl -fsS -X POST "$NTFY_URL" \
|
||||
-H "Title: Wiki.js backup FAILED ($(hostname -s))" \
|
||||
-H "Priority: high" \
|
||||
-H "Tags: rotating_light" \
|
||||
-d "Backup failed at $(date -Is). Check /var/log/wikijs-backup.log"
|
||||
}
|
||||
trap on_error ERR
|
||||
```
|
||||
|
||||
### Check backup age manually
|
||||
|
||||
```bash
|
||||
# Find most recent dump
|
||||
ls -lt /vault/backups/wiki/wikijs-db-*.sql.gz | head -3
|
||||
|
||||
# Check Kopia last snapshot time
|
||||
kopia snapshot list /vault/backups/wiki | tail -5
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```bash
|
||||
# Run backup manually
|
||||
sudo /usr/local/sbin/wikijs-backup.sh
|
||||
|
||||
# Watch backup log
|
||||
tail -f /var/log/wikijs-backup.log
|
||||
|
||||
# Check timer status
|
||||
systemctl status wikijs-backup.timer
|
||||
|
||||
# List local dumps
|
||||
ls -lh /vault/backups/wiki/
|
||||
|
||||
# List Kopia snapshots
|
||||
kopia snapshot list /vault/backups/wiki
|
||||
|
||||
# Restore dump (generic)
|
||||
zcat /vault/backups/wiki/wikijs-db-YYYYMMDD_HHMMSS.sql.gz | \
|
||||
docker exec -i CONTAINER_NAME psql -U wikijs -d wikijs
|
||||
|
||||
# Test dump is readable
|
||||
zcat /vault/backups/wiki/wikijs-db-YYYYMMDD_HHMMSS.sql.gz | head -50
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Revision History
|
||||
|
||||
| Version | Date | Notes |
|
||||
|---|---|---|
|
||||
| 1.0 | 2026-02-22 | Initial release — pg_dump + Kopia + Pocket Grimoire restore procedures |
|
||||
Loading…
Add table
Add a link
Reference in a new issue