traveler/Netgrimoire
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: update Netgrimoire/Pocket/Deployment_Guide

Browse source
This commit is contained in:
Administrator 2026-02-21 06:10:28 +00:00 • committed by John Smith
parent 9327335fb8
commit 61e402e735
1 changed files with 263 additions and 105 deletions
Download patch file Download diff file Expand all files Collapse all files

368
Netgrimoire/Pocket/Deployment_Guide.md
View file

@ -2,7 +2,7 @@
title: Pocket Grimoire
description:
published: true
date: 2026-02-21T04:25:58.904Z
date: 2026-02-21T06:10:17.060Z
tags:
editor: markdown
dateCreated: 2026-02-20T04:41:35.122Z
@ -44,14 +44,29 @@ Pocket Grimoire is a portable companion to the Netgrimoire homelab, providing of
### Storage (3 SSDs, 2 Active at a Time)
- **SSD #1 – VAULT** (1-2TB, encrypted, always connected)
- Wiki.js data, PostgreSQL, git repos, photos, documents, backups
- **SSD #2 – MEDIA-PERSONAL** (2TB+, encrypted)
- Your curated H.264/AAC movies and TV shows
- Connected for personal trips
- **SSD #3 – MEDIA-FAMILY** (2TB+, unencrypted)
- Git repository mirrors (from Forgejo)
- Wiki.js content backups
- Kopia repository (backup data)
- Photos and documents
- System backups and configs
- SSH keys
- VeraCrypt containers (optional)
- **Does NOT contain media or Stash data**
- **SSD #2 – GREEN** (2TB+, encrypted, for personal trips)
- Personal media library (H.264/AAC movies and TV)
- Stash-Pocket data (database, previews, blobs)
- Personal content organized under `/Green/` structure
- Connected during personal/solo trips
- Syncs from `/export/vault/Green/` on Netgrimoire
- **SSD #3 – MEDIA-FAMILY** (2TB+, unencrypted, for family trips)
- Family-friendly movies and TV shows (H.264/AAC)
- Connected for family visits
- Shareable/portable to other devices without Pocket Grimoire
- Simple `/library/movies/` and `/library/tv/` structure
- Connected during family visits/trips
- Unencrypted for easy sharing with relatives
- Can be used on other devices without Pocket Grimoire
- **USB drive – ISO/Rebuild** (64GB+, labeled, write-protected)
- **USB drive – Data Transfer** (128GB+, labeled)
@ -237,7 +252,7 @@ Headroom: 78W
## Directory Structure
```
/srv/pocket-grimoire/ # Main application root (on Vault SSD)
/srv/pocket-grimoire/ # Main application root (on VAULT SSD)
├── stacks/ # Docker Compose files
│ ├── wikijs/
│ │ ├── docker-compose.yml
@ -245,6 +260,9 @@ Headroom: 78W
│ ├── jellyfin/
│ │ ├── docker-compose.yml
│ │ └── .env
│ ├── stash/
│ │ ├── docker-compose.yml
│ │ └── .env
│ └── filebrowser/ # Optional
│ └── docker-compose.yml
├── data/ # Persistent container data
@ -258,23 +276,38 @@ Headroom: 78W
├── forgejo_wiki_ro # Read-only wiki deploy key
└── zfs_pull_ro # ZFS replication key
/srv/vaultpg/ # Vault SSD ZFS mount
├── veracrypt-containers/ # VeraCrypt container files (optional)
│ └── vault.vc # Encrypted container
└── (other mirrors from Netgrimoire)
/srv/vaultpg/ # VAULT SSD (always connected)
├── kopia/ # Kopia backup repository
├── backups/ # System backups
│ ├── wiki/ # Wiki.js backups
│ ├── photos/ # Photo backups
│ └── documents/ # Document backups
├── repos/ # Git repository mirrors
└── veracrypt-containers/ # VeraCrypt containers (optional)
└── vault.vc
/srv/greenpg/ # GREEN SSD (personal, rotated)
├── media/library/ # Personal media files
│ ├── movies/
│ └── tv/
└── stash/ # Stash-Pocket data
├── config/ # Stash database
├── generated/ # Previews
└── blobs/ # Scene markers
/srv/mediapg/ # MEDIA-FAMILY SSD (family, rotated)
└── library/ # Family media files
├── movies/
└── tv/
/mnt/veracrypt/ # VeraCrypt mount points (optional)
├── vault1/ # Mounted container 1
└── vault2/ # Mounted container 2 (if needed)
/srv/mediapg/ # Media SSD ZFS mount (rotated)
└── library/ # H.264 encoded media
├── movies/
└── tv/
/usr/local/sbin/ # System scripts
├── pocketgrimoire-sync.sh # Main sync script
├── pocketgrimoire-zfs-pull.sh # ZFS replication script
├── unlock-pocket-grimoire.sh # Headless unlock script
└── mount-veracrypt-vault.sh # VeraCrypt mount script (optional)
/etc/ # Config files
@ -512,11 +545,28 @@ This approach allows you to:
- Verify data integrity before moving drives
- Test encryption/unlock on powerful hardware first
#### Drive Configuration Overview
**Drive #1: VAULT** (1-2TB, encrypted, always connected)
- Purpose: Backups and system data ONLY
- Contains: Git repos, Wiki backups, Kopia repository, photos, documents
- Does NOT contain media or Stash data
**Drive #2: GREEN** (2TB+, encrypted, rotated for personal trips)
- Purpose: Personal media and Stash-Pocket data
- Contains: Personal media library, Stash database/previews/blobs
- Syncs from `/export/vault/Green/` on Netgrimoire
**Drive #3: MEDIA-FAMILY** (2TB+, unencrypted, rotated for family trips)
- Purpose: Family-friendly shareable content
- Contains: Simple library structure with movies and TV
- Unencrypted for easy sharing with relatives
#### On Netgrimoire: Create and Populate Drives
**Connect drives to Netgrimoire:**
- VAULT SSD (1-2TB) via USB 3.0 or SATA
- MEDIA-PERSONAL SSD (2TB+) via USB 3.0 or SATA
- GREEN SSD (2TB+) via USB 3.0 or SATA
- MEDIA-FAMILY SSD (2TB+) via USB 3.0 or SATA (optional, can be created later)
**Identify drives:**
@ -526,7 +576,7 @@ lsblk
# Note device names: /dev/sdX, /dev/sdY, /dev/sdZ
```
**Create VAULT pool (encrypted):**
**Create VAULT pool (encrypted - backups only):**
```bash
# On Netgrimoire
sudo zpool create -o ashift=12 \
@ -542,21 +592,21 @@ sudo zpool create -o ashift=12 \
# Enter STRONG passphrase when prompted
# Write down this passphrase - you'll need it on Pocket Grimoire
# Create datasets matching Pocket structure
sudo zfs create -o recordsize=16K pocket-vault/wiki-pg
sudo zfs create pocket-vault/repos
sudo zfs create pocket-vault/pocket-grimoire
sudo zfs create pocket-vault/Green
sudo zfs create pocket-vault/Green/Pocket
sudo zfs create pocket-vault/Green/Pocket/stash
sudo zfs create pocket-vault/Green/Pocket/media
sudo zfs create pocket-vault/veracrypt-containers
# Create datasets for backups and system data
sudo zfs create -o recordsize=16K pocket-vault/wiki-pg # PostgreSQL backups
sudo zfs create pocket-vault/repos # Git repository mirrors
sudo zfs create pocket-vault/kopia # Kopia backup repository
sudo zfs create pocket-vault/backups # General backups
sudo zfs create pocket-vault/backups/wiki # Wiki.js backups
sudo zfs create pocket-vault/backups/photos # Photo backups
sudo zfs create pocket-vault/backups/documents # Document backups
sudo zfs create pocket-vault/veracrypt-containers # VeraCrypt (optional)
# Set ownership
sudo chown -R 1000:1000 /mnt/pocket-vault
```
**Create MEDIA-PERSONAL pool (encrypted):**
**Create GREEN pool (encrypted - personal media + Stash):**
```bash
# On Netgrimoire
sudo zpool create -o ashift=12 \
@ -566,78 +616,120 @@ sudo zpool create -o ashift=12 \
-O compression=lz4 \
-O atime=off \
-O recordsize=1M \
-m /mnt/pocket-media-personal \
pocket-media-personal /dev/sdY
-m /mnt/pocket-green \
pocket-green /dev/sdY
# Enter STRONG passphrase (can be different from VAULT)
# Write down this passphrase
# Create datasets
sudo zfs create pocket-media-personal/library
sudo zfs create pocket-media-personal/library/movies
sudo zfs create pocket-media-personal/library/tv
# Create datasets matching Netgrimoire structure
sudo zfs create pocket-green/media # Media library root
sudo zfs create pocket-green/media/library # Library directory
sudo zfs create pocket-green/media/library/movies # Movies
sudo zfs create pocket-green/media/library/tv # TV shows
sudo zfs create pocket-green/stash # Stash-Pocket data
sudo zfs create pocket-green/stash/config # Stash database
sudo zfs create pocket-green/stash/generated # Previews
sudo zfs create pocket-green/stash/blobs # Scene markers
sudo zfs create pocket-green/stash/cache # Cache (don't sync)
# Set ownership
sudo chown -R 1000:1000 /mnt/pocket-media-personal
sudo chown -R 1000:1000 /mnt/pocket-green
```
**Create MEDIA-FAMILY pool (unencrypted, optional now or later):**
**Create MEDIA-FAMILY pool (unencrypted - family content):**
```bash
# On Netgrimoire
sudo zpool create -o ashift=12 \
-O compression=lz4 \
-O atime=off \
-O recordsize=1M \
-m /mnt/pocket-media-family \
pocket-media-family /dev/sdZ
-m /mnt/pocket-media \
pocket-media /dev/sdZ
# No encryption - family can use this drive on any system
# Create datasets
sudo zfs create pocket-media-family/library
sudo zfs create pocket-media-family/library/movies
sudo zfs create pocket-media-family/library/tv
# Create simple library structure
sudo zfs create pocket-media/library
sudo zfs create pocket-media/library/movies
sudo zfs create pocket-media/library/tv
# Set ownership
sudo chown -R 1000:1000 /mnt/pocket-media-family
sudo chown -R 1000:1000 /mnt/pocket-media
```
**Perform initial sync to VAULT:**
```bash
# On Netgrimoire
# Sync data from Netgrimoire to Pocket VAULT drive
# Sync backups and system data to VAULT drive
# Sync Pocket directory structure
sudo rsync -avP --exclude='cache' \
/export/vault/Green/Pocket/ \
/mnt/pocket-vault/Green/Pocket/
# Sync Wiki backups
sudo rsync -avP \
/export/vault/wiki-backups/ \
/mnt/pocket-vault/backups/wiki/
# Or use ZFS send/receive for atomic snapshot
sudo zfs snapshot vault/Green/Pocket@initial
sudo zfs send vault/Green/Pocket@initial | \
sudo zfs receive pocket-vault/Green/Pocket
# Sync Git repositories
sudo rsync -avP \
/export/vault/repos/ \
/mnt/pocket-vault/repos/
# Sync Kopia repository (if exists)
sudo rsync -avP \
/export/vault/kopia/ \
/mnt/pocket-vault/kopia/
# Sync photos and documents
sudo rsync -avP \
/export/vault/photos/ \
/mnt/pocket-vault/backups/photos/
sudo rsync -avP \
/export/vault/documents/ \
/mnt/pocket-vault/backups/documents/
# Verify data
ls -lh /mnt/pocket-vault/Green/Pocket/
du -sh /mnt/pocket-vault/Green/Pocket/
ls -lh /mnt/pocket-vault/
du -sh /mnt/pocket-vault/
```
**Populate MEDIA-PERSONAL (optional - curate content):**
**Perform initial sync to GREEN:**
```bash
# On Netgrimoire
# Copy curated personal media to Pocket media drive
# Sync personal media and Stash data from /export/vault/Green/
# Example: Copy favorite movies
sudo cp /export/vault/media/library/movies/favorites/*.mp4 \
/mnt/pocket-media-personal/library/movies/
# Sync entire Green directory structure
sudo rsync -avP --exclude='cache' \
/export/vault/Green/ \
/mnt/pocket-green/
# Or use ZFS send/receive for atomic snapshot (if Green is a ZFS dataset)
sudo zfs snapshot vault/Green@initial
sudo zfs send vault/Green@initial | \
sudo zfs receive pocket-green/Green
# Verify data
ls -lh /mnt/pocket-green/
ls -lh /mnt/pocket-green/media/library/
ls -lh /mnt/pocket-green/stash/
du -sh /mnt/pocket-green/
```
**Populate MEDIA-FAMILY (optional - curate family content):**
```bash
# On Netgrimoire
# Copy family-friendly media to MEDIA-FAMILY drive
# Example: Copy family movies
sudo cp /export/vault/media/family-movies/*.mp4 \
/mnt/pocket-media/library/movies/
# Or use rsync for large transfers
sudo rsync -avP \
/export/vault/media/library/tv/FavoriteShow/ \
/mnt/pocket-media-personal/library/tv/FavoriteShow/
/export/vault/media/family-shows/ \
/mnt/pocket-media/library/tv/
# Verify
du -sh /mnt/pocket-media-personal/library/
du -sh /mnt/pocket-media/library/
```
**Export pools before disconnecting:**
@ -646,8 +738,8 @@ du -sh /mnt/pocket-media-personal/library/
# CRITICAL: Export pools before physically disconnecting drives
sudo zpool export pocket-vault
sudo zpool export pocket-media-personal
sudo zpool export pocket-media-family # if created
sudo zpool export pocket-green
sudo zpool export pocket-media # if created
# Verify exported
zpool list
@ -659,8 +751,10 @@ zpool list
### 5. Configure ZFS Pools on Pocket Grimoire
**Now connect drives to Pocket Grimoire:**
- VAULT → Anker USB-A port #2
- MEDIA-PERSONAL or MEDIA-FAMILY → Raspberry Pi USB 3.0 port
- VAULT → Anker USB-A port #2 (always connected)
- GREEN (for personal trips) → Raspberry Pi USB 3.0 port
OR
- MEDIA-FAMILY (for family trips) → Raspberry Pi USB 3.0 port
**Import and rename pools:**
```bash
@ -670,25 +764,29 @@ ssh user@pocket-grimoire.local
# Import VAULT pool with new name
sudo zpool import pocket-vault vaultpg
# Import MEDIA pool with new name
sudo zpool import pocket-media-personal mediapg
# Or:
# sudo zpool import pocket-media-family mediapg
# Import GREEN pool with new name (for personal trips)
sudo zpool import pocket-green greenpg
# OR import MEDIA-FAMILY pool (for family trips)
# sudo zpool import pocket-media mediapg
# Verify pools imported
zpool list
# Should show: vaultpg, mediapg
# Should show: vaultpg, greenpg (or mediapg for family)
```
**Set mount points for Pocket Grimoire:**
```bash
# Set proper mount points
sudo zfs set mountpoint=/srv/vaultpg vaultpg
sudo zfs set mountpoint=/srv/mediapg mediapg
sudo zfs set mountpoint=/srv/greenpg greenpg
# Or for family drive:
# sudo zfs set mountpoint=/srv/mediapg mediapg
# Create mount points
sudo mkdir -p /srv/vaultpg
sudo mkdir -p /srv/mediapg
sudo mkdir -p /srv/greenpg
sudo mkdir -p /srv/mediapg # Create both, use as needed
# Unmount and remount with new paths
sudo zfs unmount -a
@ -698,11 +796,14 @@ sudo zfs mount -a
df -h | grep srv
# Should show:
# vaultpg mounted on /srv/vaultpg
# mediapg mounted on /srv/mediapg
# greenpg mounted on /srv/greenpg (or mediapg on /srv/mediapg)
# Verify data
ls /srv/vaultpg/Green/Pocket/
ls /srv/mediapg/library/
ls /srv/vaultpg/
ls /srv/greenpg/media/library/
ls /srv/greenpg/stash/
# Or for family:
# ls /srv/mediapg/library/
```
**Configure for headless unlock:**
@ -711,7 +812,8 @@ ls /srv/mediapg/library/
# This prevents boot hanging waiting for passphrase
sudo zfs set canmount=noauto vaultpg
sudo zfs set canmount=noauto mediapg
sudo zfs set canmount=noauto greenpg
sudo zfs set canmount=noauto mediapg # For when you swap to family drive
# Pools will need manual unlock via SSH after boot
```
@ -748,9 +850,9 @@ echo " Pocket Grimoire ZFS Unlock (Headless)"
echo "=========================================="
echo
# Check if pools are already unlocked
# Check if VAULT pool is already unlocked
if zfs list vaultpg &>/dev/null && mount | grep -q /srv/vaultpg; then
echo "✓ vaultpg already unlocked and mounted"
echo "✓ vaultpg (VAULT) already unlocked and mounted"
else
# Import pool if needed
if ! zpool list vaultpg &>/dev/null; then
@ -758,8 +860,8 @@ else
sudo zpool import vaultpg
fi
# Unlock Vault pool
echo "Unlocking vaultpg (VAULT SSD)..."
# Unlock VAULT pool
echo "Unlocking vaultpg (VAULT - backups and system data)..."
sudo zfs load-key vaultpg
# Mount all vaultpg datasets
@ -776,14 +878,37 @@ fi
echo
# Check if media pool is present and unlock
# Check for GREEN pool (personal media + Stash)
if zpool list greenpg &>/dev/null; then
if zfs list greenpg &>/dev/null && mount | grep -q /srv/greenpg; then
echo "✓ greenpg (GREEN - personal media + Stash) already unlocked"
else
echo "Unlocking greenpg (GREEN - personal media + Stash)..."
sudo zfs load-key greenpg
sudo zfs mount greenpg
sudo zfs mount -a
if mount | grep -q /srv/greenpg; then
echo "✓ greenpg unlocked and mounted at /srv/greenpg"
else
echo "✗ Failed to mount greenpg"
exit 1
fi
fi
else
echo "ℹ greenpg pool not found (GREEN drive not connected)"
fi
echo
# Check for MEDIA-FAMILY pool (family content)
if zpool list mediapg &>/dev/null; then
if zfs list mediapg &>/dev/null && mount | grep -q /srv/mediapg; then
echo "✓ mediapg already unlocked and mounted"
echo "✓ mediapg (MEDIA-FAMILY) already unlocked"
else
echo "Unlocking mediapg (MEDIA-PERSONAL or MEDIA-FAMILY)..."
echo "Unlocking mediapg (MEDIA-FAMILY - family content)..."
# Check if encrypted (MEDIA-PERSONAL) or not (MEDIA-FAMILY)
# Check if encrypted (shouldn't be, but check anyway)
if zfs get encryption mediapg | grep -q "encryption.*on"; then
sudo zfs load-key mediapg
fi
@ -799,7 +924,7 @@ if zpool list mediapg &>/dev/null; then
fi
fi
else
echo "ℹ mediapg pool not found (media drive may not be connected)"
echo "ℹ mediapg pool not found (MEDIA-FAMILY drive not connected)"
fi
echo
@ -849,6 +974,17 @@ echo "=========================================="
echo " Pocket Grimoire Ready!"
echo "=========================================="
echo
echo "Drives mounted:"
if mount | grep -q /srv/vaultpg; then
echo " ✓ VAULT (vaultpg) at /srv/vaultpg"
fi
if mount | grep -q /srv/greenpg; then
echo " ✓ GREEN (greenpg) at /srv/greenpg - Personal media + Stash"
fi
if mount | grep -q /srv/mediapg; then
echo " ✓ MEDIA-FAMILY (mediapg) at /srv/mediapg - Family content"
fi
echo
echo "Services available at:"
echo " Wiki.js: http://pocket-grimoire.local:3000"
echo " Jellyfin: http://pocket-grimoire.local:8096"
@ -945,17 +1081,21 @@ ssh user@pocket-grimoire.local
# Import pools if needed
sudo zpool import vaultpg
sudo zpool import mediapg
sudo zpool import greenpg # For GREEN (personal)
# Or:
# sudo zpool import mediapg # For MEDIA-FAMILY
# Load encryption keys
sudo zfs load-key vaultpg
sudo zfs load-key mediapg # Only if MEDIA-PERSONAL (encrypted)
sudo zfs load-key vaultpg # VAULT (always encrypted)
sudo zfs load-key greenpg # GREEN (encrypted)
# mediapg is unencrypted (MEDIA-FAMILY) - no key needed
# Mount all datasets
sudo zfs mount -a
# Verify mounted
df -h | grep srv
# Should show vaultpg and either greenpg or mediapg
# Start Docker
sudo systemctl start docker
@ -1717,8 +1857,8 @@ sudo umount /mnt/pocket-media
### 8. Document Passphrases
- [ ] ZFS encryption passphrases (written down, secured)
- VAULT (vaultpg): [write passphrase on paper]
- MEDIA-PERSONAL (mediapg): [write passphrase on paper]
- MEDIA-FAMILY: N/A (unencrypted)
- GREEN (greenpg): [write passphrase on paper]
- MEDIA-FAMILY (mediapg): N/A (unencrypted)
- [ ] VeraCrypt container passwords (if using, written down, secured)
- [ ] WiFi credentials for travel router (portapotty network)
- [ ] Jellyfin admin password
@ -1742,8 +1882,9 @@ ssh user@pocket-grimoire.local
/usr/local/sbin/unlock-pocket-grimoire.sh
# 5. Enter passphrases when prompted
# - VAULT passphrase
# - MEDIA-PERSONAL passphrase (if encrypted)
# - VAULT passphrase (always)
# - GREEN passphrase (if GREEN drive connected for personal trip)
# - MEDIA-FAMILY has no passphrase (unencrypted)
# - VeraCrypt password (if applicable)
# 6. Wait for Docker containers to start (~30 seconds)
@ -1765,20 +1906,26 @@ docker ps
### 10. Verify Data Synced from Netgrimoire
```bash
# Check Vault data present
ls /srv/vaultpg/Green/Pocket/stash/
ls /srv/vaultpg/Green/Pocket/media/
du -sh /srv/vaultpg/Green/Pocket/
# Check VAULT data present (backups only)
ls /srv/vaultpg/kopia/
ls /srv/vaultpg/backups/
ls /srv/vaultpg/repos/
du -sh /srv/vaultpg/
# Check media drive populated (if applicable)
# Check GREEN data (personal media + Stash)
ls /srv/greenpg/media/library/
ls /srv/greenpg/stash/
du -sh /srv/greenpg/
# Or check MEDIA-FAMILY data (if that drive is connected)
ls /srv/mediapg/library/
du -sh /srv/mediapg/
# Verify Stash database and previews
ls -lh /srv/vaultpg/Green/Pocket/stash/config/
# Verify Stash database and previews (only on GREEN)
ls -lh /srv/greenpg/stash/config/
# Should show: stash-go.sqlite
ls /srv/vaultpg/Green/Pocket/stash/generated/ | wc -l
ls /srv/greenpg/stash/generated/ | wc -l
# Should show: hundreds of preview files
```
@ -2178,12 +2325,23 @@ veracrypt --text --list
# Unmount and export ZFS pools
sudo zfs unmount -a
# Export VAULT (always present)
sudo zpool export vaultpg
sudo zpool export mediapg
# Export GREEN (if connected for personal trip)
if zpool list greenpg &>/dev/null; then
sudo zpool export greenpg
fi
# Export MEDIA-FAMILY (if connected for family trip)
if zpool list mediapg &>/dev/null; then
sudo zpool export mediapg
fi
# Verify pools exported
zpool list
# Should NOT show vaultpg or mediapg
# Should NOT show vaultpg, greenpg, or mediapg
# Shutdown Pi
sudo shutdown -h now