docs: create Netgrimoire/Pocket/Stash_Integration

This commit is contained in:
Administrator 2026-02-20 04:48:24 +00:00 committed by John Smith
parent ff9f5df5aa
commit 47402071b2

View file

@ -0,0 +1,741 @@
---
title: Pocket Clips
description: Integrating Stash
published: true
date: 2026-02-20T04:48:11.191Z
tags:
editor: markdown
dateCreated: 2026-02-20T04:48:11.191Z
---
# Pocket Grimoire - Stash Integration Guide
**Adding Stash media library manager to Pocket Grimoire using ZFS replication**
---
## Overview
This guide extends the Pocket Grimoire deployment to include Stash, a powerful media library manager. The architecture uses ZFS replication to mirror Stash data from Netgrimoire (home) to Pocket Grimoire (travel), allowing:
- Full Stash browsing offline
- All previews, thumbnails, and metadata available
- Zero CPU load on Pi (no scanning/generation)
- Automatic synchronization via existing sync jobs
- Read-only operation on travel
**Key Principle:** Netgrimoire does all heavy lifting (scanning, preview generation), Pocket Grimoire just serves pre-generated content.
---
## Architecture
```
┌─────────────────────────────────────────┐
│ NETGRIMOIRE (Home - Heavy Lifting) │
├─────────────────────────────────────────┤
│ Stash Container: │
│ - Full scanning │
│ - Preview generation │
│ - Scene detection │
│ - Database writes │
│ - All CPU/GPU intensive work │
│ │
│ ZFS Dataset: vault/stash │
│ - Stash database │
│ - Generated previews │
│ - Thumbnails & sprites │
│ - Scene markers │
└─────────────────────────────────────────┘
ZFS Send/Receive
(via syncoid every 6 hours)
┌─────────────────────────────────────────┐
│ POCKET GRIMOIRE (Travel - Read Only) │
├─────────────────────────────────────────┤
│ Stash Container: │
│ - Read-only mode │
│ - No scanning │
│ - No generation │
│ - Just browse existing data │
│ │
│ ZFS Dataset: vaultpg/stash │
│ - Mirrored from Netgrimoire │
│ - Complete database replica │
│ - All previews pre-generated │
└─────────────────────────────────────────┘
```
---
## Storage Requirements
### On Netgrimoire
```
Database: 500MB - 2GB (depends on library size)
Generated previews: 5GB - 50GB (depends on preview settings)
Blobs/markers: 1GB - 10GB
───────────────────────────────────────────────────────
Total: 6.5GB - 62GB
```
### On Pocket Grimoire
```
Same as Netgrimoire (full replica via ZFS)
Stored on Vault SSD (vaultpg pool)
```
### Sync Bandwidth
- **Initial sync:** 10-30GB (1-2 hours on gigabit LAN)
- **Incremental sync:** 50MB - 500MB per 6 hours (2-10 minutes)
- **After adding 10 videos:** ~210MB (database + previews)
---
## Resource Impact
### Updated Resource Profile (With Stash)
**Idle:**
```
Wiki.js + PostgreSQL: ~250MB RAM
Jellyfin (idle): ~150MB RAM
Stash (read-only): ~200MB RAM # Much lighter than active Stash
ZFS ARC: ~512MB RAM
System: ~200MB RAM
─────────────────────────────────────────
Total: ~1.3GB / 8GB RAM ✓
CPU: <10%
Temperature: Cool
```
**Browsing Stash:**
```
Stash (active): ~300MB RAM
Other services: ~1.1GB RAM
─────────────────────────────────────────
Total: ~1.4GB / 8GB RAM ✓
CPU: <15%
Temperature: Cool to Warm
```
**Media Playback + Stash Browsing:**
```
Jellyfin (serving): ~200MB RAM
Stash (active): ~300MB RAM
Wiki.js + PostgreSQL: ~250MB RAM
ZFS ARC: ~512MB RAM
System: ~200MB RAM
─────────────────────────────────────────
Total: ~1.5GB / 8GB RAM ✓
Still plenty of headroom
```
---
## Installation: Netgrimoire (Home)
### 1. Create ZFS Datasets
```bash
# On Netgrimoire
sudo zfs create -o recordsize=16K vault/stash
sudo zfs create -o recordsize=16K vault/stash/config # Database
sudo zfs create -o recordsize=1M vault/stash/generated # Previews
sudo zfs create -o recordsize=128K vault/stash/blobs # Scene markers
sudo zfs create vault/stash/cache # Temporary (won't replicate)
# Set appropriate permissions
sudo chown -R 1000:1000 /vault/stash
```
**Why different recordsizes:**
- **16K** for config (SQLite database performs best with small records)
- **1M** for generated (large video preview files)
- **128K** for blobs (medium-sized screenshots and markers)
### 2. Create Stash Docker Compose
```bash
mkdir -p /srv/netgrimoire/stacks/stash
nano /srv/netgrimoire/stacks/stash/docker-compose.yml
```
```yaml
services:
stash:
image: stashapp/stash:latest
container_name: netgrimoire_stash
environment:
- STASH_STASH=/data/
- STASH_GENERATED=/generated/
- STASH_CACHE=/cache/
- STASH_BLOBS=/blobs/
- TZ=America/Chicago
volumes:
- /vault/stash/config:/root/.stash
- /vault/stash/generated:/generated
- /vault/stash/cache:/cache
- /vault/stash/blobs:/blobs
- /vault/media:/data:ro # Your media library (read-only)
ports:
- "9999:9999"
restart: unless-stopped
```
**Start Stash:**
```bash
cd /srv/netgrimoire/stacks/stash
docker compose up -d
```
### 3. Configure Stash
1. **Access Stash:**
- Open browser: `http://netgrimoire.local:9999`
- Complete initial setup wizard
2. **Add Library:**
- Settings → Library
- Add folder: `/data/library/movies` (or your media path)
- Save
3. **Configure Previews:**
- Settings → Tasks → Generate
- Preview Generation: Enable
- Preview Settings:
- Video encoding: VP9 or H.264
- Resolution: 720p (good quality, reasonable size)
- Segment duration: 10 seconds
- Generate image previews: Enable
- Generate sprites: Enable
4. **Run Initial Scan:**
- Tasks → Scan
- Wait for completion (can take hours depending on library size)
5. **Generate Previews:**
- Tasks → Generate → Generate Previews
- This is CPU intensive - let run at home
- Can take many hours depending on library size
6. **Optional: Scene Detection:**
- Tasks → Generate → Auto Tag
- Scene detection, performer matching, etc.
- Very CPU intensive, run at home only
### 4. Take ZFS Snapshots (Optional but Recommended)
```bash
# After initial scan and preview generation
sudo zfs snapshot vault/stash/config@initial
sudo zfs snapshot vault/stash/generated@initial
sudo zfs snapshot vault/stash/blobs@initial
```
---
## Installation: Pocket Grimoire (Travel)
### 1. Create ZFS Datasets
```bash
# On Pocket Grimoire
sudo zfs create -o recordsize=16K vaultpg/stash
sudo zfs create -o recordsize=16K vaultpg/stash/config
sudo zfs create -o recordsize=1M vaultpg/stash/generated
sudo zfs create -o recordsize=128K vaultpg/stash/blobs
sudo zfs create vaultpg/stash/cache # Local cache only, not synced
# Set permissions
sudo chown -R 1000:1000 /srv/vaultpg/stash
```
### 2. Create Stash Docker Compose (Read-Only)
```bash
mkdir -p /srv/pocket-grimoire/stacks/stash
nano /srv/pocket-grimoire/stacks/stash/docker-compose.yml
```
```yaml
services:
stash:
image: stashapp/stash:latest
container_name: pocketgrimoire_stash
environment:
- STASH_STASH=/data/
- STASH_GENERATED=/generated/
- STASH_CACHE=/cache/
- STASH_BLOBS=/blobs/
- TZ=America/Chicago
volumes:
- /srv/vaultpg/stash/config:/root/.stash:ro # READ-ONLY
- /srv/vaultpg/stash/generated:/generated:ro # READ-ONLY
- /srv/vaultpg/stash/blobs:/blobs:ro # READ-ONLY
- /srv/pocket-grimoire/data/stash/cache:/cache # Local cache (writable)
- /srv/mediapg:/data:ro # Media (already present, read-only)
ports:
- "9999:9999"
restart: unless-stopped
```
**Note the `:ro` flags** - Filesystem is mounted read-only, preventing any writes.
**Start Stash:**
```bash
cd /srv/pocket-grimoire/stacks/stash
docker compose up -d
```
### 3. Configure Stash for Read-Only Operation
**After first start:**
1. **Access Stash:**
- Open browser: `http://pocket-grimoire.local:9999`
- Should show library from Netgrimoire (after first sync)
2. **Disable Background Tasks:**
- Settings → Tasks
- Disable all automatic tasks:
- ❌ Auto-scan
- ❌ Auto-tag
- ❌ Auto-preview generation
- ❌ Auto-cleanup
3. **Verify Read-Only:**
- Try to edit a scene or performer
- Should fail with permission error
- This confirms read-only mode working
---
## ZFS Replication Configuration
### Update Sync Script
Edit your existing sync script to include Stash datasets:
```bash
sudo nano /usr/local/sbin/pocketgrimoire-zfs-pull.sh
```
Add these lines:
```bash
#!/usr/bin/env bash
set -euo pipefail
SRC_HOST="netgrimoire.local"
SSH_KEY="/srv/pocket-grimoire/keys/zfs_pull_ro"
# Existing vault data syncs...
syncoid --no-sync-snap --recursive \
--sshkey "${SSH_KEY}" \
"root@${SRC_HOST}:vault/docs" \
"vaultpg/mirror/docs"
syncoid --no-sync-snap --recursive \
--sshkey "${SSH_KEY}" \
"root@${SRC_HOST}:vault/photos" \
"vaultpg/mirror/photos"
# NEW: Stash database and generated content
syncoid --no-sync-snap \
--sshkey "${SSH_KEY}" \
"root@${SRC_HOST}:vault/stash/config" \
"vaultpg/stash/config"
syncoid --no-sync-snap \
--sshkey "${SSH_KEY}" \
"root@${SRC_HOST}:vault/stash/generated" \
"vaultpg/stash/generated"
syncoid --no-sync-snap \
--sshkey "${SSH_KEY}" \
"root@${SRC_HOST}:vault/stash/blobs" \
"vaultpg/stash/blobs"
# Note: We skip cache dataset (temporary data, not needed on Pocket)
```
**Sync runs automatically every 6 hours** via existing systemd timer.
### Manual Sync (For Testing)
```bash
# Trigger sync immediately
sudo systemctl start pocketgrimoire-sync.service
# Watch progress
tail -f /var/log/pocketgrimoire-sync.log
# Verify Stash data synced
ls -lh /srv/vaultpg/stash/config
ls -lh /srv/vaultpg/stash/generated
```
---
## Media Path Consistency
**Critical:** Media paths must be consistent between Netgrimoire and Pocket Grimoire.
### Option 1: Matching Paths
**If your media is in same location:**
```
Netgrimoire: /vault/media/library/
Pocket: /srv/mediapg/library/
```
**Stash stores absolute paths** - ensure they match or use symlinks.
### Option 2: Symlinks
```bash
# On Pocket Grimoire
sudo ln -s /srv/mediapg /vault/media
```
This makes `/vault/media` point to `/srv/mediapg`, matching Netgrimoire's paths.
### Option 3: Stash Path Mapping (If Stash supports it)
Some versions of Stash support path mapping in config. Check Stash documentation.
---
## Workflow
### At Home (Before Trips)
**On Netgrimoire:**
1. Add new media to library
2. Stash auto-scans (or trigger manually)
3. Previews generate automatically (or trigger manually)
4. Tag/organize content as desired
5. Everything happens in background
**Pocket Grimoire (automatic):**
1. Every 6 hours: syncoid pulls changes from Netgrimoire
2. Stash database updated
3. New previews synced
4. Scene markers updated
5. Ready for next trip - no manual intervention
### While Traveling
**On Pocket Grimoire:**
1. Access Stash: `http://pocket-grimoire.local:9999`
2. Browse library, tags, performers, studios
3. View previews and scene markers
4. Click to play via Jellyfin or external player
5. **Cannot:** Scan, edit tags, generate previews (read-only)
**If you need to edit:**
1. VPN back to Netgrimoire (via WireGuard)
2. Access Netgrimoire Stash: `http://netgrimoire.local:9999`
3. Make changes there
4. Changes sync on next 6-hour cycle
### After Returning Home
1. Sync happens automatically
2. Any changes made on Netgrimoire while away are pulled
3. Pocket Grimoire stays up-to-date
4. No manual intervention required
---
## Accessing Stash
### From Onn Streaming Boxes
**Install StashApp for Android TV:**
- See "Onn 4K Streaming Box Setup Guide" for detailed instructions
- Download APK from: https://github.com/damontecres/StashAppAndroidTV/releases
- Sideload onto Onn boxes
- Configure server: `http://pocket-grimoire.local:9999`
### From Laptop
**Via Web Browser:**
- Open: `http://pocket-grimoire.local:9999`
- Full Stash web interface available
- Same as Netgrimoire, but read-only
### From Phone/Tablet
**Via Web Browser:**
- Connect to portapotty WiFi
- Open: `http://pocket-grimoire.local:9999`
- Mobile-responsive interface
---
## Updated Service Access Summary
**When connected to portapotty network:**
```
Wiki.js: http://pocket-grimoire.local:3000
Jellyfin: http://pocket-grimoire.local:8096
Stash: http://pocket-grimoire.local:9999 ← NEW
File Browser: http://pocket-grimoire.local:8080
Dozzle: http://pocket-grimoire.local:8888
SSH: ssh user@pocket-grimoire.local
NFS Media: nfs://pocket-grimoire.local/srv/mediapg
Router Admin: http://192.168.8.1
```
---
## Troubleshooting
### Stash Won't Start
**Check Docker container status:**
```bash
docker ps | grep stash
docker logs pocketgrimoire_stash
```
**Common issues:**
- ZFS datasets not mounted: `zfs mount -a`
- Permission denied: `sudo chown -R 1000:1000 /srv/vaultpg/stash`
- Port conflict: Another service using port 9999
### Stash Shows "Database is locked"
**This means read-only mode is working correctly.**
If you need to make changes:
1. VPN to Netgrimoire
2. Edit on Netgrimoire Stash
3. Changes sync to Pocket on next cycle
### Previews Not Showing
**Verify previews synced:**
```bash
ls /srv/vaultpg/stash/generated/
# Should show many .webp or .mp4 files
```
**If empty:**
- Previews not generated on Netgrimoire yet
- Sync hasn't completed (check sync logs)
- Check: `tail -n 200 /var/log/pocketgrimoire-sync.log`
### Media Files Not Found
**Check paths match:**
```bash
# On Netgrimoire, Stash sees files at:
# /vault/media/library/movies/
# On Pocket, files are at:
# /srv/mediapg/library/movies/
# Create symlink to match:
sudo ln -s /srv/mediapg /vault/media
```
**Or verify Docker volume mount:**
```bash
docker inspect pocketgrimoire_stash | grep -A 10 Mounts
# Should show /srv/mediapg mounted as /data
```
### Sync Takes Too Long
**Check what's being synced:**
```bash
# Watch sync in progress
sudo zfs list -t snapshot | grep stash
# Check dataset sizes
sudo zfs list | grep stash
```
**Optimization:**
- Reduce preview quality on Netgrimoire (smaller files)
- Sync less frequently (change timer from 6h to 12h)
- Only sync when on fast network (manual trigger)
### Stash API Key Issues
**If StashApp asks for API key:**
1. **Find API key on Netgrimoire:**
```bash
# On Netgrimoire
cat /vault/stash/config/config.yml | grep api_key
```
2. **Or via Stash web UI:**
- Settings → Security → API Key
- Generate key if not present
- Copy key
3. **Configure StashApp:**
- Settings → Server → API Key
- Paste key from Netgrimoire
**Note:** API key is in config, which syncs to Pocket, so should be the same.
---
## Optimization Tips
### Reduce Storage Usage
**On Netgrimoire (affects Pocket via sync):**
1. **Lower preview quality:**
- Settings → Tasks → Generate
- Video resolution: 480p instead of 720p
- Lower bitrate
- Smaller file sizes = faster sync
2. **Disable sprite generation:**
- Settings → Tasks → Generate
- Disable sprite generation (just use video previews)
- Saves significant space
3. **Limit preview duration:**
- Settings → Tasks → Generate
- Preview duration: 20 seconds instead of 60 seconds
- Smaller files
### Speed Up Sync
**Initial sync optimization:**
```bash
# First sync: Compress during transfer
syncoid --compress=lz4 \
--sshkey "${SSH_KEY}" \
"root@${SRC_HOST}:vault/stash/generated" \
"vaultpg/stash/generated"
```
**Bandwidth limiting (if needed):**
```bash
# Limit to 10MB/s
syncoid --bwlimit=10M \
--sshkey "${SSH_KEY}" \
"root@${SRC_HOST}:vault/stash/config" \
"vaultpg/stash/config"
```
### Reduce Preview Generation Load
**On Netgrimoire:**
- Generate previews during off-hours (nightly)
- Use Task Scheduler in Stash
- Limit concurrent preview generation
- Lower thread count for generation
---
## Maintenance
### Weekly
**On Netgrimoire:**
- Check Stash is running: `docker ps | grep stash`
- Verify previews generating (if auto-enabled)
- Check disk usage: `zfs list | grep stash`
**On Pocket Grimoire:**
- Verify sync is working: `tail /var/log/pocketgrimoire-sync.log`
- Check Stash is accessible: `http://pocket-grimoire.local:9999`
### Monthly
**On Netgrimoire:**
- Run manual scan if auto-scan disabled
- Clean up orphaned files: Tasks → Clean
- Check database integrity: Tasks → Optimize Database
- ZFS scrub: `sudo zfs scrub vault/stash`
**On Pocket Grimoire:**
- Verify dataset health: `sudo zpool status vaultpg`
- Check sync logs for errors
- Test Stash browsing and preview playback
### Before Trips
- [ ] Run scan on Netgrimoire (capture new content)
- [ ] Generate previews for new content
- [ ] Verify sync completed successfully
- [ ] Test Stash on Pocket Grimoire
- [ ] Verify StashApp on Onn boxes connects
### After Trips
- Check sync logs for any errors during trip
- No action needed - sync continues automatically
---
## Alternative: Manual Database Export (Not Recommended)
**If ZFS replication is not available:**
You can manually export/import Stash database:
```bash
# On Netgrimoire - Export
sqlite3 /vault/stash/config/stash-go.sqlite ".backup /tmp/stash-backup.db"
scp /tmp/stash-backup.db pocket-grimoire:/tmp/
# On Pocket Grimoire - Import
docker compose down
cp /tmp/stash-backup.db /srv/vaultpg/stash/config/stash-go.sqlite
docker compose up -d
```
**Why this is not recommended:**
- Manual process (error-prone)
- Doesn't sync previews/blobs automatically
- Must copy those separately (tens of GB)
- ZFS replication is much cleaner
---
## Summary
**What you get with Stash on Pocket Grimoire:**
- ✅ Full Stash browsing offline
- ✅ All previews and metadata available
- ✅ Zero CPU load on Pi (read-only)
- ✅ Automatic synchronization
- ✅ Professional media library management
- ✅ Scene markers and performer tagging
- ✅ Works with StashApp on Onn boxes
**What you give up:**
- ❌ Cannot edit/tag while traveling (must VPN to Netgrimoire)
- ❌ Cannot generate previews on travel (shouldn't anyway)
- ❌ 10-30GB additional storage on Vault SSD
- ❌ Slightly more complex setup (but worth it)
**Recommended for:**
- Large media libraries that need organization
- Users who want professional media management
- Those who already use or want to use Stash
- Anyone who values rich metadata and previews
**Skip if:**
- Just want simple media playback (Jellyfin is enough)
- Very limited storage on Vault SSD
- Don't want complexity of ZFS replication
- Don't need tagging/scene detection features
---
*This guide supplements the main Pocket Grimoire deployment guide. Ensure main guide is completed before adding Stash.*