diff --git a/Netgrimoire/Infrastructure/Monitoring.txt b/Netgrimoire/Infrastructure/Monitoring.txt new file mode 100644 index 0000000..b7f4a73 --- /dev/null +++ b/Netgrimoire/Infrastructure/Monitoring.txt @@ -0,0 +1,206 @@ +# Notifications — Netgrimoire + +## Overview + +All Netgrimoire notifications route through a self-hosted ntfy instance at `https://ntfy.netgrimoire.com`. Topics are organized by service category. + +## ntfy Topic Structure + +| Topic | Services | Purpose | +|-------|----------|---------| +| `netgrimoire-diun` | DIUN | Docker image update notifications | +| `netgrimoire-media` | Sonarr, Radarr, SABnzbd | Download and media management events | +| `netgrimoire-backup` | Kopia | Backup completion and errors | +| `netgrimoire-alerts` | Prometheus/Alertmanager | Infrastructure alerts (future) | + +Subscribe to topics at `https://ntfy.netgrimoire.com/` or via the ntfy mobile app. + +--- + +## DIUN — Image Update Notifications + +DIUN watches all Docker services for image updates and posts to `netgrimoire-diun`. + +**Configuration** (`swarm/diun.yaml`): + +```yaml +environment: + DIUN_NOTIF_NTFY_ENDPOINT: https://ntfy.netgrimoire.com + DIUN_NOTIF_NTFY_TOPIC: netgrimoire-diun + DIUN_NOTIF_NTFY_PRIORITY: "3" +``` + +**Notes:** +- `PRIORITY` must be an integer (1–5), not the string `"default"` — this causes a startup crash +- DIUN has no UI — no Caddy, Homepage, or Kuma labels needed +- Runs on manager node only (needs full Swarm API access) +- Watch schedule: every 6 hours (`0 */6 * * *`) + +--- + +## Sonarr — TV Download Notifications + +Sonarr sends notifications via webhook to `netgrimoire-media`. + +**Setup** (done via UI — not compose): + +1. Settings → Connect → + → **Webhook** +2. Name: `ntfy` +3. URL: `https://ntfy.netgrimoire.com/netgrimoire-media` +4. Method: `POST` +5. Triggers: On Grab, On Download, On Upgrade, On Health Issue +6. Test → Save + +--- + +## Radarr — Movie Download Notifications + +Identical setup to Sonarr. + +**Setup** (done via UI): + +1. Settings → Connect → + → **Webhook** +2. Name: `ntfy` +3. URL: `https://ntfy.netgrimoire.com/netgrimoire-media` +4. Method: `POST` +5. Triggers: On Grab, On Download, On Upgrade, On Health Issue +6. Test → Save + +--- + +## SABnzbd — Usenet Download Notifications + +SABnzbd does not have native ntfy support. Notifications are handled via a custom shell script. + +### Script Location + +``` +/data/nfs/znas/Docker/Sabnzbd/scripts/ntfy-notify.sh +``` + +Mounted into the container at `/config/scripts/ntfy-notify.sh`. + +### Script + +```bash +#!/bin/bash +# SABnzbd ntfy notification script +# SABnzbd passes: $1=Job name, $2=Final dir, $3=NZB file, +# $4=Category, $5=Group, $6=Status, $7=Fail message + +NTFY_URL="https://ntfy.netgrimoire.com/netgrimoire-media" + +JOB_NAME="$1" +STATUS_CODE="$6" +FAIL_MSG="$7" + +case "$STATUS_CODE" in + 0) TITLE="✅ SABnzbd — Download Complete" + MSG="$JOB_NAME"; PRIORITY=3 ;; + 1) TITLE="⚠️ SABnzbd — Post-Processing Error" + MSG="$JOB_NAME — $FAIL_MSG"; PRIORITY=4 ;; + 2) TITLE="❌ SABnzbd — Download Failed" + MSG="$JOB_NAME — $FAIL_MSG"; PRIORITY=5 ;; + *) TITLE="ℹ️ SABnzbd — Notification" + MSG="$JOB_NAME (status: $STATUS_CODE)"; PRIORITY=3 ;; +esac + +curl -s \ + -H "Title: $TITLE" \ + -H "Priority: $PRIORITY" \ + -H "Tags: floppy_disk" \ + -d "$MSG" \ + "$NTFY_URL" + +exit 0 +``` + +### SABnzbd UI Setup + +1. Config → Folders → **Post-Processing Scripts Folder** → set to `/config/scripts` +2. Config → Notifications → Notification Script section +3. Check **Enable notification script** +4. Script dropdown → select `ntfy-notify.sh` +5. Check: Job finished, Job failed, Warning, Error, Disk full +6. Test → Save + +**Note:** The scripts folder must be configured under Config → Folders first or the script won't appear in the dropdown. + +--- + +## Kopia — Backup Notifications + +Kopia has no native webhook support. Notifications are handled via a cron script on znas that uses the Kopia CLI inside the Docker container. + +### Script Location + +``` +/usr/local/bin/kopia-notify.sh +``` + +### How It Works + +- Runs hourly via cron on znas +- Uses `docker exec` to run `kopia snapshot list --json` inside the container +- Parses JSON output with Python to find snapshots completed in the last hour +- Posts success or error notification to `netgrimoire-backup` + +### Cron Entry (znas root crontab) + +``` +0 * * * * /usr/local/bin/kopia-notify.sh +``` + +### Notification Format + +**Success:** `✅ Kopia — Backup Complete` +``` +host:path +N files • X.X GB +``` + +**Error:** `❌ Kopia — Backup Errors` +``` +host:path +N error(s) • N files • X.X GB +``` + +### Kopia API Access + +The Kopia API is accessible inside the container only. Direct host access via port 51515 does not work due to network routing. Use `docker exec` instead: + +```bash +docker exec $(docker ps -q -f name=kopia_kopia) \ + kopia snapshot list --json +``` + +--- + +## ntfy Compose Reference + +```yaml +# swarm/ntfy.yaml +services: + ntfy: + image: binwiederhier/ntfy + command: serve + user: "1964:1964" + environment: + TZ: America/Chicago + volumes: + - /data/nfs/znas/Docker/ntfy/cache:/var/cache/ntfy + - /data/nfs/znas/Docker/ntfy/etc:/etc/ntfy + ports: + - 81:80 + networks: + - netgrimoire + deploy: + labels: + caddy: ntfy.netgrimoire.com + caddy.reverse_proxy: ntfy:80 + caddy.import: crowdsec + # Note: no authentik — ntfy must be publicly reachable + # for external services to post notifications +``` + +**Note:** ntfy intentionally has no `caddy.import_1: authentik` — it must remain publicly accessible so external services (OPNsense CrowdSec plugin, Monit, etc.) can post to it without authentication. \ No newline at end of file