New Grimoire

2026-04-12 09:53:51 -05:00 · 2026-04-12 09:53:51 -05:00 · cc574f8aed
commit cc574f8aed
parent 77d589a13d
157 changed files with 29420 additions and 0 deletions
--- a/Watch-Grimoire/Monitoring/Services.md
+++ b/Watch-Grimoire/Monitoring/Services.md
@ -0,0 +1,216 @@
+---
+title: Monitors and Alerts
+description: DIUN/NTFY on Netgrimoire
+published: true
+date: 2026-04-10T19:35:18.743Z
+tags: 
+editor: markdown
+dateCreated: 2026-04-10T19:35:18.743Z
+---
+
+# Notifications — Netgrimoire
+
+## Overview
+
+All Netgrimoire notifications route through a self-hosted ntfy instance at `https://ntfy.netgrimoire.com`. Topics are organized by service category.
+
+## ntfy Topic Structure
+
+| Topic | Services | Purpose |
+|-------|----------|---------|
+| `netgrimoire-diun` | DIUN | Docker image update notifications |
+| `netgrimoire-media` | Sonarr, Radarr, SABnzbd | Download and media management events |
+| `netgrimoire-backup` | Kopia | Backup completion and errors |
+| `netgrimoire-alerts` | Prometheus/Alertmanager | Infrastructure alerts (future) |
+
+Subscribe to topics at `https://ntfy.netgrimoire.com/<topic>` or via the ntfy mobile app.
+
+---
+
+## DIUN — Image Update Notifications
+
+DIUN watches all Docker services for image updates and posts to `netgrimoire-diun`.
+
+**Configuration** (`swarm/diun.yaml`):
+
+```yaml
+environment:
+  DIUN_NOTIF_NTFY_ENDPOINT: https://ntfy.netgrimoire.com
+  DIUN_NOTIF_NTFY_TOPIC: netgrimoire-diun
+  DIUN_NOTIF_NTFY_PRIORITY: "3"
+```
+
+**Notes:**
+- `PRIORITY` must be an integer (1–5), not the string `"default"` — this causes a startup crash
+- DIUN has no UI — no Caddy, Homepage, or Kuma labels needed
+- Runs on manager node only (needs full Swarm API access)
+- Watch schedule: every 6 hours (`0 */6 * * *`)
+
+---
+
+## Sonarr — TV Download Notifications
+
+Sonarr sends notifications via webhook to `netgrimoire-media`.
+
+**Setup** (done via UI — not compose):
+
+1. Settings → Connect → + → **Webhook**
+2. Name: `ntfy`
+3. URL: `https://ntfy.netgrimoire.com/netgrimoire-media`
+4. Method: `POST`
+5. Triggers: On Grab, On Download, On Upgrade, On Health Issue
+6. Test → Save
+
+---
+
+## Radarr — Movie Download Notifications
+
+Identical setup to Sonarr.
+
+**Setup** (done via UI):
+
+1. Settings → Connect → + → **Webhook**
+2. Name: `ntfy`
+3. URL: `https://ntfy.netgrimoire.com/netgrimoire-media`
+4. Method: `POST`
+5. Triggers: On Grab, On Download, On Upgrade, On Health Issue
+6. Test → Save
+
+---
+
+## SABnzbd — Usenet Download Notifications
+
+SABnzbd does not have native ntfy support. Notifications are handled via a custom shell script.
+
+### Script Location
+
+```
+/data/nfs/znas/Docker/Sabnzbd/scripts/ntfy-notify.sh
+```
+
+Mounted into the container at `/config/scripts/ntfy-notify.sh`.
+
+### Script
+
+```bash
+#!/bin/bash
+# SABnzbd ntfy notification script
+# SABnzbd passes: $1=Job name, $2=Final dir, $3=NZB file, 
+#                 $4=Category, $5=Group, $6=Status, $7=Fail message
+
+NTFY_URL="https://ntfy.netgrimoire.com/netgrimoire-media"
+
+JOB_NAME="$1"
+STATUS_CODE="$6"
+FAIL_MSG="$7"
+
+case "$STATUS_CODE" in
+  0) TITLE="✅ SABnzbd — Download Complete"
+     MSG="$JOB_NAME"; PRIORITY=3 ;;
+  1) TITLE="⚠️ SABnzbd — Post-Processing Error"
+     MSG="$JOB_NAME — $FAIL_MSG"; PRIORITY=4 ;;
+  2) TITLE="❌ SABnzbd — Download Failed"
+     MSG="$JOB_NAME — $FAIL_MSG"; PRIORITY=5 ;;
+  *) TITLE="ℹ️ SABnzbd — Notification"
+     MSG="$JOB_NAME (status: $STATUS_CODE)"; PRIORITY=3 ;;
+esac
+
+curl -s \
+  -H "Title: $TITLE" \
+  -H "Priority: $PRIORITY" \
+  -H "Tags: floppy_disk" \
+  -d "$MSG" \
+  "$NTFY_URL"
+
+exit 0
+```
+
+### SABnzbd UI Setup
+
+1. Config → Folders → **Post-Processing Scripts Folder** → set to `/config/scripts`
+2. Config → Notifications → Notification Script section
+3. Check **Enable notification script**
+4. Script dropdown → select `ntfy-notify.sh`
+5. Check: Job finished, Job failed, Warning, Error, Disk full
+6. Test → Save
+
+**Note:** The scripts folder must be configured under Config → Folders first or the script won't appear in the dropdown.
+
+---
+
+## Kopia — Backup Notifications
+
+Kopia has no native webhook support. Notifications are handled via a cron script on znas that uses the Kopia CLI inside the Docker container.
+
+### Script Location
+
+```
+/usr/local/bin/kopia-notify.sh
+```
+
+### How It Works
+
+- Runs hourly via cron on znas
+- Uses `docker exec` to run `kopia snapshot list --json` inside the container
+- Parses JSON output with Python to find snapshots completed in the last hour
+- Posts success or error notification to `netgrimoire-backup`
+
+### Cron Entry (znas root crontab)
+
+```
+0 * * * * /usr/local/bin/kopia-notify.sh
+```
+
+### Notification Format
+
+**Success:** `✅ Kopia — Backup Complete`
+```
+host:path
+N files • X.X GB
+```
+
+**Error:** `❌ Kopia — Backup Errors`
+```
+host:path
+N error(s) • N files • X.X GB
+```
+
+### Kopia API Access
+
+The Kopia API is accessible inside the container only. Direct host access via port 51515 does not work due to network routing. Use `docker exec` instead:
+
+```bash
+docker exec $(docker ps -q -f name=kopia_kopia) \
+  kopia snapshot list --json
+```
+
+---
+
+## ntfy Compose Reference
+
+```yaml
+# swarm/ntfy.yaml
+services:
+  ntfy:
+    image: binwiederhier/ntfy
+    command: serve
+    user: "1964:1964"
+    environment:
+      TZ: America/Chicago
+    volumes:
+      - /data/nfs/znas/Docker/ntfy/cache:/var/cache/ntfy
+      - /data/nfs/znas/Docker/ntfy/etc:/etc/ntfy
+    ports:
+      - 81:80
+    networks:
+      - netgrimoire
+    deploy:
+      labels:
+        caddy: ntfy.netgrimoire.com
+        caddy.reverse_proxy: ntfy:80
+        caddy.import: crowdsec
+        # Note: no authentik — ntfy must be publicly reachable
+        #       for external services to post notifications
+```
+
+**Note:** ntfy intentionally has no `caddy.import_1: authentik` — it must remain publicly accessible so external services (OPNsense CrowdSec plugin, Monit, etc.) can post to it without authentication.