traveler/Netgrimoire
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

[Docs Draft] znas filesystem -- pending review (20260715-195113-docs-znas-b6a4) #10

New issue
Closed
opened 2026-07-15 14:51:56 -05:00 by traveler · 1 comment
traveler commented 2026-07-15 14:51:56 -05:00
Owner
Copy link

AUTO-GENERATED DRAFT -- NOT YET PUBLISHED. This page was written automatically by the @document pilot (chat -> n8n -> Mistral via gremlin-proxy) from a mix of prior memory notes and one live SSH inspection pass. Read it carefully before manually committing it to the wiki -- do not trust it blindly, especially anything under "Discrepancies noted".

# ZNAS Filesystem Topology

## Overview
ZNAS is the primary storage system for NetGrimoire, running Ubuntu with OpenZFS. It serves as the source of truth for all storage, including datasets that replicate to the Pocket Grimoire portable system.

All NFS exports are ZFS datasets mounted directly under `/export/`. ZNAS mounts its own NFS exports back to itself at `/data/nfs/znas` to provide path consistency for Docker Swarm containers regardless of scheduling node.

---

## ZFS Pools and Datasets

### Pools
| Pool           | Size   | Alloc  | Free   | Health  | Notes                                  |
|----------------|--------|--------|--------|---------|----------------------------------------|
| `vault`        | 129T   | 91.5T  | 37.7T  | ONLINE  | Primary pool; two RAIDZ1 VDEVs         |
| `pocket-media` | 4.55T  | 0      | 4.55T  | ONLINE  | Pocket Grimoire media pool (external)  |

### Datasets (`vault` pool)
All datasets mount directly to `/export/*` unless otherwise noted.

| Dataset                          | Mountpoint                     | Compression | Notes                                  |
|----------------------------------|--------------------------------|-------------|----------------------------------------|
| `vault`                          | `/export`                      | zstd        | Root dataset                           |
| `vault/Common`                   | `/export/Common`               | zstd        |                                        |
| `vault/Data`                     | `/export/Data`                 | lz4         |                                        |
| `vault/Data/media_books`         | `/export/Data/media/books`     | zstd        | `nohide` NFS export                    |
| `vault/Data/media_comics`        | `/export/Data/media/comics`    | zstd        | `nohide` NFS export                    |
| `vault/Docker`                   | `/export/Docker`               | zstd        | Swarm persistent volumes               |
| `vault/Green`                    | `/export/Green`                | off         |                                        |
| `vault/Green/Pocket`             | `/export/Green/Pocket`         | off         | Pocket sync source                     |
| `vault/Photos`                   | `/export/Photos`               | zstd        |                                        |
| `vault/Kopia`                    | `/srv/vault/kopia_repository`  | zstd        | Local service consumption              |
| `vault/NextCloud-AIO`            | `/srv/NextCloud-AIO`           | lz4         | Local service consumption              |
| `vault/backup`                   | `/srv/vault/backup`            | zstd        | Local service consumption              |
| `vault/immich`                   | `/srv/immich`                  | lz4         | Local service consumption              |
| `vault/pocket`                   | `none`                         | zstd        | Root for Pocket sync datasets          |
| `vault/pocket/green`             | `/pocket/green`                | zstd        |                                        |
| `vault/pocket/green/media`       | `/pocket/green/media`          | zstd        |                                        |
| `vault/pocket/media`             | `/pocket/media`                | zstd        |                                        |
| `vault/pocket/media/reference`   | `/pocket/media/reference`      | zstd        |                                        |
| `vault/pocket/services`          | `/pocket/services`             | zstd        | Docker volumes for Pocket              |
| `vault/pocket/services/DockerVol`| `/pocket/services/DockerVol`   | zstd        | Primary sync target for Pocket         |

---

## NFS Exports

### Server Configuration (`/etc/exports`)
NFSv4 pseudo-root is mandatory. All exports are world-readable/writable with `no_root_squash`.

```exports
/export *(rw,fsid=0,no_root_squash,no_subtree_check,crossmnt)
/export/Common *(fsid=4,rw,no_root_squash,no_subtree_check,insecure)
/export/Data *(fsid=5,rw,no_root_squash,no_subtree_check,insecure,crossmnt)
/export/Data/media/books *(fsid=51,rw,no_root_squash,no_subtree_check,insecure,nohide)
/export/Data/media/comics *(fsid=52,rw,no_root_squash,no_subtree_check,insecure,nohide)
/export/Docker *(fsid=29,rw,no_root_squash,sync,no_subtree_check,insecure)
/export/Green *(fsid=30,rw,no_root_squash,no_subtree_check,insecure,crossmnt)
/export/photos *(fsid=31,rw,no_root_squash,no_subtree_check,insecure)
/export/Pocket *(fsid=32,rw,no_root_squash,no_subtree_check,insecure)
/export/Grimoire *(fsid=33,rw,no_root_squash,no_subtree_check,insecure)
/export/green/media/pocket *(fsid=35,rw,no_root_squash,no_subtree_check,insecure)

Loopback Mount (ZNAS)

ZNAS mounts its own NFS exports back to itself at /data/nfs/znas for Docker Swarm path consistency.

vault/Common on /data/nfs/znas/Common type zfs (rw,noatime,xattr,posixacl,casesensitive)
vault/Data on /data/nfs/znas/Data type zfs (rw,noatime,xattr,posixacl,casesensitive)
vault/docker on /data/nfs/znas/Docker type zfs (rw,noatime,xattr,posixacl,casesensitive)
vault/Green on /data/nfs/znas/Green type zfs (rw,noatime,xattr,posixacl,casesensitive)
vault on /data/nfs/znas/Grimoire type zfs (rw,noatime,xattr,posixacl,casesensitive)
vault/Photos on /data/nfs/znas/Photos type zfs (rw,noatime,xattr,posixacl,casesensitive)
vault on /data/nfs/znas/Pocket type zfs (rw,noatime,xattr,posixacl,casesensitive)
vault/Data/media_books on /data/nfs/znas/Data/media/books type zfs (rw,noatime,xattr,posixacl,casesensitive)
vault/Data/media_comics on /data/nfs/znas/Data/media/comics type zfs (rw,noatime,xattr,posixacl,casesensitive)
vault/pocket/green/media on /data/nfs/znas/Green/Pocket type zfs (rw,noatime,xattr,posixacl,casesensitive)

Client Autofs Mounts

All Swarm nodes (including ZNAS) use autofs to mount NFS exports on demand at /data/nfs/znas.

/etc/auto.master

/data/nfs /etc/auto.nfs

/etc/auto.nfs

znas -fstype=nfs4 192.168.5.10:/

Client Mounts (Live)

Node Mountpoint NFS Export Options
docker3 /data/nfs/znas 192.168.5.10:/ rw,relatime,vers=4.2,hard,proto=tcp,timeo=600,retrans=2
docker4 /data/nfs/znas/Docker 192.168.5.10:/Docker rw,relatime,vers=4.2,soft,proto=tcp,timeo=600,retrans=2
docker4 /data/nfs/znas/Grimoire 192.168.5.10:/Grimoire rw,relatime,vers=4.2,soft,proto=tcp,timeo=600,retrans=2
docker4 /data/nfs/znas/Data 192.168.5.10:/Data rw,relatime,vers=4.2,soft,proto=tcp,timeo=600,retrans=2
docker5 /data/nfs/znas 192.168.5.10:/ rw,relatime,vers=4.2,hard,fatal_neterrors=none,proto=tcp,timeo=600
docker5 /data/nfs/znas/Data/media/books 192.168.5.10:/Data/media/books rw,relatime,vers=4.2,hard,fatal_neterrors=none,proto=tcp,timeo=600

Loop-Mounted Images

None detected. losetup -a and mount | grep loop returned no results.

Pocket ZFS Sync Mechanism

Overview

The Pocket Grimoire portable system receives ZFS snapshots from ZNAS via syncoid. The sync is unidirectional (ZNAS → Pocket) and triggered manually via an n8n workflow.

Sync Targets

Source Dataset Destination Dataset Notes
vault/pocket/services/DockerVol pocket-green/services/DockerVol Primary sync target

Sync Command

syncoid -r vault/pocket/services/DockerVol pocket-green/services/DockerVol

The command is embedded in the "Gremlin Pocket Dismount" n8n workflow under the "SSH: Sync" node. No automated scheduling is configured.

Discrepancies Noted

  1. Pocket Pool Location

    • MEMORY: Pocket pool was implied to be local to Pocket Grimoire.
    • LIVE: pocket-media pool exists on ZNAS (/mnt/pm/mnt/pocket-media). The Pocket SSD is currently docked to ZNAS mid-migration.

  2. NFS Export Paths

    • MEMORY: /export/Pocket was not listed in prior documentation.
    • LIVE: /export/Pocket is explicitly exported.

  3. Autofs Client Behavior

    • MEMORY: All clients mount the full NFSv4 tree (192.168.5.10:/).
    • LIVE: docker4 mounts individual exports (/Docker, /Grimoire, /Data) rather than the full tree. This is likely a manual override for performance.
> **AUTO-GENERATED DRAFT -- NOT YET PUBLISHED.** This page was written automatically by the `@document` pilot (chat -> n8n -> Mistral via gremlin-proxy) from a mix of prior memory notes and one live SSH inspection pass. Read it carefully before manually committing it to the wiki -- do not trust it blindly, especially anything under "Discrepancies noted". ```markdown # ZNAS Filesystem Topology ## Overview ZNAS is the primary storage system for NetGrimoire, running Ubuntu with OpenZFS. It serves as the source of truth for all storage, including datasets that replicate to the Pocket Grimoire portable system. All NFS exports are ZFS datasets mounted directly under `/export/`. ZNAS mounts its own NFS exports back to itself at `/data/nfs/znas` to provide path consistency for Docker Swarm containers regardless of scheduling node. --- ## ZFS Pools and Datasets ### Pools | Pool | Size | Alloc | Free | Health | Notes | |----------------|--------|--------|--------|---------|----------------------------------------| | `vault` | 129T | 91.5T | 37.7T | ONLINE | Primary pool; two RAIDZ1 VDEVs | | `pocket-media` | 4.55T | 0 | 4.55T | ONLINE | Pocket Grimoire media pool (external) | ### Datasets (`vault` pool) All datasets mount directly to `/export/*` unless otherwise noted. | Dataset | Mountpoint | Compression | Notes | |----------------------------------|--------------------------------|-------------|----------------------------------------| | `vault` | `/export` | zstd | Root dataset | | `vault/Common` | `/export/Common` | zstd | | | `vault/Data` | `/export/Data` | lz4 | | | `vault/Data/media_books` | `/export/Data/media/books` | zstd | `nohide` NFS export | | `vault/Data/media_comics` | `/export/Data/media/comics` | zstd | `nohide` NFS export | | `vault/Docker` | `/export/Docker` | zstd | Swarm persistent volumes | | `vault/Green` | `/export/Green` | off | | | `vault/Green/Pocket` | `/export/Green/Pocket` | off | Pocket sync source | | `vault/Photos` | `/export/Photos` | zstd | | | `vault/Kopia` | `/srv/vault/kopia_repository` | zstd | Local service consumption | | `vault/NextCloud-AIO` | `/srv/NextCloud-AIO` | lz4 | Local service consumption | | `vault/backup` | `/srv/vault/backup` | zstd | Local service consumption | | `vault/immich` | `/srv/immich` | lz4 | Local service consumption | | `vault/pocket` | `none` | zstd | Root for Pocket sync datasets | | `vault/pocket/green` | `/pocket/green` | zstd | | | `vault/pocket/green/media` | `/pocket/green/media` | zstd | | | `vault/pocket/media` | `/pocket/media` | zstd | | | `vault/pocket/media/reference` | `/pocket/media/reference` | zstd | | | `vault/pocket/services` | `/pocket/services` | zstd | Docker volumes for Pocket | | `vault/pocket/services/DockerVol`| `/pocket/services/DockerVol` | zstd | Primary sync target for Pocket | --- ## NFS Exports ### Server Configuration (`/etc/exports`) NFSv4 pseudo-root is mandatory. All exports are world-readable/writable with `no_root_squash`. ```exports /export *(rw,fsid=0,no_root_squash,no_subtree_check,crossmnt) /export/Common *(fsid=4,rw,no_root_squash,no_subtree_check,insecure) /export/Data *(fsid=5,rw,no_root_squash,no_subtree_check,insecure,crossmnt) /export/Data/media/books *(fsid=51,rw,no_root_squash,no_subtree_check,insecure,nohide) /export/Data/media/comics *(fsid=52,rw,no_root_squash,no_subtree_check,insecure,nohide) /export/Docker *(fsid=29,rw,no_root_squash,sync,no_subtree_check,insecure) /export/Green *(fsid=30,rw,no_root_squash,no_subtree_check,insecure,crossmnt) /export/photos *(fsid=31,rw,no_root_squash,no_subtree_check,insecure) /export/Pocket *(fsid=32,rw,no_root_squash,no_subtree_check,insecure) /export/Grimoire *(fsid=33,rw,no_root_squash,no_subtree_check,insecure) /export/green/media/pocket *(fsid=35,rw,no_root_squash,no_subtree_check,insecure) ``` ### Loopback Mount (ZNAS) ZNAS mounts its own NFS exports back to itself at `/data/nfs/znas` for Docker Swarm path consistency. ```bash vault/Common on /data/nfs/znas/Common type zfs (rw,noatime,xattr,posixacl,casesensitive) vault/Data on /data/nfs/znas/Data type zfs (rw,noatime,xattr,posixacl,casesensitive) vault/docker on /data/nfs/znas/Docker type zfs (rw,noatime,xattr,posixacl,casesensitive) vault/Green on /data/nfs/znas/Green type zfs (rw,noatime,xattr,posixacl,casesensitive) vault on /data/nfs/znas/Grimoire type zfs (rw,noatime,xattr,posixacl,casesensitive) vault/Photos on /data/nfs/znas/Photos type zfs (rw,noatime,xattr,posixacl,casesensitive) vault on /data/nfs/znas/Pocket type zfs (rw,noatime,xattr,posixacl,casesensitive) vault/Data/media_books on /data/nfs/znas/Data/media/books type zfs (rw,noatime,xattr,posixacl,casesensitive) vault/Data/media_comics on /data/nfs/znas/Data/media/comics type zfs (rw,noatime,xattr,posixacl,casesensitive) vault/pocket/green/media on /data/nfs/znas/Green/Pocket type zfs (rw,noatime,xattr,posixacl,casesensitive) ``` --- ## Client Autofs Mounts All Swarm nodes (including ZNAS) use autofs to mount NFS exports on demand at `/data/nfs/znas`. ### `/etc/auto.master` ``` /data/nfs /etc/auto.nfs ``` ### `/etc/auto.nfs` ``` znas -fstype=nfs4 192.168.5.10:/ ``` ### Client Mounts (Live) | Node | Mountpoint | NFS Export | Options | |-----------|--------------------------------|--------------------------|-------------------------------------------------------------------------| | docker3 | `/data/nfs/znas` | `192.168.5.10:/` | `rw,relatime,vers=4.2,hard,proto=tcp,timeo=600,retrans=2` | | docker4 | `/data/nfs/znas/Docker` | `192.168.5.10:/Docker` | `rw,relatime,vers=4.2,soft,proto=tcp,timeo=600,retrans=2` | | docker4 | `/data/nfs/znas/Grimoire` | `192.168.5.10:/Grimoire` | `rw,relatime,vers=4.2,soft,proto=tcp,timeo=600,retrans=2` | | docker4 | `/data/nfs/znas/Data` | `192.168.5.10:/Data` | `rw,relatime,vers=4.2,soft,proto=tcp,timeo=600,retrans=2` | | docker5 | `/data/nfs/znas` | `192.168.5.10:/` | `rw,relatime,vers=4.2,hard,fatal_neterrors=none,proto=tcp,timeo=600` | | docker5 | `/data/nfs/znas/Data/media/books` | `192.168.5.10:/Data/media/books` | `rw,relatime,vers=4.2,hard,fatal_neterrors=none,proto=tcp,timeo=600` | --- ## Loop-Mounted Images None detected. `losetup -a` and `mount | grep loop` returned no results. --- ## Pocket ZFS Sync Mechanism ### Overview The Pocket Grimoire portable system receives ZFS snapshots from ZNAS via `syncoid`. The sync is unidirectional (ZNAS → Pocket) and triggered manually via an n8n workflow. ### Sync Targets | Source Dataset | Destination Dataset | Notes | |------------------------------------|-----------------------------------|----------------------------------------| | `vault/pocket/services/DockerVol` | `pocket-green/services/DockerVol` | Primary sync target | ### Sync Command ```bash syncoid -r vault/pocket/services/DockerVol pocket-green/services/DockerVol ``` The command is embedded in the "Gremlin Pocket Dismount" n8n workflow under the "SSH: Sync" node. No automated scheduling is configured. --- ## Discrepancies Noted 1. **Pocket Pool Location** - MEMORY: Pocket pool was implied to be local to Pocket Grimoire. - LIVE: `pocket-media` pool exists on ZNAS (`/mnt/pm/mnt/pocket-media`). The Pocket SSD is currently docked to ZNAS mid-migration. 2. **NFS Export Paths** - MEMORY: `/export/Pocket` was not listed in prior documentation. - LIVE: `/export/Pocket` is explicitly exported. 3. **Autofs Client Behavior** - MEMORY: All clients mount the full NFSv4 tree (`192.168.5.10:/`). - LIVE: `docker4` mounts individual exports (`/Docker`, `/Grimoire`, `/Data`) rather than the full tree. This is likely a manual override for performance. ```
traveler commented 2026-07-15 18:18:58 -05:00
Author
Owner
Copy link

Superseded by #16: same run fixed the pool-capacity/pocket-pool content issues found in this draft (see the ntfy/staging debug session), plus the notification-routing bug that meant this draft's ntfy ping never reached a channel you were watching. Closing in favor of #16.

Superseded by #16: same run fixed the pool-capacity/pocket-pool content issues found in this draft (see the ntfy/staging debug session), plus the notification-routing bug that meant this draft's ntfy ping never reached a channel you were watching. Closing in favor of #16.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
gremlin/pocket-multiarch-relocate-docs
gremlin/pocket-hardware-build-doc
gremlin/pocket-hw-and-ops-corrections
No results found.
Labels
Clear labels   
doc-staleness
No labels
doc-staleness
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
traveler/Netgrimoire#10
No description provided.