traveler/Netgrimoire
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

[Docs Draft] znas filesystem retry -- pending review (20260715-195324-docs-znas-8266) #13

New issue
Closed
opened 2026-07-15 14:54:20 -05:00 by traveler · 1 comment
traveler commented 2026-07-15 14:54:20 -05:00
Owner
Copy link

AUTO-GENERATED DRAFT -- NOT YET PUBLISHED. This page was written automatically by the @document pilot (chat -> n8n -> Mistral via gremlin-proxy) from a mix of prior memory notes and one live SSH inspection pass. Read it carefully before manually committing it to the wiki -- do not trust it blindly, especially anything under "Discrepancies noted".

# ZNAS Filesystem Topology

## Overview
ZNAS is the primary NAS for NetGrimoire, running Ubuntu with OpenZFS. It serves as the source of truth for all storage, including datasets that replicate to the Pocket Grimoire portable system. Storage is organized under `/export/` for NFS sharing, with select datasets mounted under `/srv/` for local service consumption.

---

## ZFS Pools and Dataset Layout

### Pools
| Pool           | Size   | Allocated | Free    | Health  | Notes                                  |
|----------------|--------|-----------|---------|---------|----------------------------------------|
| `vault`        | 129T   | 91.5T     | 37.7T   | ONLINE  | Primary storage, two RAIDZ1 VDEVs      |
| `pocket-media` | 4.55T  | 0         | 4.55T   | ONLINE  | Portable SSD pool (currently docked)   |

### Datasets (Key Paths)
All datasets mount directly to `/export/*` for NFS sharing.

#### `vault` Pool
| Dataset                     | Mountpoint                     | Purpose                          | Compression | Notes                     |
|-----------------------------|--------------------------------|----------------------------------|-------------|---------------------------|
| `vault`                     | `/export`                      | NFSv4 pseudo-root                | zstd        | fsid=0                    |
| `vault/Common`              | `/export/Common`               | Shared documents, configs        | zstd        | fsid=4                    |
| `vault/Data`                | `/export/Data`                 | General data                     | lz4         | fsid=5, crossmnt          |
| `vault/Data/media_books`    | `/export/Data/media/books`     | Books library                    | zstd        | fsid=51, nohide           |
| `vault/Data/media_comics`   | `/export/Data/media/comics`    | Comics library                   | zstd        | fsid=52, nohide           |
| `vault/Docker`              | `/export/Docker`               | Swarm volumes                    | zstd        | fsid=29                   |
| `vault/Green`               | `/export/Green`                | Green Grimoire media             | off         | fsid=30, crossmnt         |
| `vault/Green/Pocket`        | `/export/Green/Pocket`         | Pocket Grimoire sync target      | off         |                           |
| `vault/Photos`              | `/export/Photos`               | Photo library                    | zstd        | fsid=31                   |
| `vault/Kopia`               | `/srv/vault/kopia_repository`  | Backup repository                | zstd        | Local only                |
| `vault/NextCloud-AIO`       | `/srv/NextCloud-AIO`           | NextCloud data                   | lz4         | Local only                |
| `vault/immich`              | `/srv/immich`                  | Immich photo library             | lz4         | Local only                |
| `vault/backup`              | `/srv/vault/backup`            | Local backups                    | zstd        | Local only                |
| `vault/pocket`              | `none`                         | Pocket sync parent               | zstd        | No direct mount           |
| `vault/pocket/green`        | `/pocket/green`                | Green Grimoire sync source       | zstd        |                           |
| `vault/pocket/green/media`  | `/pocket/green/media`          | Media subset for Pocket          | zstd        |                           |
| `vault/pocket/media`        | `/pocket/media`                | Media subset for Pocket          | zstd        |                           |
| `vault/pocket/services`     | `/pocket/services`             | Docker volumes for Pocket        | zstd        |                           |

#### `pocket-media` Pool
| Dataset                     | Mountpoint                                      | Purpose                          | Compression |
|-----------------------------|-------------------------------------------------|----------------------------------|-------------|
| `pocket-media`              | `/mnt/pm/mnt/pocket-media`                      | Portable media root              | lz4         |
| `pocket-media/media`        | `/mnt/pm/mnt/pocket-media/media`                | Media library                    | lz4         |
| `pocket-media/media/movies` | `/mnt/pm/mnt/pocket-media/media/movies`         | Movies subset                    | lz4         |
| `pocket-media/backups`      | `/mnt/pm/mnt/pocket-media/backups`              | Portable backups                 | lz4         |

---

## NFS Exports

### Server Configuration (`/etc/exports`)
```plaintext
/export *(rw,fsid=0,no_root_squash,no_subtree_check,crossmnt)
/export/Common *(fsid=4,rw,no_root_squash,no_subtree_check,insecure)
/export/Data *(fsid=5,rw,no_root_squash,no_subtree_check,insecure,crossmnt)
/export/Data/media/books *(fsid=51,rw,no_root_squash,no_subtree_check,insecure,nohide)
/export/Data/media/comics *(fsid=52,rw,no_root_squash,no_subtree_check,insecure,nohide)
/export/Docker *(fsid=29,rw,no_root_squash,sync,no_subtree_check,insecure)
/export/Green *(fsid=30,rw,no_root_squash,no_subtree_check,insecure,crossmnt)
/export/photos *(fsid=31,rw,no_root_squash,no_subtree_check,insecure)
/export/Pocket *(fsid=32,rw,no_root_squash,no_subtree_check,insecure)
/export/Grimoire *(fsid=33,rw,no_root_squash,no_subtree_check,insecure)
/export/green/media/pocket *(fsid=35,rw,no_root_squash,no_subtree_check,insecure)

Key Notes

  • NFSv4 pseudo-root: /export with fsid=0 is mandatory for client enumeration.
  • nohide: Required for sub-datasets (books, comics) to cross ZFS boundaries.
  • crossmnt: Enables traversal into child datasets (e.g., vault/Data → vault/Data/media_books).
  • Loopback mount: ZNAS mounts its own exports at /data/nfs/znas for Docker Swarm consistency.

Autofs Mounts on Swarm Nodes

Client Configuration

All Swarm nodes (including ZNAS) use autofs to mount NFS exports on-demand at /data/nfs/znas.

/etc/auto.master

/data/nfs /etc/auto.nfs

/etc/auto.nfs

znas -fstype=nfs4 192.168.5.10:/

Mount Behavior

  • On-demand: Mounts are triggered by path access (e.g., ls /data/nfs/znas/Common).
  • Consistency: All nodes (including ZNAS) access storage via /data/nfs/znas/*, ensuring uniform paths for Swarm containers.
  • ZNAS loopback: Achieved via ZFS mounts to /data/nfs/znas/* (not NFS).

Observed Mounts (Live)

Node Mountpoint NFS Export Options
docker3 /data/nfs/znas 192.168.5.10:/ rw,relatime,vers=4.2,hard,proto=tcp
docker4 /data/nfs/znas/Docker 192.168.5.10:/Docker rw,relatime,vers=4.2,soft,proto=tcp
docker4 /data/nfs/znas/Data 192.168.5.10:/Data rw,relatime,vers=4.2,soft,proto=tcp
docker4 /data/nfs/znas/Grimoire 192.168.5.10:/Grimoire rw,relatime,vers=4.2,soft,proto=tcp
docker5 /data/nfs/znas 192.168.5.10:/ rw,relatime,vers=4.2,hard,fatal_neterrors=none,proto=tcp
docker5 /data/nfs/znas/Data/media/books 192.168.5.10:/Data/media/books rw,relatime,vers=4.2,hard,fatal_neterrors=none,proto=tcp

Loop-Mounted Images

None detected. Live inspection (losetup -a, mount | grep loop) shows no active loop devices on ZNAS.

Pocket ZFS Sync Mechanism

Overview

The Pocket Grimoire portable system synchronizes a subset of vault datasets via syncoid (part of the sanoid suite). Sync is unidirectional: vault → pocket-media.

Sync Targets

Source Dataset Destination Dataset Notes
vault/pocket/services/DockerVol pocket-media/services/DockerVol Docker volumes for portable services
vault/pocket/green/media pocket-media/media/green Media subset for Green Grimoire

Sync Command

syncoid -r vault/pocket/services/DockerVol pocket-green/services/DockerVol
  • Recursive (-r): Syncs child datasets.
  • Trigger: Executed via the "Gremlin Pocket Dismount" n8n workflow.

Current State

  • The Pocket SSD (pocket-media pool) is physically docked to ZNAS (mid-migration).
  • No separate dockerpi1 host is active; sync is dormant until the SSD is undocked.

Discrepancies Noted

  1. Dataset vault/Grimoire:

    • Memory: Not listed in prior notes.
    • Live: Exists and is exported (/export/Grimoire, fsid=33).

  2. Dataset vault/pocket:

    • Memory: Implied but not explicitly documented.
    • Live: Parent dataset with mountpoint=none; child datasets (green, media, services) are mounted.

  3. NFS Export /export/Pocket:

    • Memory: Not mentioned.
    • Live: Exists (fsid=32) but may be redundant with /export/green/media/pocket.

  4. Autofs on docker4:

    • Memory: Assumed all nodes mount the full tree (192.168.5.10:/).
    • Live: docker4 mounts individual exports (/Docker, /Data, /Grimoire) separately.

  5. Loopback Mounts:

    • Memory: Described as NFS loopback (192.168.5.10:/ mounted on ZNAS).
    • Live: ZNAS uses ZFS mounts to /data/nfs/znas/* (not NFS).
> **AUTO-GENERATED DRAFT -- NOT YET PUBLISHED.** This page was written automatically by the `@document` pilot (chat -> n8n -> Mistral via gremlin-proxy) from a mix of prior memory notes and one live SSH inspection pass. Read it carefully before manually committing it to the wiki -- do not trust it blindly, especially anything under "Discrepancies noted". ```markdown # ZNAS Filesystem Topology ## Overview ZNAS is the primary NAS for NetGrimoire, running Ubuntu with OpenZFS. It serves as the source of truth for all storage, including datasets that replicate to the Pocket Grimoire portable system. Storage is organized under `/export/` for NFS sharing, with select datasets mounted under `/srv/` for local service consumption. --- ## ZFS Pools and Dataset Layout ### Pools | Pool | Size | Allocated | Free | Health | Notes | |----------------|--------|-----------|---------|---------|----------------------------------------| | `vault` | 129T | 91.5T | 37.7T | ONLINE | Primary storage, two RAIDZ1 VDEVs | | `pocket-media` | 4.55T | 0 | 4.55T | ONLINE | Portable SSD pool (currently docked) | ### Datasets (Key Paths) All datasets mount directly to `/export/*` for NFS sharing. #### `vault` Pool | Dataset | Mountpoint | Purpose | Compression | Notes | |-----------------------------|--------------------------------|----------------------------------|-------------|---------------------------| | `vault` | `/export` | NFSv4 pseudo-root | zstd | fsid=0 | | `vault/Common` | `/export/Common` | Shared documents, configs | zstd | fsid=4 | | `vault/Data` | `/export/Data` | General data | lz4 | fsid=5, crossmnt | | `vault/Data/media_books` | `/export/Data/media/books` | Books library | zstd | fsid=51, nohide | | `vault/Data/media_comics` | `/export/Data/media/comics` | Comics library | zstd | fsid=52, nohide | | `vault/Docker` | `/export/Docker` | Swarm volumes | zstd | fsid=29 | | `vault/Green` | `/export/Green` | Green Grimoire media | off | fsid=30, crossmnt | | `vault/Green/Pocket` | `/export/Green/Pocket` | Pocket Grimoire sync target | off | | | `vault/Photos` | `/export/Photos` | Photo library | zstd | fsid=31 | | `vault/Kopia` | `/srv/vault/kopia_repository` | Backup repository | zstd | Local only | | `vault/NextCloud-AIO` | `/srv/NextCloud-AIO` | NextCloud data | lz4 | Local only | | `vault/immich` | `/srv/immich` | Immich photo library | lz4 | Local only | | `vault/backup` | `/srv/vault/backup` | Local backups | zstd | Local only | | `vault/pocket` | `none` | Pocket sync parent | zstd | No direct mount | | `vault/pocket/green` | `/pocket/green` | Green Grimoire sync source | zstd | | | `vault/pocket/green/media` | `/pocket/green/media` | Media subset for Pocket | zstd | | | `vault/pocket/media` | `/pocket/media` | Media subset for Pocket | zstd | | | `vault/pocket/services` | `/pocket/services` | Docker volumes for Pocket | zstd | | #### `pocket-media` Pool | Dataset | Mountpoint | Purpose | Compression | |-----------------------------|-------------------------------------------------|----------------------------------|-------------| | `pocket-media` | `/mnt/pm/mnt/pocket-media` | Portable media root | lz4 | | `pocket-media/media` | `/mnt/pm/mnt/pocket-media/media` | Media library | lz4 | | `pocket-media/media/movies` | `/mnt/pm/mnt/pocket-media/media/movies` | Movies subset | lz4 | | `pocket-media/backups` | `/mnt/pm/mnt/pocket-media/backups` | Portable backups | lz4 | --- ## NFS Exports ### Server Configuration (`/etc/exports`) ```plaintext /export *(rw,fsid=0,no_root_squash,no_subtree_check,crossmnt) /export/Common *(fsid=4,rw,no_root_squash,no_subtree_check,insecure) /export/Data *(fsid=5,rw,no_root_squash,no_subtree_check,insecure,crossmnt) /export/Data/media/books *(fsid=51,rw,no_root_squash,no_subtree_check,insecure,nohide) /export/Data/media/comics *(fsid=52,rw,no_root_squash,no_subtree_check,insecure,nohide) /export/Docker *(fsid=29,rw,no_root_squash,sync,no_subtree_check,insecure) /export/Green *(fsid=30,rw,no_root_squash,no_subtree_check,insecure,crossmnt) /export/photos *(fsid=31,rw,no_root_squash,no_subtree_check,insecure) /export/Pocket *(fsid=32,rw,no_root_squash,no_subtree_check,insecure) /export/Grimoire *(fsid=33,rw,no_root_squash,no_subtree_check,insecure) /export/green/media/pocket *(fsid=35,rw,no_root_squash,no_subtree_check,insecure) ``` ### Key Notes - **NFSv4 pseudo-root**: `/export` with `fsid=0` is mandatory for client enumeration. - **`nohide`**: Required for sub-datasets (`books`, `comics`) to cross ZFS boundaries. - **`crossmnt`**: Enables traversal into child datasets (e.g., `vault/Data` → `vault/Data/media_books`). - **Loopback mount**: ZNAS mounts its own exports at `/data/nfs/znas` for Docker Swarm consistency. --- ## Autofs Mounts on Swarm Nodes ### Client Configuration All Swarm nodes (including ZNAS) use autofs to mount NFS exports on-demand at `/data/nfs/znas`. #### `/etc/auto.master` ```plaintext /data/nfs /etc/auto.nfs ``` #### `/etc/auto.nfs` ```plaintext znas -fstype=nfs4 192.168.5.10:/ ``` ### Mount Behavior - **On-demand**: Mounts are triggered by path access (e.g., `ls /data/nfs/znas/Common`). - **Consistency**: All nodes (including ZNAS) access storage via `/data/nfs/znas/*`, ensuring uniform paths for Swarm containers. - **ZNAS loopback**: Achieved via ZFS mounts to `/data/nfs/znas/*` (not NFS). ### Observed Mounts (Live) | Node | Mountpoint | NFS Export | Options | |------------|--------------------------------|--------------------------|-------------------------------------------------------------------------| | docker3 | `/data/nfs/znas` | `192.168.5.10:/` | `rw,relatime,vers=4.2,hard,proto=tcp` | | docker4 | `/data/nfs/znas/Docker` | `192.168.5.10:/Docker` | `rw,relatime,vers=4.2,soft,proto=tcp` | | docker4 | `/data/nfs/znas/Data` | `192.168.5.10:/Data` | `rw,relatime,vers=4.2,soft,proto=tcp` | | docker4 | `/data/nfs/znas/Grimoire` | `192.168.5.10:/Grimoire` | `rw,relatime,vers=4.2,soft,proto=tcp` | | docker5 | `/data/nfs/znas` | `192.168.5.10:/` | `rw,relatime,vers=4.2,hard,fatal_neterrors=none,proto=tcp` | | docker5 | `/data/nfs/znas/Data/media/books` | `192.168.5.10:/Data/media/books` | `rw,relatime,vers=4.2,hard,fatal_neterrors=none,proto=tcp` | --- ## Loop-Mounted Images **None detected**. Live inspection (`losetup -a`, `mount | grep loop`) shows no active loop devices on ZNAS. --- ## Pocket ZFS Sync Mechanism ### Overview The Pocket Grimoire portable system synchronizes a subset of `vault` datasets via `syncoid` (part of the `sanoid` suite). Sync is unidirectional: `vault` → `pocket-media`. ### Sync Targets | Source Dataset | Destination Dataset | Notes | |------------------------------------|-----------------------------------------|----------------------------------------| | `vault/pocket/services/DockerVol` | `pocket-media/services/DockerVol` | Docker volumes for portable services | | `vault/pocket/green/media` | `pocket-media/media/green` | Media subset for Green Grimoire | ### Sync Command ```bash syncoid -r vault/pocket/services/DockerVol pocket-green/services/DockerVol ``` - **Recursive (`-r`)**: Syncs child datasets. - **Trigger**: Executed via the "Gremlin Pocket Dismount" n8n workflow. ### Current State - The Pocket SSD (`pocket-media` pool) is physically docked to ZNAS (mid-migration). - No separate `dockerpi1` host is active; sync is dormant until the SSD is undocked. --- ## Discrepancies Noted 1. **Dataset `vault/Grimoire`**: - **Memory**: Not listed in prior notes. - **Live**: Exists and is exported (`/export/Grimoire`, fsid=33). 2. **Dataset `vault/pocket`**: - **Memory**: Implied but not explicitly documented. - **Live**: Parent dataset with `mountpoint=none`; child datasets (`green`, `media`, `services`) are mounted. 3. **NFS Export `/export/Pocket`**: - **Memory**: Not mentioned. - **Live**: Exists (`fsid=32`) but may be redundant with `/export/green/media/pocket`. 4. **Autofs on `docker4`**: - **Memory**: Assumed all nodes mount the full tree (`192.168.5.10:/`). - **Live**: `docker4` mounts individual exports (`/Docker`, `/Data`, `/Grimoire`) separately. 5. **Loopback Mounts**: - **Memory**: Described as NFS loopback (`192.168.5.10:/` mounted on ZNAS). - **Live**: ZNAS uses ZFS mounts to `/data/nfs/znas/*` (not NFS). ```
traveler commented 2026-07-15 14:55:18 -05:00
Author
Owner
Copy link

Closing: this was a second live verification call (topic echoed literally, no separate topic routing exists yet) made while building the @document pilot, not a real doc request. Issue #10 has the canonical first successful draft.

Closing: this was a second live verification call (topic echoed literally, no separate topic routing exists yet) made while building the @document pilot, not a real doc request. Issue #10 has the canonical first successful draft.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
gremlin/pocket-multiarch-relocate-docs
gremlin/pocket-hardware-build-doc
gremlin/pocket-hw-and-ops-corrections
No results found.
Labels
Clear labels   
doc-staleness
No labels
doc-staleness
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
traveler/Netgrimoire#13
No description provided.