traveler/Netgrimoire
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: update Infrastructure/Backups

Browse source
This commit is contained in:
Administrator 2026-02-13 17:10:50 +00:00 • committed by John Smith
parent 24a05fa08e
commit 84ac33004c
1 changed files with 116 additions and 99 deletions
Download patch file Download diff file Expand all files Collapse all files

215
Infrastructure/Backups.md
View file

@ -2,7 +2,7 @@
title: Setting Up Kopia title: Setting Up Kopia
description: description:
published: true published: true
date: 2026-02-11T22:31:30.667Z date: 2026-02-13T17:10:40.442Z
tags: tags:
editor: markdown editor: markdown
dateCreated: 2026-01-23T22:14:17.009Z dateCreated: 2026-01-23T22:14:17.009Z
@ -12,12 +12,12 @@ dateCreated: 2026-01-23T22:14:17.009Z
## Overview ## Overview
This system implements a two-tier backup strategy: This system implements a two-tier backup strategy using **two separate Kopia Server instances**:
1. **Primary Repository** (`/srv/vault/kopia_repository`) - Full backups of all clients 1. **Primary Repository** (`/srv/vault/kopia_repository`) - Full backups of all clients, served on port 51515
2. **Vault Repository** (`/srv/vault/backup`) - Targeted critical data backups, replicated offsite via ZFS send/receive 2. **Vault Repository** (`/srv/vault/backup`) - Targeted critical data backups, served on port 51516, replicated offsite via ZFS send/receive
The Vault repository sits on its own ZFS dataset to enable clean replication to offsite Pi systems. The Vault repository sits on its own ZFS dataset to enable clean replication to offsite Pi systems. Running two separate Kopia servers allows independent management of each repository while maintaining the same HTTPS-based client connection model for both.
--- ---
@ -26,8 +26,11 @@ The Vault repository sits on its own ZFS dataset to enable clean replication to
``` ```
Clients (docker2, cindy's desktop, etc.) Clients (docker2, cindy's desktop, etc.)
↓ ↓
├─→ Primary Backup → /srv/vault/kopia_repository (all data) ├─→ Primary Backup → Kopia Server Primary (port 51515)
└─→ Vault Backup → /srv/vault/backup (critical data only) │ → /srv/vault/kopia_repository (all data)
│
└─→ Vault Backup → Kopia Server Vault (port 51516)
→ /srv/vault/backup (critical data only)
↓ ↓
ZFS Send/Receive ZFS Send/Receive
↓ ↓
@ -57,101 +60,93 @@ zfs create zpool/vault/kopia_repository
zfs create zpool/vault/backup zfs create zpool/vault/backup
``` ```
### 2. Install Kopia Server (Docker) ### 2. Install Kopia Servers (Docker)
We run **two separate Kopia Server containers** - one for primary backups, one for vault backups.
```bash ```bash
services: # Primary repository server (port 51515)
kopia: docker run -d \
image: kopia/kopia:latest --name kopia-server-primary \
container_name: kopia --restart unless-stopped \
hostname: kopia -p 51515:51515 \
restart: unless-stopped -v /srv/vault/kopia_repository:/app/repository \
user: "1964:1964" -v /srv/vault/config-primary:/app/config \
ports: -v /srv/vault/logs-primary:/app/logs \
- 51515:51515 kopia/kopia:latest server start \
environment: --address=0.0.0.0:51515 \
PUID: 1964 --tls-generate-cert
PGID: 1964
TZ: America/Chicago
KOPIA_PASSWORD: F@lcon13
KOPIA_SERVER_USERNAME: admin
KOPIA_SERVER_PASSWORD: F@lcon13
command:
- server
- start
#- --tls-generate-cert
- --tls-cert-file=/app/cert/my.cert
- --tls-key-file=/app/cert/my.key
- --address=0.0.0.0:51515
- --server-username=admin
- --server-password=F@lcon13
volumes:
- /DockerVol/kopia/config:/app/config
- /DockerVol/kopia/cache:/app/cache
- /DockerVol/kopia/cert:/app/cert
- /srv/vault/kopia_repository:/repository
- /srv/vault/backup:/vault
- /DockerVol/kopia/logs:/app/logs
networks:
- netgrimoire
deploy:
placement:
constraints:
- node.hostname == znas
labels:
diun.enable: "true"
homepage.group: "Backup"
homepage.name: "Kopia"
homepage.icon: "kopia.png"
homepage.href: "https://kopia.netgrimoire.com"
homepage.description: "Snapshot backup and deduplication"
kuma.kopia.http.name: "Kopia Web"
kuma.kopia.http.url: "http://kopia:51515"
# Optional Caddy reverse proxy
caddy: kopia.netgrimoire.com
caddy.import: authentik
caddy.reverse_proxy: "kopia.netgrimoire.com:51515"
networks:
netgrimoire:
external: true
# Vault repository server (port 51516)
docker run -d \
--name kopia-server-vault \
--restart unless-stopped \
-p 51516:51516 \
-v /srv/vault/backup:/app/repository \
-v /srv/vault/config-vault:/app/config \
-v /srv/vault/logs-vault:/app/logs \
kopia/kopia:latest server start \
--address=0.0.0.0:51516 \
--tls-generate-cert
``` ```
**Note:** Server cert SHA256 fingerprint: `696a4999f594b5273a174fd7cab677d8dd1628f9b9d27e557daa87103ee064b2` **Get the certificate fingerprints:**
```bash
# Primary server fingerprint
docker exec kopia-server-primary kopia server status
# Vault server fingerprint
docker exec kopia-server-vault kopia server status
```
**Note:** Record both certificate fingerprints - you'll need them for client connections.
- **Primary server cert SHA256:** `696a4999f594b5273a174fd7cab677d8dd1628f9b9d27e557daa87103ee064b2`
- **Vault server cert SHA256:** *(get from command above)*
### 3. Create Kopia Repositories ### 3. Create Kopia Repositories
Each server manages its own repository. These are created during first server start, but you can initialize them manually if needed.
```bash ```bash
# Enter the container # Primary repository (usually created via GUI on first use)
docker exec -it kopia-server /bin/sh docker exec -it kopia-server-primary kopia repository create filesystem \
--path=/app/repository \
--description="Primary backup repository"
# Create primary repository (if not already done via GUI) # Vault repository
# This was created via GUI at /srv/vault/kopia_repository docker exec -it kopia-server-vault kopia repository create filesystem \
--path=/app/repository \
--description="Vault backup repository for offsite replication"
```
# Create vault repository for offsite backups **Note:** If you created the primary repository via the Kopia UI, you don't need to run the first command.
kopia repository create filesystem --path=/app/vault
### 4. Create User Accounts
Create users on each server separately.
**Primary repository users:**
```bash
# Enter primary server container
docker exec -it kopia-server-primary /bin/sh
# Create users
kopia server users add admin@docker2
kopia server users add cindy@DESKTOP-QLSVD8P
# Password for cindy: LucyDog123
# Exit container # Exit container
exit exit
``` ```
### 4. Create User Accounts **Vault repository users:**
```bash ```bash
# Enter container # Enter vault server container
docker exec -it kopia-server /bin/sh docker exec -it kopia-server-vault /bin/sh
# Primary repository users # Create users
kopia server users add --ask-password admin@docker2 kopia server users add admin@docker2-vault
kopia server users add --ask-password cindy@DESKTOP-QLSVD8P kopia server users add cindy@DESKTOP-QLSVD8P-vault
# Password for cindy: LucyDog123
# Vault repository users (for targeted backups)
kopia repository connect filesystem --path=/app/vault
kopia server users add --ask-password admin@docker2-vault
kopia server users add --ask-password cindy@DESKTOP-QLSVD8P-vault
# Use same passwords or different based on security requirements # Use same passwords or different based on security requirements
# Exit container # Exit container
@ -211,10 +206,12 @@ exit
```bash ```bash
sudo kopia --config-file=/root/.config/kopia-vault/repository.config \ sudo kopia --config-file=/root/.config/kopia-vault/repository.config \
repository connect server \ repository connect server \
--url=https://192.168.5.10:51515 \ --url=https://192.168.5.10:51516 \
--override-username=admin@docker2-vault \ --override-username=admin@docker2-vault \
--server-cert-fingerprint=696a4999f594b5273a174fd7cab677d8dd1628f9b9d27e557daa87103ee064b2 --server-cert-fingerprint=<VAULT_SERVER_CERT_FINGERPRINT>
``` ```
**Note:** Replace `<VAULT_SERVER_CERT_FINGERPRINT>` with the actual fingerprint from the vault server (see setup section).
3. **Create vault backup script** 3. **Create vault backup script**
```bash ```bash
@ -368,10 +365,12 @@ exit
```powershell ```powershell
kopia --config-file="C:\Users\cindy\.config\kopia-vault\repository.config" ` kopia --config-file="C:\Users\cindy\.config\kopia-vault\repository.config" `
repository connect server ` repository connect server `
--url=https://192.168.5.10:51515 ` --url=https://192.168.5.10:51516 `
--override-username=cindy@DESKTOP-QLSVD8P-vault ` --override-username=cindy@DESKTOP-QLSVD8P-vault `
--server-cert-fingerprint=696a4999f594b5273a174fd7cab677d8dd1628f9b9d27e557daa87103ee064b2 --server-cert-fingerprint=<VAULT_SERVER_CERT_FINGERPRINT>
``` ```
**Note:** Replace `<VAULT_SERVER_CERT_FINGERPRINT>` with the actual fingerprint from the vault server.
3. **Create vault backup script** 3. **Create vault backup script**
```powershell ```powershell
@ -609,11 +608,18 @@ kopia repository status
# Check ZFS health # Check ZFS health
zpool status zpool status
# Check both Kopia servers are running
docker ps | grep kopia
# Check vault snapshots # Check vault snapshots
zfs list -t snapshot | grep "vault/backup" zfs list -t snapshot | grep "vault/backup"
# Check replication logs # Check replication logs
tail -f /var/log/vault-replicate.log tail -f /var/log/vault-replicate.log
# View server statuses
docker exec kopia-server-primary kopia server status
docker exec kopia-server-vault kopia server status
``` ```
**On Pi Vaults:** **On Pi Vaults:**
@ -718,14 +724,22 @@ If ZNAS is unavailable, restore directly from Pi vault:
### Client can't connect to repository ### Client can't connect to repository
```bash ```bash
# Check server is running # Check both servers are running
docker ps | grep kopia docker ps | grep kopia
# Should see both kopia-server-primary and kopia-server-vault
# Check firewall # Check firewall
sudo ufw status | grep 51515 sudo ufw status | grep 51515
sudo ufw status | grep 51516
# Verify certificate fingerprint # Verify certificate fingerprints
# SERVER CERT SHA256: 696a4999f594b5273a174fd7cab677d8dd1628f9b9d27e557daa87103ee064b2 docker exec kopia-server-primary kopia server status
docker exec kopia-server-vault kopia server status
# Check server logs
docker logs kopia-server-primary
docker logs kopia-server-vault
``` ```
### Vault replication failing ### Vault replication failing
@ -834,7 +848,9 @@ zfs list -o space zpool/vault/backup
**ZNAS:** **ZNAS:**
- ZFS fileserver - ZFS fileserver
- Docker running Kopia server - Docker running **two** Kopia servers:
- **kopia-server-primary** on port 51515
- **kopia-server-vault** on port 51516
- IP: 192.168.5.10 - IP: 192.168.5.10
- Datasets: - Datasets:
- `/srv/vault/kopia_repository` (zpool/vault/kopia_repository) - Primary repository - `/srv/vault/kopia_repository` (zpool/vault/kopia_repository) - Primary repository
@ -842,11 +858,11 @@ zfs list -o space zpool/vault/backup
**Clients:** **Clients:**
- **docker2** (Linux) - Backs up /DockerVol/ - **docker2** (Linux) - Backs up /DockerVol/
- Primary: Every 3 hours - Primary: Every 3 hours → port 51515
- Vault: Daily at 3 AM (critical directories only) - Vault: Daily at 3 AM (critical directories only) → port 51516
- **DESKTOP-QLSVD8P** (Windows - Cindy's desktop) - Backs up C:\Users\cindy - **DESKTOP-QLSVD8P** (Windows - Cindy's desktop) - Backs up C:\Users\cindy
- Primary: Daily at 2 AM - Primary: Daily at 2 AM → port 51515
- Vault: Daily at 3 AM (Documents, Pictures, Important files) - Vault: Daily at 3 AM (Documents, Pictures, Important files) → port 51516
- Kopia password: LucyDog123 - Kopia password: LucyDog123
- Task Scheduler credential: Harvey123= - Task Scheduler credential: Harvey123=
@ -854,8 +870,9 @@ zfs list -o space zpool/vault/backup
- **Pi Vault 1** - Raspberry Pi with SSD (tank/vault-backup) - **Pi Vault 1** - Raspberry Pi with SSD (tank/vault-backup)
- **Pi Vault 2** - Raspberry Pi with SSD (tank/vault-backup) - **Pi Vault 2** - Raspberry Pi with SSD (tank/vault-backup)
**Server Certificate:** **Server Certificates:**
- SHA256: `696a4999f594b5273a174fd7cab677d8dd1628f9b9d27e557daa87103ee064b2` - Primary server SHA256: `696a4999f594b5273a174fd7cab677d8dd1628f9b9d27e557daa87103ee064b2`
- Vault server SHA256: *(get from `docker exec kopia-server-vault kopia server status`)*
--- ---
@ -920,4 +937,4 @@ For issues or improvements to this documentation, contact the system administrat
**Useful Resources:** **Useful Resources:**
- Kopia Documentation: https://kopia.io/docs/ - Kopia Documentation: https://kopia.io/docs/
- ZFS Administration Guide: https://openzfs.github.io/openzfs-docs/ - ZFS Administration Guide: https://openzfs.github.io/openzfs-docs/
- Kopia GitHub: https://github.com/kopia/kopia - Kopia GitHub: https://github.com/kopia/kopia