netbird-selfhosted/netbird-selfhosted-setup.md

# NetBird Self-Hosted 安裝指南

> 環境：Proxmox LXC (CT127) / Ubuntu 24.04 / 內網 IP-only (無域名)

## 環境資訊

| 項目 | 值 |
|------|-----|
| 主機 | 192.168.42.127 |
| 系統 | Ubuntu 24.04 LTS (Noble Numbat) |
| 核心 | 6.8.12-16-pve (Proxmox VE) |
| 容器類型 | LXC (非特權) |

## 1. 安裝前置套件

```bash
apt-get update
apt-get install -y curl jq ca-certificates gnupg openssl
```

## 2. 安裝 Docker

```bash
curl -fsSL https://get.docker.com | sh
systemctl start docker
systemctl enable docker
```

## 3. 解決 LXC 容器中 Docker 無法啟動的問題

Proxmox LXC 容器中 `/proc/sys` 被掛載為唯讀，導致 runc 1.3+ 嘗試存取 `net.ipv4.ip_unprivileged_port_start` sysctl 時失敗：

```
OCI runtime create failed: runc create failed: unable to start container process:
error during container init: open sysctl net.ipv4.ip_unprivileged_port_start file:
reopen fd 8: permission denied
```

**解決方案：以 crun 取代 runc 作為 OCI runtime**

```bash
# 下載 crun 1.20
curl -sL "https://github.com/containers/crun/releases/download/1.20/crun-1.20-linux-amd64" \
  -o /usr/local/bin/crun
chmod +x /usr/local/bin/crun

# 設定 Docker 使用 crun
cat > /etc/docker/daemon.json << 'EOF'
{
  "runtimes": {
    "crun": {
      "path": "/usr/local/bin/crun"
    }
  },
  "default-runtime": "crun"
}
EOF

systemctl restart docker

# 驗證
docker run --rm hello-world
```

## 4. 建立 NetBird 設定檔

```bash
mkdir -p /opt/netbird
cd /opt/netbird
```

### 4.1 產生金鑰

```bash
RELAY_SECRET=$(openssl rand -base64 32 | sed 's/=//g')
ENCRYPTION_KEY=$(openssl rand -base64 32)

echo "RELAY_SECRET=$RELAY_SECRET"
echo "ENCRYPTION_KEY=$ENCRYPTION_KEY"
```

> 記下這兩個值，下方設定檔需要用到。

### 4.2 docker-compose.yml

```yaml
services:
  caddy:
    image: caddy:2
    container_name: netbird-caddy
    restart: unless-stopped
    networks: [netbird]
    ports:
      - "80:80"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
    logging:
      driver: "json-file"
      options:
        max-size: "500m"
        max-file: "2"

  dashboard:
    image: netbirdio/dashboard:latest
    container_name: netbird-dashboard
    restart: unless-stopped
    networks: [netbird]
    env_file:
      - ./dashboard.env
    logging:
      driver: "json-file"
      options:
        max-size: "500m"
        max-file: "2"

  netbird-server:
    image: netbirdio/netbird-server:latest
    container_name: netbird-server
    restart: unless-stopped
    networks: [netbird]
    ports:
      - "3478:3478/udp"
    volumes:
      - netbird_data:/var/lib/netbird
      - ./config.yaml:/etc/netbird/config.yaml
    command: ["--config", "/etc/netbird/config.yaml"]
    logging:
      driver: "json-file"
      options:
        max-size: "500m"
        max-file: "2"

volumes:
  netbird_data:

networks:
  netbird:
```

### 4.3 Caddyfile

Caddy 作為反向代理，將所有服務統一在 port 80：

```
:80 {
    @grpc header Content-Type application/grpc*
    reverse_proxy @grpc h2c://netbird-server:80

    @backend path /relay* /ws-proxy/* /api/* /oauth2/*
    reverse_proxy @backend netbird-server:80

    reverse_proxy /* netbird-dashboard:80
}
```

### 4.4 config.yaml

將 `<RELAY_SECRET>` 和 `<ENCRYPTION_KEY>` 替換為步驟 4.1 產生的值：

```yaml
server:
  listenAddress: ":80"
  exposedAddress: "http://192.168.42.127:80"
  stunPorts:
    - 3478
  metricsPort: 9090
  healthcheckAddress: ":9000"
  logLevel: "info"
  logFile: "console"

  authSecret: "<RELAY_SECRET>"
  dataDir: "/var/lib/netbird"

  auth:
    issuer: "http://192.168.42.127/oauth2"
    signKeyRefreshEnabled: true
    dashboardRedirectURIs:
      - "http://192.168.42.127/nb-auth"
      - "http://192.168.42.127/nb-silent-auth"
    cliRedirectURIs:
      - "http://localhost:53000/"

  reverseProxy:
    trustedHTTPProxies:
      - "0.0.0.0/0"

  store:
    engine: "sqlite"
    encryptionKey: "<ENCRYPTION_KEY>"
```

### 4.5 dashboard.env

```env
NETBIRD_MGMT_API_ENDPOINT=http://192.168.42.127
NETBIRD_MGMT_GRPC_API_ENDPOINT=http://192.168.42.127
AUTH_AUDIENCE=netbird-dashboard
AUTH_CLIENT_ID=netbird-dashboard
AUTH_CLIENT_SECRET=
AUTH_AUTHORITY=http://192.168.42.127/oauth2
USE_AUTH0=false
AUTH_SUPPORTED_SCOPES=openid profile email groups
AUTH_REDIRECT_URI=/nb-auth
AUTH_SILENT_REDIRECT_URI=/nb-silent-auth
NGINX_SSL_PORT=443
LETSENCRYPT_DOMAIN=none
```

## 5. 啟動服務

```bash
cd /opt/netbird
docker compose up -d
```

首次啟動時 netbird-server 會下載 GeoLite2 地理位置資料庫（約 60MB），視網路速度可能需要數分鐘。

驗證所有容器正常運行：

```bash
docker ps
```

預期輸出三個容器皆為 `Up` 狀態：

```
NAMES               STATUS         PORTS
netbird-caddy       Up             0.0.0.0:80->80/tcp
netbird-dashboard   Up             80/tcp, 443/tcp
netbird-server      Up             0.0.0.0:3478->3478/udp
```

驗證 OAuth2 端點：

```bash
curl -s http://127.0.0.1/oauth2/.well-known/openid-configuration | jq .issuer
# 預期輸出: "http://192.168.42.127/oauth2"
```

## 6. 使用方式

### Web 管理介面

瀏覽器打開 `http://192.168.42.127`，完成首次設定（建立管理員帳號）。

### 客戶端連線

```bash
netbird up --management-url http://192.168.42.127
```

## 服務架構

```
Client Request (:80)
       |
     Caddy (反向代理)
       |
       +-- /                          --> netbird-dashboard (Web UI)
       +-- /api/* /oauth2/*           --> netbird-server (管理 API + 內建 IdP)
       +-- /relay* /ws-proxy/*        --> netbird-server (Relay + WebSocket)
       +-- gRPC (Content-Type 判斷)   --> netbird-server (Signal + Management gRPC)
       
       STUN (:3478/udp)              --> netbird-server
```

## 開放端口

| 端口 | 協定 | 用途 |
|------|------|------|
| 80 | TCP | Dashboard + API + Signal + Relay |
| 3478 | UDP | STUN (NAT 穿透) — **本部署改用 3479；見 [stun-port-conflict.md](./stun-port-conflict.md)** |

## 常用維運指令

```bash
cd /opt/netbird

# 查看日誌
docker compose logs -f netbird-server
docker compose logs -f --tail=50

# 重啟服務
docker compose restart

# 停止服務
docker compose down

# 更新映像
docker compose pull
docker compose up -d

# 完全重置（刪除所有資料）
docker compose down --volumes
```

## 注意事項

- 此為 HTTP-only 內網部署，無 TLS 加密，不適合暴露至公網
- 如需公網存取，應配置域名並啟用 HTTPS（使用 Traefik + Let's Encrypt）
- Proxmox LXC 容器需使用 crun 替代 runc，否則 Docker 容器無法啟動
- 設定檔位於 `/opt/netbird/`，資料儲存在 Docker volume `netbird_netbird_data`