netbird-selfhosted/npm-grpc-fix.md

# 透過公網域名暴露 NetBird (含 gRPC 修復)

> 情境：原本 NetBird 部署於內網 `http://192.168.42.127`，需透過 Nginx Proxy Manager (`192.168.42.124`) 以公開域名 `https://netbird.timmy.us.kg` 對外提供服務。

## 問題現象

手機端 NetBird App 連線 `https://netbird.timmy.us.kg` 失敗：

```
login failed: failed getting Management Service public key:
rpc error: code = Internal desc = server closed the stream without sending trailers
```

### 觀察

- Dashboard 本身 `https://netbird.timmy.us.kg/` 正常（HTTP/2 200）
- OAuth2 discovery `/oauth2/.well-known/openid-configuration` 正常
- 只有 gRPC（Management / Signal）失敗

「server closed the stream without sending trailers」是反向代理不正確處理 gRPC 的典型特徵 — 多數情況是代理用了 HTTP/1.1 的 `proxy_pass` 轉送 gRPC。

## 流量路徑

```
Internet → 125.229.110.50 (HiNet WAN)
       → 192.168.42.10   (OpenWrt, L4 DNAT :443)
       → 192.168.42.124  (Nginx Proxy Manager / openresty)
       → 192.168.42.127  (Caddy → netbird-server)
```

OpenWrt 只做 L4 DNAT，與 HTTP/2 無關。問題在最末兩段：
1. **NPM** 用 `proxy_pass` + `proxy_http_version 1.1`，gRPC 走不過
2. **Caddy** 的 `:80` listener 預設只支援 HTTP/1.1，即使 NPM 改成 `grpc_pass`（HTTP/2 明文）也接不到

## 修復方案

### 1. NPM: 為 NetBird proxy host 加上 gRPC 路由

NPM 的 proxy host 設定存於 `/opt/nginx-proxy-manager/data/database.sqlite` 的 `proxy_host` 表。找到 NetBird 那筆（本例為 id=55），將 gRPC/WebSocket 路由寫入 `advanced_config`。

**設定內容** (`/tmp/netbird_advanced.conf`)：

```nginx
# NetBird: long-lived gRPC/WebSocket connections need high timeouts
client_header_timeout 1d;
client_body_timeout 1d;

# Native gRPC (Signal + Management)
location ~ ^/(signalexchange\.SignalExchange|management\.ManagementService)/ {
    grpc_pass grpc://192.168.42.127:80;
    grpc_read_timeout 1d;
    grpc_send_timeout 1d;
    grpc_socket_keepalive on;
}

# WebSocket upgrades (relay, ws-proxy)
location ~ ^/(relay|ws-proxy/) {
    proxy_pass http://192.168.42.127:80;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_read_timeout 1d;
}

# Plain HTTP routes (API + embedded IdP)
location ~ ^/(api|oauth2)/ {
    proxy_pass http://192.168.42.127:80;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}
```

#### 1a. 寫入資料庫

```bash
# 備份
cp /opt/nginx-proxy-manager/data/database.sqlite \
   /opt/nginx-proxy-manager/data/database.sqlite.bak-$(date +%F)

# 用 python 寫入（避免 shell quoting 踩到 "upgrade" 裡的雙引號）
python3 <<'PY'
import sqlite3
with open('/tmp/netbird_advanced.conf') as f:
    adv = f.read()
conn = sqlite3.connect('/opt/nginx-proxy-manager/data/database.sqlite')
conn.execute(
    'UPDATE proxy_host SET advanced_config = ?, modified_on = datetime("now") WHERE id = 55',
    (adv,))
conn.commit()
conn.close()
PY
```

#### 1b. 重新渲染 nginx config

⚠️ **注意**：NPM 只有在 UI/API 儲存 proxy host 時才會重新渲染 `/data/nginx/proxy_host/<id>.conf`，重啟容器**不會**觸發。

兩種處理方式：

- **方式 A（推薦）**：登入 NPM UI，進入該 proxy host，按一下 Save。NPM 會從 DB 重新渲染。
- **方式 B（SSH 直接處理）**：手動把 advanced_config 注入 `.conf`，reload nginx。DB 已更新，日後 NPM 正常渲染會一致。

本次採 B：

```bash
cp /opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf \
   /opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf.bak

python3 <<'PY'
path = '/opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf'
with open(path) as f: conf = f.read()
with open('/tmp/netbird_advanced.conf') as f: adv = f.read()
marker = '  location / {'
adv_block = ('\n  # --- NetBird advanced_config (injected) ---\n' +
             '\n'.join('  '+l for l in adv.splitlines()) +
             '\n  # --- end advanced_config ---\n\n')
open(path,'w').write(conf.replace(marker, adv_block + marker, 1))
PY

docker exec nginxproxymanager nginx -t
docker exec nginxproxymanager nginx -s reload
```

### 2. Caddy: 啟用 h2c 讓 :80 接受 HTTP/2 明文

NPM 的 `grpc_pass grpc://192.168.42.127:80` 會以 HTTP/2 明文連到 Caddy，但 Caddy `:80` listener 預設只支援 HTTP/1.1。在全域設定加上 `protocols h1 h2 h2c`。

編輯 `/opt/netbird/Caddyfile`：

```caddy
{
    servers :80 {
        protocols h1 h2 h2c
    }
}

:80 {
    @grpc header Content-Type application/grpc*
    reverse_proxy @grpc h2c://netbird-server:80

    @backend path /relay* /ws-proxy/* /api/* /oauth2/*
    reverse_proxy @backend netbird-server:80

    reverse_proxy /* netbird-dashboard:80
}
```

重啟：

```bash
cd /opt/netbird
docker exec netbird-caddy caddy validate --config /etc/caddy/Caddyfile
docker compose restart caddy
```

## 驗證

```bash
# gRPC 端點應回 HTTP/2 200
curl -sS -o /dev/null -w 'HTTP/%{http_version} %{http_code}\n' \
  --http2-prior-knowledge \
  -X POST -H 'content-type: application/grpc' --data-binary '' \
  https://netbird.timmy.us.kg/management.ManagementService/GetPublicKey

# OAuth2 也應維持正常
curl -sS -o /dev/null -w 'HTTP/%{http_version} %{http_code}\n' \
  https://netbird.timmy.us.kg/oauth2/.well-known/openid-configuration
```

兩者皆需回 `HTTP/2 200`。

## 為什麼不在 Caddy 那層完全繞過 NPM？

NPM 是這個家用環境所有公開域名共用的入口（處理 Let's Encrypt、存取控管、HTTP/2），只為 NetBird 改 DNAT 會讓證書續簽流程複雜化。保留現有流量路徑，只在 NPM 跟 Caddy 兩邊分別解決各自的協定協商問題，是最小侵入的做法。

## 故障排除

| 症狀 | 可能原因 | 檢查 |
|------|---------|------|
| 手機 App 仍報同樣 gRPC 錯 | NPM 55.conf 沒 reload 或 Caddy 沒重啟 | `docker exec nginxproxymanager cat /data/nginx/proxy_host/55.conf \| grep grpc_pass` |
| Caddy 報 "unknown directive protocols" | Caddy 版本太舊 | 升級到 `caddy:2.6+` |
| `grpc_pass` 回 502 | Caddy h2c 沒啟用或 netbird-server 沒起來 | `docker ps` + `docker logs netbird-caddy` |
| 改完後 NPM UI 編輯這個 proxy host 看不到 advanced config 內容 | NPM UI 有快取 | 重新整理瀏覽器 |

## 備份位置

- `/opt/nginx-proxy-manager/data/database.sqlite.bak-2026-04-18`
- `/opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf.bak`
- `/opt/netbird/Caddyfile.bak`