docs: add NPM gRPC fix for external HTTPS exposure

Document the fix for 'server closed the stream without sending trailers'
gRPC error when exposing NetBird via Nginx Proxy Manager:
- NPM proxy_host advanced_config with grpc_pass for Signal/Management
- Caddy h2c listener enabled via 'protocols h1 h2 h2c'
This commit is contained in:
2026-04-18 09:23:23 +08:00
parent 6e956f7334
commit 655f07a198
3 changed files with 199 additions and 1 deletions

View File

@@ -15,6 +15,7 @@
|------|------| |------|------|
| [QUICKSTART.md](./QUICKSTART.md) | 快速部署步驟(適合複製貼上執行) | | [QUICKSTART.md](./QUICKSTART.md) | 快速部署步驟(適合複製貼上執行) |
| [netbird-selfhosted-setup.md](./netbird-selfhosted-setup.md) | 完整安裝指南(含原理說明) | | [netbird-selfhosted-setup.md](./netbird-selfhosted-setup.md) | 完整安裝指南(含原理說明) |
| [npm-grpc-fix.md](./npm-grpc-fix.md) | 透過 Nginx Proxy Manager 對外公開時的 gRPC 修復 |
| [SUMMARY.md](./SUMMARY.md) | 部署摘要與疑難排解記錄 | | [SUMMARY.md](./SUMMARY.md) | 部署摘要與疑難排解記錄 |
## 🚀 快速開始 ## 🚀 快速開始

View File

@@ -80,6 +80,7 @@ reopen fd 8: permission denied
| Docker Compose `crun` runtime 未知 | `/etc/docker/daemon.json` 設定未生效 | `systemctl restart docker` | | Docker Compose `crun` runtime 未知 | `/etc/docker/daemon.json` 設定未生效 | `systemctl restart docker` |
| apt 的 crun 1.14.1 報 `unknown version specified` | crun 的 OCI spec 1.0.0 與 Docker 29.4 不相容 | 從 GitHub 下載 crun 1.20 | | apt 的 crun 1.14.1 報 `unknown version specified` | crun 的 OCI spec 1.0.0 與 Docker 29.4 不相容 | 從 GitHub 下載 crun 1.20 |
| 502 Bad Gateway所有 API 端點) | netbird-server 下載 GeoLite2 中,尚未開始監聽 port 80 | 等待 2-5 分鐘 | | 502 Bad Gateway所有 API 端點) | netbird-server 下載 GeoLite2 中,尚未開始監聽 port 80 | 等待 2-5 分鐘 |
| 手機 App`server closed the stream without sending trailers` | 透過 NPM 對外公開時NPM 用 `proxy_pass` 而非 `grpc_pass` 轉 gRPC加上 Caddy `:80` 預設不支援 h2c | NPM 加 `advanced_config`gRPC 路由)+ Caddy 全域啟用 `protocols h1 h2 h2c`;見 [npm-grpc-fix.md](./npm-grpc-fix.md) |
## 檔案位置 ## 檔案位置
@@ -89,7 +90,8 @@ reopen fd 8: permission denied
├── README.md ← 專案總覽 ├── README.md ← 專案總覽
├── QUICKSTART.md ← 快速部署步驟 ├── QUICKSTART.md ← 快速部署步驟
├── SUMMARY.md ← 本檔(部署摘要) ├── SUMMARY.md ← 本檔(部署摘要)
── netbird-selfhosted-setup.md ← 完整安裝指南 ── netbird-selfhosted-setup.md ← 完整安裝指南
└── npm-grpc-fix.md ← 經 NPM 對外公開時的 gRPC 修復
``` ```
### 遠端伺服器192.168.42.127:/opt/netbird/ ### 遠端伺服器192.168.42.127:/opt/netbird/

195
npm-grpc-fix.md Normal file
View File

@@ -0,0 +1,195 @@
# 透過公網域名暴露 NetBird (含 gRPC 修復)
> 情境:原本 NetBird 部署於內網 `http://192.168.42.127`,需透過 Nginx Proxy Manager (`192.168.42.124`) 以公開域名 `https://netbird.timmy.us.kg` 對外提供服務。
## 問題現象
手機端 NetBird App 連線 `https://netbird.timmy.us.kg` 失敗:
```
login failed: failed getting Management Service public key:
rpc error: code = Internal desc = server closed the stream without sending trailers
```
### 觀察
- Dashboard 本身 `https://netbird.timmy.us.kg/` 正常HTTP/2 200
- OAuth2 discovery `/oauth2/.well-known/openid-configuration` 正常
- 只有 gRPCManagement / Signal失敗
「server closed the stream without sending trailers」是反向代理不正確處理 gRPC 的典型特徵 — 多數情況是代理用了 HTTP/1.1 的 `proxy_pass` 轉送 gRPC。
## 流量路徑
```
Internet → 125.229.110.50 (HiNet WAN)
→ 192.168.42.10 (OpenWrt, L4 DNAT :443)
→ 192.168.42.124 (Nginx Proxy Manager / openresty)
→ 192.168.42.127 (Caddy → netbird-server)
```
OpenWrt 只做 L4 DNAT與 HTTP/2 無關。問題在最末兩段:
1. **NPM**`proxy_pass` + `proxy_http_version 1.1`gRPC 走不過
2. **Caddy**`:80` listener 預設只支援 HTTP/1.1,即使 NPM 改成 `grpc_pass`HTTP/2 明文)也接不到
## 修復方案
### 1. NPM: 為 NetBird proxy host 加上 gRPC 路由
NPM 的 proxy host 設定存於 `/opt/nginx-proxy-manager/data/database.sqlite``proxy_host` 表。找到 NetBird 那筆(本例為 id=55將 gRPC/WebSocket 路由寫入 `advanced_config`
**設定內容** (`/tmp/netbird_advanced.conf`)
```nginx
# NetBird: long-lived gRPC/WebSocket connections need high timeouts
client_header_timeout 1d;
client_body_timeout 1d;
# Native gRPC (Signal + Management)
location ~ ^/(signalexchange\.SignalExchange|management\.ManagementService)/ {
grpc_pass grpc://192.168.42.127:80;
grpc_read_timeout 1d;
grpc_send_timeout 1d;
grpc_socket_keepalive on;
}
# WebSocket upgrades (relay, ws-proxy)
location ~ ^/(relay|ws-proxy/) {
proxy_pass http://192.168.42.127:80;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 1d;
}
# Plain HTTP routes (API + embedded IdP)
location ~ ^/(api|oauth2)/ {
proxy_pass http://192.168.42.127:80;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
```
#### 1a. 寫入資料庫
```bash
# 備份
cp /opt/nginx-proxy-manager/data/database.sqlite \
/opt/nginx-proxy-manager/data/database.sqlite.bak-$(date +%F)
# 用 python 寫入(避免 shell quoting 踩到 "upgrade" 裡的雙引號)
python3 <<'PY'
import sqlite3
with open('/tmp/netbird_advanced.conf') as f:
adv = f.read()
conn = sqlite3.connect('/opt/nginx-proxy-manager/data/database.sqlite')
conn.execute(
'UPDATE proxy_host SET advanced_config = ?, modified_on = datetime("now") WHERE id = 55',
(adv,))
conn.commit()
conn.close()
PY
```
#### 1b. 重新渲染 nginx config
⚠️ **注意**NPM 只有在 UI/API 儲存 proxy host 時才會重新渲染 `/data/nginx/proxy_host/<id>.conf`,重啟容器**不會**觸發。
兩種處理方式:
- **方式 A推薦**:登入 NPM UI進入該 proxy host按一下 Save。NPM 會從 DB 重新渲染。
- **方式 BSSH 直接處理)**:手動把 advanced_config 注入 `.conf`reload nginx。DB 已更新,日後 NPM 正常渲染會一致。
本次採 B
```bash
cp /opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf \
/opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf.bak
python3 <<'PY'
path = '/opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf'
with open(path) as f: conf = f.read()
with open('/tmp/netbird_advanced.conf') as f: adv = f.read()
marker = ' location / {'
adv_block = ('\n # --- NetBird advanced_config (injected) ---\n' +
'\n'.join(' '+l for l in adv.splitlines()) +
'\n # --- end advanced_config ---\n\n')
open(path,'w').write(conf.replace(marker, adv_block + marker, 1))
PY
docker exec nginxproxymanager nginx -t
docker exec nginxproxymanager nginx -s reload
```
### 2. Caddy: 啟用 h2c 讓 :80 接受 HTTP/2 明文
NPM 的 `grpc_pass grpc://192.168.42.127:80` 會以 HTTP/2 明文連到 Caddy但 Caddy `:80` listener 預設只支援 HTTP/1.1。在全域設定加上 `protocols h1 h2 h2c`
編輯 `/opt/netbird/Caddyfile`
```caddy
{
servers :80 {
protocols h1 h2 h2c
}
}
:80 {
@grpc header Content-Type application/grpc*
reverse_proxy @grpc h2c://netbird-server:80
@backend path /relay* /ws-proxy/* /api/* /oauth2/*
reverse_proxy @backend netbird-server:80
reverse_proxy /* netbird-dashboard:80
}
```
重啟:
```bash
cd /opt/netbird
docker exec netbird-caddy caddy validate --config /etc/caddy/Caddyfile
docker compose restart caddy
```
## 驗證
```bash
# gRPC 端點應回 HTTP/2 200
curl -sS -o /dev/null -w 'HTTP/%{http_version} %{http_code}\n' \
--http2-prior-knowledge \
-X POST -H 'content-type: application/grpc' --data-binary '' \
https://netbird.timmy.us.kg/management.ManagementService/GetPublicKey
# OAuth2 也應維持正常
curl -sS -o /dev/null -w 'HTTP/%{http_version} %{http_code}\n' \
https://netbird.timmy.us.kg/oauth2/.well-known/openid-configuration
```
兩者皆需回 `HTTP/2 200`
## 為什麼不在 Caddy 那層完全繞過 NPM
NPM 是這個家用環境所有公開域名共用的入口(處理 Let's Encrypt、存取控管、HTTP/2只為 NetBird 改 DNAT 會讓證書續簽流程複雜化。保留現有流量路徑,只在 NPM 跟 Caddy 兩邊分別解決各自的協定協商問題,是最小侵入的做法。
## 故障排除
| 症狀 | 可能原因 | 檢查 |
|------|---------|------|
| 手機 App 仍報同樣 gRPC 錯 | NPM 55.conf 沒 reload 或 Caddy 沒重啟 | `docker exec nginxproxymanager cat /data/nginx/proxy_host/55.conf \| grep grpc_pass` |
| Caddy 報 "unknown directive protocols" | Caddy 版本太舊 | 升級到 `caddy:2.6+` |
| `grpc_pass` 回 502 | Caddy h2c 沒啟用或 netbird-server 沒起來 | `docker ps` + `docker logs netbird-caddy` |
| 改完後 NPM UI 編輯這個 proxy host 看不到 advanced config 內容 | NPM UI 有快取 | 重新整理瀏覽器 |
## 備份位置
- `/opt/nginx-proxy-manager/data/database.sqlite.bak-2026-04-18`
- `/opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf.bak`
- `/opt/netbird/Caddyfile.bak`