docs: add NPM gRPC fix for external HTTPS exposure
Document the fix for 'server closed the stream without sending trailers' gRPC error when exposing NetBird via Nginx Proxy Manager: - NPM proxy_host advanced_config with grpc_pass for Signal/Management - Caddy h2c listener enabled via 'protocols h1 h2 h2c'
This commit is contained in:
@@ -15,6 +15,7 @@
|
|||||||
|------|------|
|
|------|------|
|
||||||
| [QUICKSTART.md](./QUICKSTART.md) | 快速部署步驟(適合複製貼上執行) |
|
| [QUICKSTART.md](./QUICKSTART.md) | 快速部署步驟(適合複製貼上執行) |
|
||||||
| [netbird-selfhosted-setup.md](./netbird-selfhosted-setup.md) | 完整安裝指南(含原理說明) |
|
| [netbird-selfhosted-setup.md](./netbird-selfhosted-setup.md) | 完整安裝指南(含原理說明) |
|
||||||
|
| [npm-grpc-fix.md](./npm-grpc-fix.md) | 透過 Nginx Proxy Manager 對外公開時的 gRPC 修復 |
|
||||||
| [SUMMARY.md](./SUMMARY.md) | 部署摘要與疑難排解記錄 |
|
| [SUMMARY.md](./SUMMARY.md) | 部署摘要與疑難排解記錄 |
|
||||||
|
|
||||||
## 🚀 快速開始
|
## 🚀 快速開始
|
||||||
|
|||||||
@@ -80,6 +80,7 @@ reopen fd 8: permission denied
|
|||||||
| Docker Compose `crun` runtime 未知 | `/etc/docker/daemon.json` 設定未生效 | `systemctl restart docker` |
|
| Docker Compose `crun` runtime 未知 | `/etc/docker/daemon.json` 設定未生效 | `systemctl restart docker` |
|
||||||
| apt 的 crun 1.14.1 報 `unknown version specified` | crun 的 OCI spec 1.0.0 與 Docker 29.4 不相容 | 從 GitHub 下載 crun 1.20 |
|
| apt 的 crun 1.14.1 報 `unknown version specified` | crun 的 OCI spec 1.0.0 與 Docker 29.4 不相容 | 從 GitHub 下載 crun 1.20 |
|
||||||
| 502 Bad Gateway(所有 API 端點) | netbird-server 下載 GeoLite2 中,尚未開始監聽 port 80 | 等待 2-5 分鐘 |
|
| 502 Bad Gateway(所有 API 端點) | netbird-server 下載 GeoLite2 中,尚未開始監聽 port 80 | 等待 2-5 分鐘 |
|
||||||
|
| 手機 App:`server closed the stream without sending trailers` | 透過 NPM 對外公開時,NPM 用 `proxy_pass` 而非 `grpc_pass` 轉 gRPC;加上 Caddy `:80` 預設不支援 h2c | NPM 加 `advanced_config`(gRPC 路由)+ Caddy 全域啟用 `protocols h1 h2 h2c`;見 [npm-grpc-fix.md](./npm-grpc-fix.md) |
|
||||||
|
|
||||||
## 檔案位置
|
## 檔案位置
|
||||||
|
|
||||||
@@ -89,7 +90,8 @@ reopen fd 8: permission denied
|
|||||||
├── README.md ← 專案總覽
|
├── README.md ← 專案總覽
|
||||||
├── QUICKSTART.md ← 快速部署步驟
|
├── QUICKSTART.md ← 快速部署步驟
|
||||||
├── SUMMARY.md ← 本檔(部署摘要)
|
├── SUMMARY.md ← 本檔(部署摘要)
|
||||||
└── netbird-selfhosted-setup.md ← 完整安裝指南
|
├── netbird-selfhosted-setup.md ← 完整安裝指南
|
||||||
|
└── npm-grpc-fix.md ← 經 NPM 對外公開時的 gRPC 修復
|
||||||
```
|
```
|
||||||
|
|
||||||
### 遠端伺服器(192.168.42.127:/opt/netbird/)
|
### 遠端伺服器(192.168.42.127:/opt/netbird/)
|
||||||
|
|||||||
195
npm-grpc-fix.md
Normal file
195
npm-grpc-fix.md
Normal file
@@ -0,0 +1,195 @@
|
|||||||
|
# 透過公網域名暴露 NetBird (含 gRPC 修復)
|
||||||
|
|
||||||
|
> 情境:原本 NetBird 部署於內網 `http://192.168.42.127`,需透過 Nginx Proxy Manager (`192.168.42.124`) 以公開域名 `https://netbird.timmy.us.kg` 對外提供服務。
|
||||||
|
|
||||||
|
## 問題現象
|
||||||
|
|
||||||
|
手機端 NetBird App 連線 `https://netbird.timmy.us.kg` 失敗:
|
||||||
|
|
||||||
|
```
|
||||||
|
login failed: failed getting Management Service public key:
|
||||||
|
rpc error: code = Internal desc = server closed the stream without sending trailers
|
||||||
|
```
|
||||||
|
|
||||||
|
### 觀察
|
||||||
|
|
||||||
|
- Dashboard 本身 `https://netbird.timmy.us.kg/` 正常(HTTP/2 200)
|
||||||
|
- OAuth2 discovery `/oauth2/.well-known/openid-configuration` 正常
|
||||||
|
- 只有 gRPC(Management / Signal)失敗
|
||||||
|
|
||||||
|
「server closed the stream without sending trailers」是反向代理不正確處理 gRPC 的典型特徵 — 多數情況是代理用了 HTTP/1.1 的 `proxy_pass` 轉送 gRPC。
|
||||||
|
|
||||||
|
## 流量路徑
|
||||||
|
|
||||||
|
```
|
||||||
|
Internet → 125.229.110.50 (HiNet WAN)
|
||||||
|
→ 192.168.42.10 (OpenWrt, L4 DNAT :443)
|
||||||
|
→ 192.168.42.124 (Nginx Proxy Manager / openresty)
|
||||||
|
→ 192.168.42.127 (Caddy → netbird-server)
|
||||||
|
```
|
||||||
|
|
||||||
|
OpenWrt 只做 L4 DNAT,與 HTTP/2 無關。問題在最末兩段:
|
||||||
|
1. **NPM** 用 `proxy_pass` + `proxy_http_version 1.1`,gRPC 走不過
|
||||||
|
2. **Caddy** 的 `:80` listener 預設只支援 HTTP/1.1,即使 NPM 改成 `grpc_pass`(HTTP/2 明文)也接不到
|
||||||
|
|
||||||
|
## 修復方案
|
||||||
|
|
||||||
|
### 1. NPM: 為 NetBird proxy host 加上 gRPC 路由
|
||||||
|
|
||||||
|
NPM 的 proxy host 設定存於 `/opt/nginx-proxy-manager/data/database.sqlite` 的 `proxy_host` 表。找到 NetBird 那筆(本例為 id=55),將 gRPC/WebSocket 路由寫入 `advanced_config`。
|
||||||
|
|
||||||
|
**設定內容** (`/tmp/netbird_advanced.conf`):
|
||||||
|
|
||||||
|
```nginx
|
||||||
|
# NetBird: long-lived gRPC/WebSocket connections need high timeouts
|
||||||
|
client_header_timeout 1d;
|
||||||
|
client_body_timeout 1d;
|
||||||
|
|
||||||
|
# Native gRPC (Signal + Management)
|
||||||
|
location ~ ^/(signalexchange\.SignalExchange|management\.ManagementService)/ {
|
||||||
|
grpc_pass grpc://192.168.42.127:80;
|
||||||
|
grpc_read_timeout 1d;
|
||||||
|
grpc_send_timeout 1d;
|
||||||
|
grpc_socket_keepalive on;
|
||||||
|
}
|
||||||
|
|
||||||
|
# WebSocket upgrades (relay, ws-proxy)
|
||||||
|
location ~ ^/(relay|ws-proxy/) {
|
||||||
|
proxy_pass http://192.168.42.127:80;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Upgrade $http_upgrade;
|
||||||
|
proxy_set_header Connection "upgrade";
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
proxy_read_timeout 1d;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Plain HTTP routes (API + embedded IdP)
|
||||||
|
location ~ ^/(api|oauth2)/ {
|
||||||
|
proxy_pass http://192.168.42.127:80;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 1a. 寫入資料庫
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 備份
|
||||||
|
cp /opt/nginx-proxy-manager/data/database.sqlite \
|
||||||
|
/opt/nginx-proxy-manager/data/database.sqlite.bak-$(date +%F)
|
||||||
|
|
||||||
|
# 用 python 寫入(避免 shell quoting 踩到 "upgrade" 裡的雙引號)
|
||||||
|
python3 <<'PY'
|
||||||
|
import sqlite3
|
||||||
|
with open('/tmp/netbird_advanced.conf') as f:
|
||||||
|
adv = f.read()
|
||||||
|
conn = sqlite3.connect('/opt/nginx-proxy-manager/data/database.sqlite')
|
||||||
|
conn.execute(
|
||||||
|
'UPDATE proxy_host SET advanced_config = ?, modified_on = datetime("now") WHERE id = 55',
|
||||||
|
(adv,))
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
PY
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 1b. 重新渲染 nginx config
|
||||||
|
|
||||||
|
⚠️ **注意**:NPM 只有在 UI/API 儲存 proxy host 時才會重新渲染 `/data/nginx/proxy_host/<id>.conf`,重啟容器**不會**觸發。
|
||||||
|
|
||||||
|
兩種處理方式:
|
||||||
|
|
||||||
|
- **方式 A(推薦)**:登入 NPM UI,進入該 proxy host,按一下 Save。NPM 會從 DB 重新渲染。
|
||||||
|
- **方式 B(SSH 直接處理)**:手動把 advanced_config 注入 `.conf`,reload nginx。DB 已更新,日後 NPM 正常渲染會一致。
|
||||||
|
|
||||||
|
本次採 B:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp /opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf \
|
||||||
|
/opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf.bak
|
||||||
|
|
||||||
|
python3 <<'PY'
|
||||||
|
path = '/opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf'
|
||||||
|
with open(path) as f: conf = f.read()
|
||||||
|
with open('/tmp/netbird_advanced.conf') as f: adv = f.read()
|
||||||
|
marker = ' location / {'
|
||||||
|
adv_block = ('\n # --- NetBird advanced_config (injected) ---\n' +
|
||||||
|
'\n'.join(' '+l for l in adv.splitlines()) +
|
||||||
|
'\n # --- end advanced_config ---\n\n')
|
||||||
|
open(path,'w').write(conf.replace(marker, adv_block + marker, 1))
|
||||||
|
PY
|
||||||
|
|
||||||
|
docker exec nginxproxymanager nginx -t
|
||||||
|
docker exec nginxproxymanager nginx -s reload
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Caddy: 啟用 h2c 讓 :80 接受 HTTP/2 明文
|
||||||
|
|
||||||
|
NPM 的 `grpc_pass grpc://192.168.42.127:80` 會以 HTTP/2 明文連到 Caddy,但 Caddy `:80` listener 預設只支援 HTTP/1.1。在全域設定加上 `protocols h1 h2 h2c`。
|
||||||
|
|
||||||
|
編輯 `/opt/netbird/Caddyfile`:
|
||||||
|
|
||||||
|
```caddy
|
||||||
|
{
|
||||||
|
servers :80 {
|
||||||
|
protocols h1 h2 h2c
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
:80 {
|
||||||
|
@grpc header Content-Type application/grpc*
|
||||||
|
reverse_proxy @grpc h2c://netbird-server:80
|
||||||
|
|
||||||
|
@backend path /relay* /ws-proxy/* /api/* /oauth2/*
|
||||||
|
reverse_proxy @backend netbird-server:80
|
||||||
|
|
||||||
|
reverse_proxy /* netbird-dashboard:80
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
重啟:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /opt/netbird
|
||||||
|
docker exec netbird-caddy caddy validate --config /etc/caddy/Caddyfile
|
||||||
|
docker compose restart caddy
|
||||||
|
```
|
||||||
|
|
||||||
|
## 驗證
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# gRPC 端點應回 HTTP/2 200
|
||||||
|
curl -sS -o /dev/null -w 'HTTP/%{http_version} %{http_code}\n' \
|
||||||
|
--http2-prior-knowledge \
|
||||||
|
-X POST -H 'content-type: application/grpc' --data-binary '' \
|
||||||
|
https://netbird.timmy.us.kg/management.ManagementService/GetPublicKey
|
||||||
|
|
||||||
|
# OAuth2 也應維持正常
|
||||||
|
curl -sS -o /dev/null -w 'HTTP/%{http_version} %{http_code}\n' \
|
||||||
|
https://netbird.timmy.us.kg/oauth2/.well-known/openid-configuration
|
||||||
|
```
|
||||||
|
|
||||||
|
兩者皆需回 `HTTP/2 200`。
|
||||||
|
|
||||||
|
## 為什麼不在 Caddy 那層完全繞過 NPM?
|
||||||
|
|
||||||
|
NPM 是這個家用環境所有公開域名共用的入口(處理 Let's Encrypt、存取控管、HTTP/2),只為 NetBird 改 DNAT 會讓證書續簽流程複雜化。保留現有流量路徑,只在 NPM 跟 Caddy 兩邊分別解決各自的協定協商問題,是最小侵入的做法。
|
||||||
|
|
||||||
|
## 故障排除
|
||||||
|
|
||||||
|
| 症狀 | 可能原因 | 檢查 |
|
||||||
|
|------|---------|------|
|
||||||
|
| 手機 App 仍報同樣 gRPC 錯 | NPM 55.conf 沒 reload 或 Caddy 沒重啟 | `docker exec nginxproxymanager cat /data/nginx/proxy_host/55.conf \| grep grpc_pass` |
|
||||||
|
| Caddy 報 "unknown directive protocols" | Caddy 版本太舊 | 升級到 `caddy:2.6+` |
|
||||||
|
| `grpc_pass` 回 502 | Caddy h2c 沒啟用或 netbird-server 沒起來 | `docker ps` + `docker logs netbird-caddy` |
|
||||||
|
| 改完後 NPM UI 編輯這個 proxy host 看不到 advanced config 內容 | NPM UI 有快取 | 重新整理瀏覽器 |
|
||||||
|
|
||||||
|
## 備份位置
|
||||||
|
|
||||||
|
- `/opt/nginx-proxy-manager/data/database.sqlite.bak-2026-04-18`
|
||||||
|
- `/opt/nginx-proxy-manager/data/nginx/proxy_host/55.conf.bak`
|
||||||
|
- `/opt/netbird/Caddyfile.bak`
|
||||||
Reference in New Issue
Block a user