docs: NetBird management DNS flap leaves daemon stuck (CT124 case)

Why: CT124 observed "Connecting" from iPhone for hours while
CT124 self-reported Connected. Root cause: DNS timeout ~7h earlier
broke management gRPC, daemon didn't auto-recover after DNS returned.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-18 23:26:02 +08:00
parent a80b254a4d
commit cb1b5f4bf2
2 changed files with 89 additions and 0 deletions

View File

@@ -243,6 +243,94 @@ Docker 容器(本文部署的方式):
docker restart netbird
```
## Management 斷線後 daemon 不自癒DNS flap
### 症狀
本部署 CT124 曾發生:從 iPhone 看 CT124 一直顯示 `Connecting`,但 CT124 自己跑 `netbird status` 顯示 `Peers count: 5/5 Connected`。細看:
```
Management: Disconnected, reason: rpc error: code = DeadlineExceeded
desc = context deadline exceeded while waiting for connections to become ready
Signal: Connected
```
且其他 peer 雖然顯示 `Connected`,但 `Last WireGuard handshake: -`(從未握手)、`Connection type: Relayed`(沒做 ICE— 表示只是殘留的舊狀態,沒有實際流量。
### 根因
容器 log 往上翻會看到類似這段:
```
grpc: addrConn.createTransport failed to connect to {Addr: "netbird.timmy.us.kg:443", ...}.
Err: transport: Error while dialing: nbnet.NewDialer().DialContext:
dial tcp: lookup netbird.timmy.us.kg: i/o timeout
```
NetBird daemon 啟動後跟 management server 是 gRPC 長連線,若期間 DNS 解析短暫失敗OpenWrt dnsmasq 抖動、上游 DNS timeout 等),**daemon 的重連邏輯不一定能自癒** — 在 0.68.3 觀察到 DNS 已恢復、其他連線都好了,但 management gRPC 一直卡在失敗狀態。
後果:
- 不會收 network map 更新 → 新加入的 peer 握不到手
- 既有 peer 顯示為 Connected 但實際是前一次 session 的陰影handshake 永不更新
- Dashboard 上該 peer 看起來像「Connecting」
### 辨識方式(黃金一行)
```bash
docker exec netbird netbird status | grep -E "^Management:|^Signal:"
```
正常:
```
Management: Connected
Signal: Connected
```
壞掉:
```
Management: Disconnected, reason: ...
Signal: Connected
```
### 修復
直接重啟 daemon / 容器:
```bash
# Docker 部署
docker restart netbird
# Linux systemd
systemctl restart netbird
# macOS
sudo launchctl kickstart -k system/netbird
```
Signal 斷線通常會自動重連,但 Management 不一定 — 看到這個症狀**直接重啟是最快的**,不要試著 `netbird down && netbird up`,那不會重建 gRPC channel。
### 可選:自動 health check
若部署規模變大、手動檢查不實際,可以加一個 cron / systemd timer 每 5 分鐘掃一次:
```bash
cat > /usr/local/sbin/netbird-mgmt-watchdog.sh << 'EOF'
#!/bin/sh
if docker exec netbird netbird status 2>/dev/null | grep -q "^Management: Disconnected"; then
logger -t netbird-watchdog "Management disconnected, restarting container"
docker restart netbird
fi
EOF
chmod +x /usr/local/sbin/netbird-mgmt-watchdog.sh
# 加進 cron每 5 分鐘)
echo "*/5 * * * * root /usr/local/sbin/netbird-mgmt-watchdog.sh" > /etc/cron.d/netbird-mgmt-watchdog
```
目前本部署未啟用自動 watchdogpeer 數量還在可人工巡檢的範圍。
## 意外發現NetBird 內建 UPNP 自動打洞
觀察 CT101 的啟動日誌: