docs: add NetBird Network Routes (Subnet Router) guide
CT100 now routes 192.168.42.0/24 for the personal group, letting Mac/iPhone reach LAN devices that don't run NetBird (OpenWrt, Proxmox nodes, NAS, printers, cameras, etc). - IP forwarding persisted via /etc/sysctl.d/99-netbird-route.conf - Route created with masquerade=true, metric=9999 - access_control_groups restricts who can use the route - Chose CT100 over CT124 to avoid the Tailscale kernel-mode iptables collision documented in peer-deployment-ops.md Doc covers: concept, API + UI creation, verification, troubleshooting (forwarding, masquerade, LXC caveats, Tailscale interference), and security considerations for narrowing the route CIDR.
This commit is contained in:
@@ -21,6 +21,7 @@
|
||||
| [client-troubleshooting.md](./client-troubleshooting.md) | 客戶端 P2P 失敗、bufferbloat、Force Relay 等坑 |
|
||||
| [peer-deployment-ops.md](./peer-deployment-ops.md) | 在 LXC 用 Docker 部署客戶端、改名、Mac daemon 重啟 |
|
||||
| [groups-and-policies.md](./groups-and-policies.md) | Groups 與 Access Control Policies 概念、設計、API 操作 |
|
||||
| [network-routes.md](./network-routes.md) | Subnet Router：讓遠端 peer 存取 LAN 上沒裝 NetBird 的裝置 |
|
||||
| [SUMMARY.md](./SUMMARY.md) | 部署摘要與疑難排解記錄 |
|
||||
|
||||
## 快速開始
|
||||
|
||||
@@ -103,7 +103,8 @@ reopen fd 8: permission denied
|
||||
├── stun-port-conflict.md ← STUN 改用 3479 (3478 被 Headscale 佔用)
|
||||
├── client-troubleshooting.md ← 客戶端 Force Relay / bufferbloat 排錯
|
||||
├── peer-deployment-ops.md ← Docker 部署客戶端、改名、Mac daemon 重啟
|
||||
└── groups-and-policies.md ← Groups 與 Access Control 設計與操作
|
||||
├── groups-and-policies.md ← Groups 與 Access Control 設計與操作
|
||||
└── network-routes.md ← Subnet Router 讓 peer 存取 LAN 非 NetBird 裝置
|
||||
```
|
||||
|
||||
### 遠端伺服器（192.168.42.127:/opt/netbird/）
|
||||
|
||||
180
network-routes.md
Normal file
180
network-routes.md
Normal file
@@ -0,0 +1,180 @@
|
||||
# Network Routes（Subnet Router）
|
||||
|
||||
讓 NetBird peers 透過某個「帶路人」存取 LAN 上**沒裝 NetBird 的**裝置（印表機、NAS、監視器、OpenWrt 介面等）。
|
||||
|
||||
## 概念
|
||||
|
||||
NetBird 的 peer 彼此在 `100.64.0.0/10` 的 overlay 網路互通。LAN 裡的印表機、NAS 沒 NetBird 客戶端，overlay 網路看不到它們。
|
||||
|
||||
**Subnet Router** 的作法：
|
||||
|
||||
1. 挑一個在家裡的 peer 當「帶路人」(Routing Peer)
|
||||
2. 在 NetBird 後台設定「`192.168.42.0/24` 這段要透過這個 peer 走」
|
||||
3. 在帶路人上開啟 IP forwarding + masquerade
|
||||
4. 遠端 peer 的路由表拿到 `192.168.42.0/24 → <Routing Peer overlay IP>`，LAN 存取自動轉送到帶路人，由帶路人 NAT 進 LAN
|
||||
|
||||
```
|
||||
Mac (外面咖啡店)
|
||||
│
|
||||
│ NetBird tunnel
|
||||
▼
|
||||
CT100 (家裡 192.168.42.100, 帶路人)
|
||||
│
|
||||
│ Masquerade (src NAT → 192.168.42.100)
|
||||
▼
|
||||
192.168.42.10 ← OpenWrt
|
||||
192.168.42.40 ← Proxmox
|
||||
192.168.42.X ← NAS / 印表機 / 任何 LAN 裝置
|
||||
```
|
||||
|
||||
## 本部署的 Route
|
||||
|
||||
| 項目 | 值 |
|
||||
|------|-----|
|
||||
| Network ID | `home-lan` |
|
||||
| CIDR | `192.168.42.0/24` |
|
||||
| Routing Peer | `CT100` (`ct100.netbird.selfhosted`, LAN IP `192.168.42.100`) |
|
||||
| Distribution Groups | `personal` — Mac、iPhone 的路由表會拿到這條 route |
|
||||
| Access Control Groups | `personal`, `servers` |
|
||||
| Masquerade | 開 |
|
||||
| Metric | 9999（低優先度，不影響直連 peer 的流量） |
|
||||
|
||||
### 為何選 CT100 而不是 CT124
|
||||
|
||||
- **CT100** 的 Tailscale 跑 userspace 模式，沒 iptables-legacy 規則，不會跟 NetBird 的 route iptables 打架
|
||||
- **CT124** 的 Tailscale 是 kernel-mode，有個既有的 `ts-input / ts-forward` DROP 100.64.0.0/10 非 tailscale0 規則（見 `peer-deployment-ops.md`）— 當 routing peer 的話 masquerade/forwarding 鏈更容易出狀況
|
||||
|
||||
CT100 的 SSH Jump Box 角色也更符合「gateway 型」peer 的語意。
|
||||
|
||||
## 部署步驟
|
||||
|
||||
### 1. Routing Peer 上開啟 IP forwarding
|
||||
|
||||
```bash
|
||||
ssh root@192.168.42.100
|
||||
|
||||
# 即時
|
||||
sysctl -w net.ipv4.ip_forward=1
|
||||
|
||||
# 持久化
|
||||
cat > /etc/sysctl.d/99-netbird-route.conf << 'EOF'
|
||||
net.ipv4.ip_forward=1
|
||||
EOF
|
||||
sysctl -p /etc/sysctl.d/99-netbird-route.conf
|
||||
```
|
||||
|
||||
NetBird client 容器本身會 handle masquerade 相關 iptables 規則（前提：容器有 `NET_ADMIN` cap，且 `masquerade: true` 設在 route 上）。
|
||||
|
||||
### 2. 建 Network Route（API）
|
||||
|
||||
```bash
|
||||
TOKEN=<你的 PAT>
|
||||
CT100_PEER_ID="<從 GET /api/peers 查>"
|
||||
PERSONAL_GID="<personal group id>"
|
||||
SERVERS_GID="<servers group id>"
|
||||
|
||||
curl -X POST -H "Authorization: Token $TOKEN" -H "Content-Type: application/json" \
|
||||
-d "{
|
||||
\"description\": \"Home LAN via CT100 jump box\",
|
||||
\"network_id\": \"home-lan\",
|
||||
\"network\": \"192.168.42.0/24\",
|
||||
\"peer\": \"$CT100_PEER_ID\",
|
||||
\"masquerade\": true,
|
||||
\"metric\": 9999,
|
||||
\"enabled\": true,
|
||||
\"groups\": [\"$PERSONAL_GID\"],
|
||||
\"access_control_groups\": [\"$PERSONAL_GID\", \"$SERVERS_GID\"]
|
||||
}" \
|
||||
https://netbird.timmy.us.kg/api/routes | jq
|
||||
```
|
||||
|
||||
欄位說明：
|
||||
- `peer` — 帶路人 peer ID（單一 peer；若要 HA 可用 `peer_groups`）
|
||||
- `groups` — **路由分發群組**：這些群組的 peers 會在路由表拿到 `192.168.42.0/24 → 100.x.x.x(routing peer)`
|
||||
- `access_control_groups` — **存取控管群組**：誰能使用這條 route
|
||||
- `masquerade` — SNAT 給回應封包找得到路
|
||||
- `metric` — 優先度，值越大越低優先；直連 peer 的路由 metric 小，不會被 subnet route 蓋掉
|
||||
|
||||
### 3. UI 操作（替代方式）
|
||||
|
||||
Dashboard → **Network Routes** → **Add Route**
|
||||
- Network CIDR: `192.168.42.0/24`
|
||||
- Network Identifier: `home-lan`
|
||||
- Routing Peer: 選 CT100
|
||||
- Distribution Groups: 選 `personal`
|
||||
- Access Control Groups: 選 `personal`, `servers`
|
||||
- Masquerade: **on**
|
||||
- Metric: `9999`
|
||||
|
||||
## 驗證
|
||||
|
||||
### 從遠端 peer 確認路由表
|
||||
|
||||
macOS:
|
||||
```bash
|
||||
netstat -rn | grep 192.168.42
|
||||
# 192.168.42 link#39 UCS utun20 ← NetBird 介面
|
||||
```
|
||||
|
||||
Linux:
|
||||
```bash
|
||||
ip route | grep 192.168.42
|
||||
# 192.168.42.0/24 dev wt0 scope link
|
||||
```
|
||||
|
||||
### 測試 ping LAN 裝置
|
||||
|
||||
```bash
|
||||
ping 192.168.42.10 # OpenWrt
|
||||
ping 192.168.42.40 # pve-03
|
||||
```
|
||||
|
||||
延遲應該相當於「到帶路人的 NetBird 延遲 + 帶路人到 LAN 設備的延遲」，本部署約 20ms。
|
||||
|
||||
## 排錯
|
||||
|
||||
### 路由表拿到但 ping 不通
|
||||
|
||||
通常是**帶路人的 IP forwarding 沒開**。再檢查：
|
||||
|
||||
```bash
|
||||
ssh <routing peer>
|
||||
sysctl net.ipv4.ip_forward
|
||||
# 應為 1
|
||||
```
|
||||
|
||||
### Masquerade 相關
|
||||
|
||||
NetBird client 會自動設定 iptables MASQUERADE 規則。如果 LAN 設備看到源 IP 是 `100.x.x.x`（NetBird overlay）而不是 `192.168.42.100`，表示 masquerade 沒生效 — 檢查 route 設定中 `masquerade: true`，且容器有 `NET_ADMIN` cap。
|
||||
|
||||
### 帶路人是 LXC 容器時
|
||||
|
||||
有些 LXC CT（非特權、沒 `NET_ADMIN`）不能改 iptables / MASQUERADE。確認 CT config 有：
|
||||
|
||||
```
|
||||
features: nesting=1
|
||||
lxc.cgroup.devices.allow: a # 或其他合適設定
|
||||
```
|
||||
|
||||
本部署的 CT100 就是這樣配置，沒問題。
|
||||
|
||||
### 遇到 Tailscale 干擾
|
||||
|
||||
如果帶路人同時跑 Tailscale kernel-mode，`iptables-legacy` 的 `ts-forward` 會擋掉往 tailscale0 外的 forward。見 `peer-deployment-ops.md` 的 Tailscale 衝突章節。
|
||||
|
||||
**本部署避開這問題的方式：選 CT100（Tailscale userspace）當 routing peer**，不選 CT124。
|
||||
|
||||
## 安全考量
|
||||
|
||||
**Routing Peer 被攻破 = 攻擊者能透過 overlay 打進家裡所有 LAN 設備。**
|
||||
|
||||
降低風險：
|
||||
- 帶路人只開最小必要服務（本部署 CT100 就是純 jump box，表面積小）
|
||||
- 透過 `access_control_groups` 限制哪些 group 能走這條 route
|
||||
- 考慮把 route 拆細：例如 `192.168.42.10/32`（只 OpenWrt）、`192.168.42.40/30`（只 PVE 節點）各自一條 route，不要整段開
|
||||
- 未來可加上 port-specific 的 policy（例如 personal → home-lan 只開 22/443）
|
||||
|
||||
## 相關文件
|
||||
|
||||
- `groups-and-policies.md` — Network Route 用 group 控管存取
|
||||
- `peer-deployment-ops.md` — Tailscale/NetBird iptables 衝突（選 routing peer 的考量）
|
||||
Reference in New Issue
Block a user