Malar Invention
1e4a46c4bf
Setup script fixes: - _cidr_to_mask: pad to 4 octets (/24 -> 255.255.255.0) - UCI quoting: remove embedded shell quotes from uci set calls - Bridge ports: auto-detect zt* interface instead of hardcoding ztabc0 - Bridge netmask: default to /23 (255.255.254.0) for ZT+WIBLAN - DHCP/WiFi AP: reference interface name (zt_wiblan) not device name (br_zt) - Firewall zone: add zt_wiblan to LAN zone for nftables fw4 - ZT IP persistence: ensure ZT-assigned IP stays on interface for ARP - Exit gateway routing: table 100/101 route via exit gateway, not self - New setup-wifi-ap subcommand for WIBLAN WiFi AP UBUS handler: - Add setup-wifi-ap to validation regex and error message Deploy task: - Auto-discover files from root/ and htdocs/ instead of hardcoded list - Clear LuCI cache before restarting services Documentation: - New docs/SETUP-GATEWAY.md with architecture, config, pitfalls, checklist - Updated docs/INSTALL.md with deploy task and setup wizard sections - Updated docs/PROGRESS.md with session log and learnings
351 lines
9.9 KiB
Markdown
351 lines
9.9 KiB
Markdown
# ZeroTier Exit Gateway Setup Guide
|
|
|
|
This document covers configuring an OpenWrt router as a ZeroTier exit gateway,
|
|
including the common pitfalls encountered during development.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
WiFi Client (10.11.13.x)
|
|
│
|
|
▼
|
|
WIBLAN AP (phy0-ap2) ─── br-zt bridge ─── ZeroTier (ztk4jpk77j)
|
|
│ │
|
|
│ ZeroTier tunnel
|
|
│ │
|
|
│ ▼
|
|
│ Exit Gateway (amsterdam)
|
|
│ │
|
|
│ Internet (NAT)
|
|
│
|
|
├── DHCP (dnsmasq on br-zt)
|
|
├── DNS (dnsmasq → upstream)
|
|
└── Policy Routing (table 100 → exit gateway)
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
### ZeroTier Network Configuration
|
|
|
|
**Critical**: Enable "Allow Ethernet Bridging" on the ZeroTier network controller
|
|
at [my.zerotier.com](https://my.zerotier.com) → network → Settings.
|
|
|
|
Without this, L2 frames (ARP, DHCP) cannot be bridged across peers, and clients
|
|
will get IPs but cannot communicate.
|
|
|
|
### ZeroTier Managed Routes
|
|
|
|
The ZeroTier network must have a managed route for the WIBLAN subnet:
|
|
|
|
```
|
|
Managed Routes → Add: 10.11.13.0/24 → (empty = auto via member)
|
|
```
|
|
|
|
This tells ZeroTier to route traffic for `10.11.13.0/24` through the exit gateway.
|
|
|
|
## Setup Script Commands
|
|
|
|
The `zt-gateway-setup` script provides these commands:
|
|
|
|
| Command | Purpose |
|
|
|---|---|
|
|
| `setup-bridge` | Create `br-zt` bridge, add ZT interface, configure firewall |
|
|
| `setup-routing` | Policy routing (tables 100/101, ip rules) |
|
|
| `setup-dhcp` | DHCP for WIBLAN clients on `br-zt` |
|
|
| `setup-wifi-ap` | Create WIBLAN WiFi AP bridged to `br-zt` |
|
|
| `setup-hotplug` | Hotplug script to re-apply routes on ifup |
|
|
| `setup-persistence` | rc.local + UCI routes for reboot survival |
|
|
| `setup-all` | Run all commands in order |
|
|
|
|
Run from LuCI UI or CLI:
|
|
```bash
|
|
/usr/sbin/zt-gateway-setup setup-all
|
|
```
|
|
|
|
## Network Configuration
|
|
|
|
### Bridge Device
|
|
|
|
```uci
|
|
config device 'br_zt'
|
|
option type 'bridge'
|
|
option name 'br-zt'
|
|
list ports 'ztk4jpk77j' # ZeroTier interface
|
|
```
|
|
|
|
**Note**: UCI section names cannot contain hyphens. Use underscores (`br_zt`)
|
|
for section names, but the actual device name uses hyphens (`br-zt`).
|
|
|
|
### Bridge Interface
|
|
|
|
```uci
|
|
config interface 'zt_wiblan'
|
|
option proto 'static'
|
|
option device 'br-zt'
|
|
option ipaddr '10.11.13.1'
|
|
option netmask '255.255.254.0' # /23 to cover ZT (10.11.12.x) + WIBLAN (10.11.13.x)
|
|
```
|
|
|
|
**Key**: The netmask MUST be `/23` (255.255.254.0), not `/24`. The bridge needs
|
|
to be in the same subnet as the ZeroTier network (10.11.12.0/23) for ARP to work.
|
|
|
|
### ZeroTier Interface
|
|
|
|
```uci
|
|
config interface 'wbtier'
|
|
option proto 'none'
|
|
option device 'ztk4jpk77j'
|
|
```
|
|
|
|
**Important**: The ZT interface must keep its assigned IP even when added to
|
|
the bridge. If the IP is lost, ARP responses fail and connectivity breaks.
|
|
|
|
## Firewall Configuration (nftables fw4)
|
|
|
|
### Add Bridge Interface to LAN Zone
|
|
|
|
```uci
|
|
config zone
|
|
option name 'lan'
|
|
list network 'lan'
|
|
list network 'zt_wiblan' # Add this
|
|
```
|
|
|
|
Without this, nftables fw4's default `drop` policy blocks all traffic from `br-zt`.
|
|
|
|
### Verify
|
|
|
|
```bash
|
|
nft list chain inet fw4 input | grep br-zt
|
|
# Should show: iifname { "br-zt", ... } jump input_lan
|
|
```
|
|
|
|
## Routing Configuration
|
|
|
|
### Policy Routing Rules
|
|
|
|
```uci
|
|
config rule
|
|
option src '10.11.13.0/24'
|
|
option lookup '100'
|
|
option priority '100'
|
|
```
|
|
|
|
This routes traffic FROM WIBLAN clients through table 100.
|
|
|
|
### Table 100 (Main Policy)
|
|
|
|
```uci
|
|
config route
|
|
option interface 'brzt' # Interface name, not device
|
|
option target '0.0.0.0'
|
|
option netmask '0.0.0.0'
|
|
option gateway '10.11.12.3' # Exit gateway IP (NOT WIBLAN_GW)
|
|
option table '100'
|
|
|
|
config route
|
|
option interface 'brzt'
|
|
option target '10.11.13.0'
|
|
option netmask '255.255.255.0'
|
|
option table '100'
|
|
```
|
|
|
|
**Critical**: The default route in table 100 MUST point to the exit gateway IP
|
|
(e.g., `10.11.12.3`), NOT to the local bridge IP (`10.11.13.1`). Using the
|
|
local IP creates a routing loop.
|
|
|
|
## DHCP Configuration
|
|
|
|
```uci
|
|
config dhcp 'br_zt'
|
|
option interface 'zt_wiblan' # Interface name, NOT device name
|
|
option start '356' # Offset in /23: 10.11.12.0 + 356 = 10.11.13.100
|
|
option limit '101' # 101 addresses: 10.11.13.100 - 10.11.13.200
|
|
option leasetime '12h'
|
|
list dhcp_option '3,10.11.13.1' # Gateway
|
|
list dhcp_option '6,10.11.13.1' # DNS (use router's dnsmasq)
|
|
```
|
|
|
|
**Key points**:
|
|
- `interface` must reference the **interface** name (`zt_wiblan`), not the
|
|
device name (`br-zt` or `br_zt`). dnsmasq binds to interfaces, not devices.
|
|
- In a `/23` network, `start` is an offset from the network base
|
|
(`10.11.12.0`). To get `10.11.13.100`, use offset `356` (256 + 100).
|
|
- DNS should point to the router's dnsmasq (`10.11.13.1`) for reliability.
|
|
Direct `8.8.8.8` works but adds routing complexity.
|
|
|
|
## WiFi AP Configuration
|
|
|
|
```uci
|
|
config wifi-iface 'wifinetN'
|
|
option device 'radio0'
|
|
option mode 'ap'
|
|
option ssid 'WIBLAN'
|
|
option encryption 'psk2'
|
|
option key 'your-key'
|
|
option network 'zt_wiblan' # Interface name, NOT device name
|
|
```
|
|
|
|
**Same rule as DHCP**: `network` must reference the **interface** name,
|
|
not the device name.
|
|
|
|
After configuration:
|
|
```bash
|
|
wifi reload
|
|
# Verify bridge membership:
|
|
brctl show br-zt
|
|
# Should show both ztk4jpk77j and phy0-apX
|
|
```
|
|
|
|
## Common Pitfalls
|
|
|
|
### 1. Bridge Port Has No IP (ARP Fails)
|
|
|
|
**Symptom**: One-way connectivity (A→B works, B→A doesn't).
|
|
|
|
**Cause**: When the ZT interface is added to a bridge, its assigned IP can be
|
|
lost. Without an IP, the interface cannot respond to ARP requests.
|
|
|
|
**Fix**: Ensure the ZT interface keeps its assigned IP:
|
|
```bash
|
|
# Detect ZT IP:
|
|
ZT_IP=$(zerotier-cli listnetworks | awk '/OK/{for(i=6;i<=NF;i++) if($i~/\//){split($i,a,"/"); print a[1]; exit}}')
|
|
ZT_BITS=$(zerotier-cli listnetworks | awk '/OK/{for(i=6;i<=NF;i++) if($i~/\//){split($i,a,"/"); print a[2]; exit}}')
|
|
|
|
# Add to ZT interface:
|
|
ip addr add "${ZT_IP}/${ZT_BITS}" dev ztk4jpk77j
|
|
```
|
|
|
|
### 2. Wrong Interface Reference in UCI
|
|
|
|
**Symptom**: dnsmasq doesn't serve DHCP, WiFi AP not bridged.
|
|
|
|
**Cause**: Using device name (`br-zt`, `br_zt`) instead of interface name
|
|
(`zt_wiblan`) in `dhcp.*.interface` or `wireless.*.network`.
|
|
|
|
**Fix**: Always reference the **interface** name:
|
|
```bash
|
|
uci set dhcp.br_zt.interface=zt_wiblan
|
|
uci set wireless.wifinet1.network=zt_wiblan
|
|
```
|
|
|
|
### 3. Firewall Zone Missing
|
|
|
|
**Symptom**: Traffic from WIBLAN clients is silently dropped.
|
|
|
|
**Cause**: nftables fw4 has `policy drop` on INPUT/FORWARD. The bridge
|
|
interface isn't in any firewall zone.
|
|
|
|
**Fix**: Add the interface to the LAN zone:
|
|
```bash
|
|
uci add_list firewall.@zone[0].network=zt_wiblan
|
|
uci commit firewall
|
|
/etc/init.d/firewall restart
|
|
```
|
|
|
|
### 4. DHCP Range Wrong in /23
|
|
|
|
**Symptom**: Clients get IPs in wrong subnet (e.g., `10.11.12.x` instead
|
|
of `10.11.13.x`).
|
|
|
|
**Cause**: In a `/23` network, dnsmasq's `start` is an offset from the
|
|
network base (`10.11.12.0`), not from `10.11.13.0`.
|
|
|
|
**Fix**: Calculate correct offset:
|
|
```
|
|
10.11.13.100 = 10.11.12.0 + 356 → start=356
|
|
10.11.13.200 = 10.11.12.0 + 456 → limit=101 (356+101-1=456)
|
|
```
|
|
|
|
### 5. Table 100 Routes to Self
|
|
|
|
**Symptom**: Client traffic loops back to the gateway.
|
|
|
|
**Cause**: Table 100 default route points to `WIBLAN_GW` (local bridge IP)
|
|
instead of the exit gateway IP.
|
|
|
|
**Fix**: Route via the exit gateway:
|
|
```bash
|
|
ip route replace default via 10.11.12.3 dev ztk4jpk77j table 100
|
|
```
|
|
|
|
### 6. UCI Values Have Embedded Quotes
|
|
|
|
**Symptom**: UCI values contain literal single quotes (e.g., `'static'`
|
|
instead of `static`).
|
|
|
|
**Cause**: Shell quotes in `uci set` commands are passed as part of the value:
|
|
```bash
|
|
# WRONG:
|
|
uci set "network.zt_wiblan.proto='static'" # Value becomes 'static'
|
|
# RIGHT:
|
|
uci set "network.zt_wiblan.proto=static" # Value becomes static
|
|
```
|
|
|
|
### 7. ZeroTier Ethernet Bridging Disabled
|
|
|
|
**Symptom**: WiFi clients get DHCP leases but cannot reach gateway or internet.
|
|
|
|
**Cause**: ZeroTier network controller has "Allow Ethernet Bridging" disabled.
|
|
L2 frames (ARP, DHCP) cannot traverse the tunnel.
|
|
|
|
**Fix**: Enable at my.zerotier.com → network → Settings → "Allow Ethernet
|
|
Bridging".
|
|
|
|
## Verification Checklist
|
|
|
|
After setup, verify each component:
|
|
|
|
```bash
|
|
# 1. Bridge membership
|
|
brctl show br-zt
|
|
# Should show: ztk4jpk77j + phy0-apX
|
|
|
|
# 2. Bridge IPs
|
|
ip addr show br-zt
|
|
# Should show: 10.11.13.1/23
|
|
|
|
# 3. ZT interface IP
|
|
ip addr show ztk4jpk77j
|
|
# Should show: 10.11.12.x/23
|
|
|
|
# 4. Firewall zones
|
|
nft list chain inet fw4 input | grep br-zt
|
|
# Should show: iifname { "br-zt", ... } jump input_lan
|
|
|
|
# 5. Policy routing
|
|
ip rule show | grep "from 10.11.13.0/24"
|
|
# Should show: 100: from 10.11.13.0/24 lookup 100
|
|
|
|
# 6. Table 100 route
|
|
ip route show table 100
|
|
# Should show: default via 10.11.12.3 dev ztk4jpk77j
|
|
|
|
# 7. DHCP
|
|
cat /tmp/dhcp.leases | grep 10.11.13
|
|
# Should show client leases
|
|
|
|
# 8. Connectivity
|
|
ping -c 3 10.11.13.135 # From router to client
|
|
# From client: ping 10.11.13.1 (gateway)
|
|
# From client: ping 8.8.8.8 (internet via exit gateway)
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
The setup script supports these overrides for testing:
|
|
|
|
| Variable | Default | Description |
|
|
|---|---|---|
|
|
| `ZTG_BRIDGE` | `br-zt` | Bridge device name |
|
|
| `ZTG_BRIDGE_PORTS` | auto-detect | Space-separated bridge ports |
|
|
| `ZTG_WIBLAN_CIDR` | `10.11.13.0/24` | WIBLAN subnet |
|
|
| `ZTG_WIBLAN_GW` | `10.11.13.1` | WIBLAN gateway IP |
|
|
| `ZTG_BRIDGE_NETMASK` | `255.255.254.0` | Bridge netmask (/23) |
|
|
| `ZTG_TABLE_MAIN` | `100` | Main policy table |
|
|
| `ZTG_TABLE_DRAIN` | `101` | Drain policy table |
|
|
| `ZTG_WIFI_SSID` | `WIBLAN` | WiFi AP SSID |
|
|
| `ZTG_WIFI_KEY` | `zt-r0ute-2026` | WiFi AP WPA2 key |
|
|
| `ZTG_WIFI_RADIO` | auto-detect | WiFi radio device |
|
|
| `ZTG_SKIP_PERSIST` | `0` | Skip UCI persistence (testing) |
|