luci-app-zt-gateway/docs/SETUP-GATEWAY.md

ZeroTier Exit Gateway Setup Guide

This document covers configuring an OpenWrt router as a ZeroTier exit gateway, including the common pitfalls encountered during development.

Architecture

WiFi Client (10.11.13.x)
    │
    ▼
WIBLAN AP (phy0-ap2) ─── br-zt bridge ─── ZeroTier (ztk4jpk77j)
    │                                           │
    │                                    ZeroTier tunnel
    │                                           │
    │                                           ▼
    │                                    Exit Gateway (amsterdam)
    │                                           │
    │                                      Internet (NAT)
    │
    ├── DHCP (dnsmasq on br-zt)
    ├── DNS (dnsmasq → upstream)
    └── Policy Routing (table 100 → exit gateway)

Prerequisites

ZeroTier Network Configuration

Critical: Enable "Allow Ethernet Bridging" on the ZeroTier network controller at my.zerotier.com → network → Settings.

Without this, L2 frames (ARP, DHCP) cannot be bridged across peers, and clients will get IPs but cannot communicate.

ZeroTier Managed Routes

The ZeroTier network must have a managed route for the WIBLAN subnet:

Managed Routes → Add: 10.11.13.0/24 → (empty = auto via member)

This tells ZeroTier to route traffic for 10.11.13.0/24 through the exit gateway.

Setup Script Commands

The zt-gateway-setup script provides these commands:

Command	Purpose
setup-bridge	Create br-zt bridge, add ZT interface, configure firewall
setup-routing	Policy routing (tables 100/101, ip rules)
setup-dhcp	DHCP for WIBLAN clients on br-zt
setup-wifi-ap	Create WIBLAN WiFi AP bridged to br-zt
setup-hotplug	Hotplug script to re-apply routes on ifup
setup-persistence	rc.local + UCI routes for reboot survival
setup-all	Run all commands in order

Run from LuCI UI or CLI:

/usr/sbin/zt-gateway-setup setup-all

Network Configuration

Bridge Device

config device 'br_zt'
    option type 'bridge'
    option name 'br-zt'
    list ports 'ztk4jpk77j'    # ZeroTier interface

Note: UCI section names cannot contain hyphens. Use underscores (br_zt) for section names, but the actual device name uses hyphens (br-zt).

Bridge Interface

config interface 'zt_wiblan'
    option proto 'static'
    option device 'br-zt'
    option ipaddr '10.11.13.1'
    option netmask '255.255.254.0'    # /23 to cover ZT (10.11.12.x) + WIBLAN (10.11.13.x)

Key: The netmask MUST be /23 (255.255.254.0), not /24. The bridge needs to be in the same subnet as the ZeroTier network (10.11.12.0/23) for ARP to work.

ZeroTier Interface

config interface 'wbtier'
    option proto 'none'
    option device 'ztk4jpk77j'

Important: The ZT interface must keep its assigned IP even when added to the bridge. If the IP is lost, ARP responses fail and connectivity breaks.

Firewall Configuration (nftables fw4)

Add Bridge Interface to LAN Zone

config zone
    option name 'lan'
    list network 'lan'
    list network 'zt_wiblan'    # Add this

Without this, nftables fw4's default drop policy blocks all traffic from br-zt.

Verify

nft list chain inet fw4 input | grep br-zt
# Should show: iifname { "br-zt", ... } jump input_lan

Routing Configuration

Policy Routing Rules

config rule
    option src '10.11.13.0/24'
    option lookup '100'
    option priority '100'

This routes traffic FROM WIBLAN clients through table 100.

Table 100 (Main Policy)

config route
    option interface 'brzt'           # Interface name, not device
    option target '0.0.0.0'
    option netmask '0.0.0.0'
    option gateway '10.11.12.3'       # Exit gateway IP (NOT WIBLAN_GW)
    option table '100'

config route
    option interface 'brzt'
    option target '10.11.13.0'
    option netmask '255.255.255.0'
    option table '100'

Critical: The default route in table 100 MUST point to the exit gateway IP (e.g., 10.11.12.3), NOT to the local bridge IP (10.11.13.1). Using the local IP creates a routing loop.

DHCP Configuration

config dhcp 'br_zt'
    option interface 'zt_wiblan'      # Interface name, NOT device name
    option start '356'                # Offset in /23: 10.11.12.0 + 356 = 10.11.13.100
    option limit '101'                # 101 addresses: 10.11.13.100 - 10.11.13.200
    option leasetime '12h'
    list dhcp_option '3,10.11.13.1'   # Gateway
    list dhcp_option '6,10.11.13.1'   # DNS (use router's dnsmasq)

Key points:

• interface must reference the interface name (zt_wiblan), not the device name (br-zt or br_zt). dnsmasq binds to interfaces, not devices.
• In a /23 network, start is an offset from the network base (10.11.12.0). To get 10.11.13.100, use offset 356 (256 + 100).
• DNS should point to the router's dnsmasq (10.11.13.1) for reliability. Direct 8.8.8.8 works but adds routing complexity.

WiFi AP Configuration

config wifi-iface 'wifinetN'
    option device 'radio0'
    option mode 'ap'
    option ssid 'WIBLAN'
    option encryption 'psk2'
    option key 'your-key'
    option network 'zt_wiblan'       # Interface name, NOT device name

Same rule as DHCP: network must reference the interface name, not the device name.

After configuration:

wifi reload
# Verify bridge membership:
brctl show br-zt
# Should show both ztk4jpk77j and phy0-apX

Common Pitfalls

1. Bridge Port Has No IP (ARP Fails)

Symptom: One-way connectivity (A→B works, B→A doesn't).

Cause: When the ZT interface is added to a bridge, its assigned IP can be lost. Without an IP, the interface cannot respond to ARP requests.

Fix: Ensure the ZT interface keeps its assigned IP:

# Detect ZT IP:
ZT_IP=$(zerotier-cli listnetworks | awk '/OK/{for(i=6;i<=NF;i++) if($i~/\//){split($i,a,"/"); print a[1]; exit}}')
ZT_BITS=$(zerotier-cli listnetworks | awk '/OK/{for(i=6;i<=NF;i++) if($i~/\//){split($i,a,"/"); print a[2]; exit}}')

# Add to ZT interface:
ip addr add "${ZT_IP}/${ZT_BITS}" dev ztk4jpk77j

2. Wrong Interface Reference in UCI

Symptom: dnsmasq doesn't serve DHCP, WiFi AP not bridged.

Cause: Using device name (br-zt, br_zt) instead of interface name (zt_wiblan) in dhcp.*.interface or wireless.*.network.

Fix: Always reference the interface name:

uci set dhcp.br_zt.interface=zt_wiblan
uci set wireless.wifinet1.network=zt_wiblan

3. Firewall Zone Missing

Symptom: Traffic from WIBLAN clients is silently dropped.

Cause: nftables fw4 has policy drop on INPUT/FORWARD. The bridge interface isn't in any firewall zone.

Fix: Add the interface to the LAN zone:

uci add_list firewall.@zone[0].network=zt_wiblan
uci commit firewall
/etc/init.d/firewall restart

4. DHCP Range Wrong in /23

Symptom: Clients get IPs in wrong subnet (e.g., 10.11.12.x instead of 10.11.13.x).

Cause: In a /23 network, dnsmasq's start is an offset from the network base (10.11.12.0), not from 10.11.13.0.

Fix: Calculate correct offset:

10.11.13.100 = 10.11.12.0 + 356  →  start=356
10.11.13.200 = 10.11.12.0 + 456  →  limit=101 (356+101-1=456)

5. Table 100 Routes to Self

Symptom: Client traffic loops back to the gateway.

Cause: Table 100 default route points to WIBLAN_GW (local bridge IP) instead of the exit gateway IP.

Fix: Route via the exit gateway:

ip route replace default via 10.11.12.3 dev ztk4jpk77j table 100

6. UCI Values Have Embedded Quotes

Symptom: UCI values contain literal single quotes (e.g., 'static' instead of static).

Cause: Shell quotes in uci set commands are passed as part of the value:

# WRONG:
uci set "network.zt_wiblan.proto='static'"    # Value becomes 'static'
# RIGHT:
uci set "network.zt_wiblan.proto=static"      # Value becomes static

7. ZeroTier Ethernet Bridging Disabled

Symptom: WiFi clients get DHCP leases but cannot reach gateway or internet.

Cause: ZeroTier network controller has "Allow Ethernet Bridging" disabled. L2 frames (ARP, DHCP) cannot traverse the tunnel.

Fix: Enable at my.zerotier.com → network → Settings → "Allow Ethernet Bridging".

Verification Checklist

After setup, verify each component:

# 1. Bridge membership
brctl show br-zt
# Should show: ztk4jpk77j + phy0-apX

# 2. Bridge IPs
ip addr show br-zt
# Should show: 10.11.13.1/23

# 3. ZT interface IP
ip addr show ztk4jpk77j
# Should show: 10.11.12.x/23

# 4. Firewall zones
nft list chain inet fw4 input | grep br-zt
# Should show: iifname { "br-zt", ... } jump input_lan

# 5. Policy routing
ip rule show | grep "from 10.11.13.0/24"
# Should show: 100: from 10.11.13.0/24 lookup 100

# 6. Table 100 route
ip route show table 100
# Should show: default via 10.11.12.3 dev ztk4jpk77j

# 7. DHCP
cat /tmp/dhcp.leases | grep 10.11.13
# Should show client leases

# 8. Connectivity
ping -c 3 10.11.13.135    # From router to client
# From client: ping 10.11.13.1 (gateway)
# From client: ping 8.8.8.8 (internet via exit gateway)

Environment Variables

The setup script supports these overrides for testing:

Variable	Default	Description
ZTG_BRIDGE	br-zt	Bridge device name
ZTG_BRIDGE_PORTS	auto-detect	Space-separated bridge ports
ZTG_WIBLAN_CIDR	10.11.13.0/24	WIBLAN subnet
ZTG_WIBLAN_GW	10.11.13.1	WIBLAN gateway IP
ZTG_BRIDGE_NETMASK	255.255.254.0	Bridge netmask (/23)
ZTG_TABLE_MAIN	100	Main policy table
ZTG_TABLE_DRAIN	101	Drain policy table
ZTG_WIFI_SSID	WIBLAN	WiFi AP SSID
ZTG_WIFI_KEY	zt-r0ute-2026	WiFi AP WPA2 key
ZTG_WIFI_RADIO	auto-detect	WiFi radio device
ZTG_SKIP_PERSIST	0	Skip UCI persistence (testing)