# ZeroTier Exit Gateway Setup Guide

This document covers configuring an OpenWrt router as a ZeroTier exit gateway,
including the common pitfalls encountered during development.

## Architecture

```
WiFi Client (10.11.13.x)
    │
    ▼
WIBLAN AP (phy0-ap2) ─── br-zt bridge ─── ZeroTier (ztk4jpk77j)
    │                                           │
    │                                    ZeroTier tunnel
    │                                           │
    │                                           ▼
    │                                    Exit Gateway (amsterdam)
    │                                           │
    │                                      Internet (NAT)
    │
    ├── DHCP (dnsmasq on br-zt)
    ├── DNS (dnsmasq → upstream)
    └── Policy Routing (table 100 → exit gateway)
```

## Prerequisites

### ZeroTier Network Configuration

**Critical**: Enable "Allow Ethernet Bridging" on the ZeroTier network controller
at [my.zerotier.com](https://my.zerotier.com) → network → Settings.

Without this, L2 frames (ARP, DHCP) cannot be bridged across peers, and clients
will get IPs but cannot communicate.

### ZeroTier Managed Routes

The ZeroTier network must have a managed route for the WIBLAN subnet:

```
Managed Routes → Add: 10.11.13.0/24 → (empty = auto via member)
```

This tells ZeroTier to route traffic for `10.11.13.0/24` through the exit gateway.

## Setup Script Commands

The `zt-gateway-setup` script provides these commands:

| Command | Purpose |
|---|---|
| `setup-bridge` | Create `br-zt` bridge, add ZT interface, configure firewall |
| `setup-routing` | Policy routing (tables 100/101, ip rules) |
| `setup-dhcp` | DHCP for WIBLAN clients on `br-zt` |
| `setup-wifi-ap` | Create WIBLAN WiFi AP bridged to `br-zt` |
| `setup-hotplug` | Hotplug script to re-apply routes on ifup |
| `setup-persistence` | rc.local + UCI routes for reboot survival |
| `setup-all` | Run all commands in order |

Run from LuCI UI or CLI:
```bash
/usr/sbin/zt-gateway-setup setup-all
```

## Network Configuration

### Bridge Device

```uci
config device 'br_zt'
    option type 'bridge'
    option name 'br-zt'
    list ports 'ztk4jpk77j'    # ZeroTier interface
```

**Note**: UCI section names cannot contain hyphens. Use underscores (`br_zt`)
for section names, but the actual device name uses hyphens (`br-zt`).

### Bridge Interface

```uci
config interface 'zt_wiblan'
    option proto 'static'
    option device 'br-zt'
    option ipaddr '10.11.13.1'
    option netmask '255.255.254.0'    # /23 to cover ZT (10.11.12.x) + WIBLAN (10.11.13.x)
```

**Key**: The netmask MUST be `/23` (255.255.254.0), not `/24`. The bridge needs
to be in the same subnet as the ZeroTier network (10.11.12.0/23) for ARP to work.

### ZeroTier Interface

```uci
config interface 'wbtier'
    option proto 'none'
    option device 'ztk4jpk77j'
```

**Important**: The ZT interface must keep its assigned IP even when added to
the bridge. If the IP is lost, ARP responses fail and connectivity breaks.

## Firewall Configuration (nftables fw4)

### Add Bridge Interface to LAN Zone

```uci
config zone
    option name 'lan'
    list network 'lan'
    list network 'zt_wiblan'    # Add this
```

Without this, nftables fw4's default `drop` policy blocks all traffic from `br-zt`.

### Verify

```bash
nft list chain inet fw4 input | grep br-zt
# Should show: iifname { "br-zt", ... } jump input_lan
```

## Routing Configuration

### Policy Routing Rules

```uci
config rule
    option src '10.11.13.0/24'
    option lookup '100'
    option priority '100'
```

This routes traffic FROM WIBLAN clients through table 100.

### Table 100 (Main Policy)

```uci
config route
    option interface 'brzt'           # Interface name, not device
    option target '0.0.0.0'
    option netmask '0.0.0.0'
    option gateway '10.11.12.3'       # Exit gateway IP (NOT WIBLAN_GW)
    option table '100'

config route
    option interface 'brzt'
    option target '10.11.13.0'
    option netmask '255.255.255.0'
    option table '100'
```

**Critical**: The default route in table 100 MUST point to the exit gateway IP
(e.g., `10.11.12.3`), NOT to the local bridge IP (`10.11.13.1`). Using the
local IP creates a routing loop.

## DHCP Configuration

```uci
config dhcp 'br_zt'
    option interface 'zt_wiblan'      # Interface name, NOT device name
    option start '356'                # Offset in /23: 10.11.12.0 + 356 = 10.11.13.100
    option limit '101'                # 101 addresses: 10.11.13.100 - 10.11.13.200
    option leasetime '12h'
    list dhcp_option '3,10.11.13.1'   # Gateway
    list dhcp_option '6,10.11.13.1'   # DNS (use router's dnsmasq)
```

**Key points**:
- `interface` must reference the **interface** name (`zt_wiblan`), not the
  device name (`br-zt` or `br_zt`). dnsmasq binds to interfaces, not devices.
- In a `/23` network, `start` is an offset from the network base
  (`10.11.12.0`). To get `10.11.13.100`, use offset `356` (256 + 100).
- DNS should point to the router's dnsmasq (`10.11.13.1`) for reliability.
  Direct `8.8.8.8` works but adds routing complexity.

## WiFi AP Configuration

```uci
config wifi-iface 'wifinetN'
    option device 'radio0'
    option mode 'ap'
    option ssid 'WIBLAN'
    option encryption 'psk2'
    option key 'your-key'
    option network 'zt_wiblan'       # Interface name, NOT device name
```

**Same rule as DHCP**: `network` must reference the **interface** name,
not the device name.

After configuration:
```bash
wifi reload
# Verify bridge membership:
brctl show br-zt
# Should show both ztk4jpk77j and phy0-apX
```

## Common Pitfalls

### 1. Bridge Port Has No IP (ARP Fails)

**Symptom**: One-way connectivity (A→B works, B→A doesn't).

**Cause**: When the ZT interface is added to a bridge, its assigned IP can be
lost. Without an IP, the interface cannot respond to ARP requests.

**Fix**: Ensure the ZT interface keeps its assigned IP:
```bash
# Detect ZT IP:
ZT_IP=$(zerotier-cli listnetworks | awk '/OK/{for(i=6;i<=NF;i++) if($i~/\//){split($i,a,"/"); print a[1]; exit}}')
ZT_BITS=$(zerotier-cli listnetworks | awk '/OK/{for(i=6;i<=NF;i++) if($i~/\//){split($i,a,"/"); print a[2]; exit}}')

# Add to ZT interface:
ip addr add "${ZT_IP}/${ZT_BITS}" dev ztk4jpk77j
```

### 2. Wrong Interface Reference in UCI

**Symptom**: dnsmasq doesn't serve DHCP, WiFi AP not bridged.

**Cause**: Using device name (`br-zt`, `br_zt`) instead of interface name
(`zt_wiblan`) in `dhcp.*.interface` or `wireless.*.network`.

**Fix**: Always reference the **interface** name:
```bash
uci set dhcp.br_zt.interface=zt_wiblan
uci set wireless.wifinet1.network=zt_wiblan
```

### 3. Firewall Zone Missing

**Symptom**: Traffic from WIBLAN clients is silently dropped.

**Cause**: nftables fw4 has `policy drop` on INPUT/FORWARD. The bridge
interface isn't in any firewall zone.

**Fix**: Add the interface to the LAN zone:
```bash
uci add_list firewall.@zone[0].network=zt_wiblan
uci commit firewall
/etc/init.d/firewall restart
```

### 4. DHCP Range Wrong in /23

**Symptom**: Clients get IPs in wrong subnet (e.g., `10.11.12.x` instead
of `10.11.13.x`).

**Cause**: In a `/23` network, dnsmasq's `start` is an offset from the
network base (`10.11.12.0`), not from `10.11.13.0`.

**Fix**: Calculate correct offset:
```
10.11.13.100 = 10.11.12.0 + 356  →  start=356
10.11.13.200 = 10.11.12.0 + 456  →  limit=101 (356+101-1=456)
```

### 5. Table 100 Routes to Self

**Symptom**: Client traffic loops back to the gateway.

**Cause**: Table 100 default route points to `WIBLAN_GW` (local bridge IP)
instead of the exit gateway IP.

**Fix**: Route via the exit gateway:
```bash
ip route replace default via 10.11.12.3 dev ztk4jpk77j table 100
```

### 6. UCI Values Have Embedded Quotes

**Symptom**: UCI values contain literal single quotes (e.g., `'static'`
instead of `static`).

**Cause**: Shell quotes in `uci set` commands are passed as part of the value:
```bash
# WRONG:
uci set "network.zt_wiblan.proto='static'"    # Value becomes 'static'
# RIGHT:
uci set "network.zt_wiblan.proto=static"      # Value becomes static
```

### 7. ZeroTier Ethernet Bridging Disabled

**Symptom**: WiFi clients get DHCP leases but cannot reach gateway or internet.

**Cause**: ZeroTier network controller has "Allow Ethernet Bridging" disabled.
L2 frames (ARP, DHCP) cannot traverse the tunnel.

**Fix**: Enable at my.zerotier.com → network → Settings → "Allow Ethernet
Bridging".

## Verification Checklist

After setup, verify each component:

```bash
# 1. Bridge membership
brctl show br-zt
# Should show: ztk4jpk77j + phy0-apX

# 2. Bridge IPs
ip addr show br-zt
# Should show: 10.11.13.1/23

# 3. ZT interface IP
ip addr show ztk4jpk77j
# Should show: 10.11.12.x/23

# 4. Firewall zones
nft list chain inet fw4 input | grep br-zt
# Should show: iifname { "br-zt", ... } jump input_lan

# 5. Policy routing
ip rule show | grep "from 10.11.13.0/24"
# Should show: 100: from 10.11.13.0/24 lookup 100

# 6. Table 100 route
ip route show table 100
# Should show: default via 10.11.12.3 dev ztk4jpk77j

# 7. DHCP
cat /tmp/dhcp.leases | grep 10.11.13
# Should show client leases

# 8. Connectivity
ping -c 3 10.11.13.135    # From router to client
# From client: ping 10.11.13.1 (gateway)
# From client: ping 8.8.8.8 (internet via exit gateway)
```

## Environment Variables

The setup script supports these overrides for testing:

| Variable | Default | Description |
|---|---|---|
| `ZTG_BRIDGE` | `br-zt` | Bridge device name |
| `ZTG_BRIDGE_PORTS` | auto-detect | Space-separated bridge ports |
| `ZTG_WIBLAN_CIDR` | `10.11.13.0/24` | WIBLAN subnet |
| `ZTG_WIBLAN_GW` | `10.11.13.1` | WIBLAN gateway IP |
| `ZTG_BRIDGE_NETMASK` | `255.255.254.0` | Bridge netmask (/23) |
| `ZTG_TABLE_MAIN` | `100` | Main policy table |
| `ZTG_TABLE_DRAIN` | `101` | Drain policy table |
| `ZTG_WIFI_SSID` | `WIBLAN` | WiFi AP SSID |
| `ZTG_WIFI_KEY` | `zt-r0ute-2026` | WiFi AP WPA2 key |
| `ZTG_WIFI_RADIO` | auto-detect | WiFi radio device |
| `ZTG_SKIP_PERSIST` | `0` | Skip UCI persistence (testing) |