feat: implement ZeroTier exit gateway switcher
Adds the luci-app-zt-gateway package: a LuCI app + rpcd/ucode backend +
shell switch script that reconfigures which remote ZeroTier node acts as
the internet exit gateway for WIBLAN clients (10.11.13.0/24).
- Makefile (luci.mk, arch-independent)
- UCI config skeleton with three sample gateways
- rpcd ACL + menu entry
- zt-gateway.uc rpcd backend exposing status / switch / health /
drain_status / cancel_drain ubus methods
- zt-gateway-switch shell script implementing force + graceful modes:
* force: pre-flight ping, atomic route replace, conntrack flush,
UCI/hotplug/rc.local persistence
* graceful: dual-table drain using CONNMARK fwmark 0x100 at
priority 99, background drain monitor with two-consecutive-zero
completion and timeout-forced fallback to force
- LuCI overview.js: gateway radio list, mode select, drain progress
panel, cancel-drain button, health polling
- Docker test harness (docker-compose + Dockerfile.router +
router/gw entrypoints) exercising the switch script against real
iproute2/iptables/conntrack on two simulated exit nodes
Verified against the harness: force switch, graceful drain to natural
completion, pre-flight blocking of unreachable gateways (force + graceful),
and drain-timeout forced fallback.
This commit is contained in:
16
Dockerfile.router
Normal file
16
Dockerfile.router
Normal file
@@ -0,0 +1,16 @@
|
||||
FROM archlinux:latest
|
||||
|
||||
# archlinux:latest ships iproute2, iptables, awk, sed, ip6tables, busybox
|
||||
# tools. Install conntrack-tools for the drain monitor + conntrack -D path.
|
||||
# Tolerate offline builds: the script falls back to /proc/net/nf_conntrack
|
||||
# when conntrack CLI is unavailable.
|
||||
RUN pacman --noconfirm -Sy conntrack-tools 2>/dev/null || \
|
||||
echo "[build] conntrack-tools unavailable; script will degrade gracefully"
|
||||
|
||||
# Harness + switch script
|
||||
COPY docker/router-entrypoint.sh /usr/local/bin/router-entrypoint.sh
|
||||
COPY root/usr/sbin/zt-gateway-switch /usr/sbin/zt-gateway-switch
|
||||
RUN chmod +x /usr/local/bin/router-entrypoint.sh /usr/sbin/zt-gateway-switch
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/router-entrypoint.sh"]
|
||||
CMD ["sleep", "infinity"]
|
||||
14
Makefile
Normal file
14
Makefile
Normal file
@@ -0,0 +1,14 @@
|
||||
include $(TOPDIR)/rules.mk
|
||||
|
||||
PKG_NAME:=luci-app-zt-gateway
|
||||
LUCI_TITLE:=ZeroTier Exit Gateway Switching
|
||||
LUCI_DEPENDS:=+luci-base +ucode +conntrack +iptables-mod-conntrack-extra +iptables-mod-connmark
|
||||
LUCI_PKGARCH:=all
|
||||
PKG_VERSION:=1.0.0
|
||||
PKG_RELEASE:=1
|
||||
PKG_LICENSE:=Apache-2.0
|
||||
PKG_MAINTAINER:=WIBLAN Project <root@wiblan.net>
|
||||
|
||||
include $(TOPDIR)/feeds/luci/luci.mk
|
||||
|
||||
# call BuildPackage - OpenWrt buildroot signature
|
||||
106
README.md
106
README.md
@@ -17,21 +17,107 @@ disruption.
|
||||
the new gateway immediately. Auto-falls-back to a force switch after a
|
||||
configurable timeout if drain does not complete.
|
||||
|
||||
## Status
|
||||
## Files
|
||||
|
||||
> **Planning / pre-implementation.**
|
||||
```
|
||||
luci-app-zt-gateway/
|
||||
├── Makefile # OpenWRT build (luci.mk)
|
||||
├── htdocs/luci-static/resources/view/zt-gateway/overview.js # LuCI UI
|
||||
├── root/etc/config/zt-gateway # UCI config: gateway registry + state
|
||||
├── root/usr/sbin/zt-gateway-switch # shell: actual switch logic
|
||||
├── root/usr/share/rpcd/acl.d/luci-app-zt-gateway.json # ACL
|
||||
├── root/usr/share/rpcd/ucode/zt-gateway.uc # rpcd backend (ubus)
|
||||
└── root/usr/share/luci/menu.d/luci-app-zt-gateway.json # menu entry
|
||||
└── docker-compose.yml # local Docker test harness
|
||||
└── Dockerfile.router # test image (archlinux + tools)
|
||||
└── docker/router-entrypoint.sh # brings up br-zt + baseline routing
|
||||
└── docker/gw-entrypoint.sh # simulated ZT exit node startup
|
||||
```
|
||||
|
||||
The full design — file structure, UCI config schema, rpcd backend (ucode),
|
||||
`zt-gateway-switch` shell script, LuCI frontend, switch logic for both modes,
|
||||
persistence model, health checks, rollback/safety, Docker macvlan test harness,
|
||||
and build/install instructions — is the source of truth in:
|
||||
The `zt-gateway-switch` script is callable directly from SSH for testing; the
|
||||
rpcd/ucode backend wraps it for the LuCI UI and exposes these ubus methods
|
||||
under the `zt-gateway` object:
|
||||
|
||||
[`docs/.omp/plans/zerotier-gateway-switching.md`](.omp/plans/zerotier-gateway-switching.md)
|
||||
| Method | Args | Description |
|
||||
| -------------- | ----------------------------- | ---------------------------------------------- |
|
||||
| `status` | — | active gateway, settings, gateway registry |
|
||||
| `switch` | `{ region, mode }` | execute a force or graceful switch |
|
||||
| `health` | `{ region }` | ping a gateway over br-zt, return latency |
|
||||
| `drain_status` | — | graceful drain progress + remaining flows |
|
||||
| `cancel_drain` | — | cancel drain and force-switch to the new gw |
|
||||
|
||||
(also tracked at `.omp/plans/zerotier-gateway-switching.md`)
|
||||
## UCI config
|
||||
|
||||
Implementation (Makefile, ucode backend, switch script, `overview.js`, UCI
|
||||
config, ACL, menu entry) lands in subsequent commits against this repository.
|
||||
`/etc/config/zt-gateway` holds the gateway registry and active state:
|
||||
|
||||
```
|
||||
config global 'global'
|
||||
option active_gateway 'amsterdam' # region key of active gateway
|
||||
option switch_mode 'force' # 'force' | 'graceful'
|
||||
option drain_timeout '600' # graceful drain timeout (seconds)
|
||||
option health_interval '60'
|
||||
option auto_failback '0'
|
||||
|
||||
config gateway
|
||||
option region 'amsterdam'
|
||||
option label 'Amsterdam (ocirosea641)'
|
||||
option ip '10.11.12.3'
|
||||
option default '1'
|
||||
option health_check 'ping'
|
||||
```
|
||||
|
||||
## Build & install
|
||||
|
||||
Build against the OpenWRT SDK with `luci.mk` (architecture-independent —
|
||||
`LUCI_PKGARCH:=all`). Full instructions are in
|
||||
[`.omp/plans/zerotier-gateway-switching.md`](.omp/plans/zerotier-gateway-switching.md).
|
||||
|
||||
A manual `opkg-build` cheat sheet, the `apk` install path for OpenWRT 25.12+,
|
||||
and post-install cache clearing are all documented there too.
|
||||
|
||||
## Local test harness (Docker)
|
||||
|
||||
End-to-end verification without a physical router is possible with the bundled
|
||||
`docker-compose.yml` + `Dockerfile.router` + `docker/*-entrypoint.sh`. The
|
||||
harness emulates two bridge networks (WIBLAN clients at `10.99.13.0/24` and
|
||||
ZeroTier exit gateways at `10.99.12.0/24`) plus two simulated exit nodes, and
|
||||
exercises `zt-gateway-switch` against real `iproute2`, `iptables`, and the
|
||||
host kernel's conntrack table.
|
||||
|
||||
Run it with `docker compose up -d --build`, then `docker exec openwrt-router
|
||||
zt-gateway-switch <ip> <mode> [drain_timeout]`.
|
||||
|
||||
## Verified behavior
|
||||
|
||||
The following scenarios have been exercised against the Docker harness:
|
||||
|
||||
- Force switch: `ip route show table 100` reflects the new gateway, host route
|
||||
and mwan3 table 1 return route preserved.
|
||||
- Graceful drain: table 100 → new gateway, table 101 → old gateway, fwmark
|
||||
`0x100` rule at priority 99 inserted above the priority-100 src rule,
|
||||
ESTABLISHED WIBLAN connections keep flowing through the old gateway until
|
||||
they naturally age out, after which the mangle rules, drain table, and
|
||||
pidfiles are cleaned up automatically.
|
||||
- Pre-flight: switching to an unreachable gateway aborts with exit code 2 and
|
||||
leaves routes untouched (verified for both `force` and `graceful` modes).
|
||||
- Drain timeout: a forced-fallback fires after the configured timeout,
|
||||
preserving the new gateway in table 100 and tearing down drain state.
|
||||
|
||||
## Persistence model
|
||||
|
||||
The gateway IP is recorded in five places on the router; the switch script
|
||||
updates them all atomically:
|
||||
|
||||
| Location | What |
|
||||
| --------------------------------- | -------------------------------------- |
|
||||
| Kernel: host route | `<gw_ip> dev br-zt` |
|
||||
| Kernel: table 100 default | `default via <gw_ip> dev br-zt table 100` |
|
||||
| UCI: `/etc/config/network` | `zt_gateway_host` + `zt_gateway_default` routes |
|
||||
| Hotplug: `99-zerotier-bridge` | host route + table 100 route + mwan3 table 1 return |
|
||||
| Boot: `/etc/rc.local` | all three routes |
|
||||
|
||||
The hotplug and rc.local scripts are rewritten via a dot-escaped `sed` replace
|
||||
so a reboot boots against the new gateway.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
70
docker-compose.yml
Normal file
70
docker-compose.yml
Normal file
@@ -0,0 +1,70 @@
|
||||
services:
|
||||
openwrt-router:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.router
|
||||
image: zt-gateway-router:dev
|
||||
container_name: openwrt-router
|
||||
privileged: true
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
- SYS_ADMIN
|
||||
networks:
|
||||
zt-gateway-lan:
|
||||
ipv4_address: "10.99.13.1"
|
||||
zt-exit-net:
|
||||
ipv4_address: "10.99.12.1"
|
||||
environment:
|
||||
ZTG_SKIP_PERSIST: "1"
|
||||
ZTG_PING_IFACE: "br-zt"
|
||||
ZTG_WIBLAN_CIDR: "10.99.13.0/24"
|
||||
|
||||
wiblan-client:
|
||||
image: archlinux:latest
|
||||
container_name: wiblan-client
|
||||
networks:
|
||||
zt-gateway-lan:
|
||||
ipv4_address: "10.99.13.10"
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
command: ["sleep", "infinity"]
|
||||
|
||||
zt-gw-amsterdam:
|
||||
image: archlinux:latest
|
||||
container_name: zt-gw-amsterdam
|
||||
privileged: true
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
networks:
|
||||
zt-exit-net:
|
||||
ipv4_address: "10.99.12.3"
|
||||
volumes:
|
||||
- ./docker/gw-entrypoint.sh:/usr/local/bin/gw-entrypoint.sh:ro
|
||||
entrypoint: ["/bin/sh", "/usr/local/bin/gw-entrypoint.sh", "10.99.12.3", "10.99.12.1"]
|
||||
|
||||
zt-gw-tirunelveli:
|
||||
image: archlinux:latest
|
||||
container_name: zt-gw-tirunelveli
|
||||
privileged: true
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
networks:
|
||||
zt-exit-net:
|
||||
ipv4_address: "10.99.12.5"
|
||||
volumes:
|
||||
- ./docker/gw-entrypoint.sh:/usr/local/bin/gw-entrypoint.sh:ro
|
||||
entrypoint: ["/bin/sh", "/usr/local/bin/gw-entrypoint.sh", "10.99.12.5", "10.99.12.1"]
|
||||
|
||||
networks:
|
||||
zt-gateway-lan:
|
||||
driver: bridge
|
||||
ipam:
|
||||
config:
|
||||
- subnet: "10.99.13.0/24"
|
||||
gateway: "10.99.13.1"
|
||||
zt-exit-net:
|
||||
driver: bridge
|
||||
ipam:
|
||||
config:
|
||||
- subnet: "10.99.12.0/24"
|
||||
gateway: "10.99.12.1"
|
||||
21
docker/gw-entrypoint.sh
Executable file
21
docker/gw-entrypoint.sh
Executable file
@@ -0,0 +1,21 @@
|
||||
#!/bin/sh
|
||||
# gw-entrypoint.sh — shared by all simulated ZT exit gateways.
|
||||
#
|
||||
# Argument 1: this gateway's IP (within 10.11.12.0/24)
|
||||
# Argument 2: router's ZT-side IP (10.11.12.1 by default; here we use the
|
||||
# docker-compose gateway of the zt-exit-net network).
|
||||
set -eu
|
||||
my_ip="${1:?missing my_ip}"
|
||||
router_ip="${2:-10.11.12.1}"
|
||||
|
||||
echo 1 > /proc/sys/net/ipv4/ip_forward
|
||||
|
||||
# Reply to ARP for our own IP (already done by the kernel). Make sure we
|
||||
# can reach the WIBLAN subnet by routing back through the router.
|
||||
ip route replace 10.11.13.0/24 via "$router_ip" 2>/dev/null || true
|
||||
|
||||
# Simple ping responder is all we need for pre-flight tests. No NAT/NAT
|
||||
# rules required because test traffic only verifies the policy routing on
|
||||
# the router side, not end-to-end internet egress.
|
||||
echo "[gw ${my_ip}] up; routes via ${router_ip}"
|
||||
exec sleep infinity
|
||||
56
docker/router-entrypoint.sh
Executable file
56
docker/router-entrypoint.sh
Executable file
@@ -0,0 +1,56 @@
|
||||
#!/bin/sh
|
||||
# router-entrypoint.sh
|
||||
#
|
||||
# Brings up the simulated br-zt bridge that mirrors production: the
|
||||
# ZeroTier member interface (here, the container's interface on
|
||||
# zt-exit-net) is enslaved to br-zt and its IP is moved onto br-zt. This
|
||||
# makes 'ip route ... dev br-zt' actually reach 10.99.12.0/24.
|
||||
set -eu
|
||||
|
||||
WIBLAN_CIDR="${ZTG_WIBLAN_CIDR:-10.99.13.0/24}"
|
||||
|
||||
echo "[entrypoint] bringing up br-zt bridge"
|
||||
|
||||
# Find the interface holding 10.99.12.x.
|
||||
ZT_IFACE=$(ip -o -4 addr show \
|
||||
| awk '$4 ~ /^10\.99\.12\./ {print $2; exit}')
|
||||
if [ -z "$ZT_IFACE" ]; then
|
||||
echo "[entrypoint] WARNING: no interface in 10.99.12.0/24; tests will fail" >&2
|
||||
ZT_IFACE=eth1
|
||||
fi
|
||||
echo "[entrypoint] ZT-side interface: $ZT_IFACE"
|
||||
|
||||
ip link add name br-zt type bridge 2>/dev/null || true
|
||||
ip link set br-zt up
|
||||
|
||||
if [ "$ZT_IFACE" != "br-zt" ]; then
|
||||
ADDR=$(ip -o -4 addr show dev "$ZT_IFACE" \
|
||||
| awk '$4 ~ /^10\.99\.12\./ {print $4; exit}')
|
||||
if [ -n "$ADDR" ]; then
|
||||
ip addr del "$ADDR" dev "$ZT_IFACE" 2>/dev/null || true
|
||||
ip addr add "$ADDR" dev br-zt
|
||||
fi
|
||||
ip link set "$ZT_IFACE" master br-zt
|
||||
fi
|
||||
|
||||
echo 1 > /proc/sys/net/ipv4/ip_forward
|
||||
echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects 2>/dev/null || true
|
||||
|
||||
# Seed the baseline policy routing the production router boots with.
|
||||
ip rule del from "$WIBLAN_CIDR" table 100 2>/dev/null || true
|
||||
ip rule add from "$WIBLAN_CIDR" table 100 priority 100
|
||||
|
||||
# Seed the current gateway's host route + table 100 default. These mirror
|
||||
# what /etc/rc.local installs on boot in production.
|
||||
ip route replace 10.99.12.3 dev br-zt
|
||||
ip route replace default via 10.99.12.3 dev br-zt table 100
|
||||
ip route replace "$WIBLAN_CIDR" dev br-zt table 1
|
||||
|
||||
echo "[entrypoint] baseline state:"
|
||||
ip -o -4 addr show dev br-zt
|
||||
echo "--- table 100:"
|
||||
ip route show table 100
|
||||
echo "--- rule:"
|
||||
ip rule show
|
||||
|
||||
exec "$@"
|
||||
316
htdocs/luci-static/resources/view/zt-gateway/overview.js
Normal file
316
htdocs/luci-static/resources/view/zt-gateway/overview.js
Normal file
@@ -0,0 +1,316 @@
|
||||
'use strict';
|
||||
/* luci-app-zt-gateway - ZeroTier Exit Gateway switching UI
|
||||
*
|
||||
* Provides a Services -> ZeroTier Gateway page that lists all gateways from
|
||||
* /etc/config/zt-gateway, shows the active gateway, health, and drain
|
||||
* progress, and lets the user pick a target + switch mode (force|graceful).
|
||||
* Calls zt-gateway.* ubus methods exposed by
|
||||
* /usr/share/rpcd/ucode/zt-gateway.uc, which in turn invokes
|
||||
* /usr/sbin/zt-gateway-switch for the actual system changes.
|
||||
*/
|
||||
|
||||
import { view } from 'luci.view';
|
||||
import { rpc } from 'luci.rpc';
|
||||
import { ui } from 'luci.ui';
|
||||
import { dom } from 'luci.dom';
|
||||
import { Poll } from 'luci.poll';
|
||||
|
||||
const UBUS_NAMESPACE = 'zt-gateway';
|
||||
|
||||
function ubusStatus() {
|
||||
return rpc.declare({
|
||||
object: UBUS_NAMESPACE,
|
||||
method: 'status',
|
||||
params: []
|
||||
})();
|
||||
}
|
||||
|
||||
function ubusSwitch(region, mode) {
|
||||
return rpc.declare({
|
||||
object: UBUS_NAMESPACE,
|
||||
method: 'switch',
|
||||
params: ['region', 'mode']
|
||||
})(region, mode);
|
||||
}
|
||||
|
||||
function ubusHealth(region) {
|
||||
return rpc.declare({
|
||||
object: UBUS_NAMESPACE,
|
||||
method: 'health',
|
||||
params: ['region']
|
||||
})(region);
|
||||
}
|
||||
|
||||
function ubusCancelDrain() {
|
||||
return rpc.declare({
|
||||
object: UBUS_NAMESPACE,
|
||||
method: 'cancel_drain',
|
||||
params: []
|
||||
})();
|
||||
}
|
||||
|
||||
return view.extend({
|
||||
state: {
|
||||
selectedRegion: null,
|
||||
switchMode: 'force'
|
||||
},
|
||||
|
||||
load() {
|
||||
return ubusStatus().catch(err => {
|
||||
ui.addNotification(null, E('p', {}, _('Failed to load status: ') + (err.message || err)));
|
||||
return null;
|
||||
});
|
||||
},
|
||||
|
||||
renderHealthDot(reachable, latencyMs) {
|
||||
const cls = reachable ? 'zt-health-up' : 'zt-health-down';
|
||||
const tip = reachable
|
||||
? (latencyMs != null ? _('UP (%sms').format(latencyMs) : _('UP'))
|
||||
: _('DOWN');
|
||||
return E('span', { class: 'zt-health ' + cls, title: tip }, '●');
|
||||
},
|
||||
|
||||
renderGatewayRow(gw, activeRegion, isSelected) {
|
||||
const isActive = gw.region === activeRegion;
|
||||
return E('tr', { class: 'zt-gateway-row', 'data-region': gw.region }, [
|
||||
E('td', {}, E('input', {
|
||||
type: 'radio',
|
||||
name: 'zt_gateway_region',
|
||||
value: gw.region,
|
||||
change: ev => this.onSelectRegion(ev.target.value),
|
||||
checked: isSelected || null,
|
||||
disabled: isActive
|
||||
})),
|
||||
E('td', { class: 'zt-region-cell' }, [
|
||||
this.renderHealthDot(false, null),
|
||||
' ',
|
||||
isActive ? E('strong', {}, gw.region) : document.createTextNode(gw.region)
|
||||
]),
|
||||
E('td', {}, gw.label),
|
||||
E('td', {}, gw.ip),
|
||||
E('td', {}, isActive ? E('em', {}, _('active')) : '')
|
||||
]);
|
||||
},
|
||||
|
||||
renderGatewaysTable(data) {
|
||||
const activeRegion = data.active_gateway;
|
||||
const rows = (data.gateways || []).map(gw => {
|
||||
const isSelected = gw.region === this.state.selectedRegion;
|
||||
return this.renderGatewayRow(gw, activeRegion, isSelected);
|
||||
});
|
||||
|
||||
return E('table', { class: 'table zt-gateways' }, [
|
||||
E('thead', {}, E('tr', {}, [
|
||||
E('th', {}, ''),
|
||||
E('th', {}, _('Region')),
|
||||
E('th', {}, _('Label')),
|
||||
E('th', {}, _('IP')),
|
||||
E('th', {}, _('Status'))
|
||||
])),
|
||||
E('tbody', {}, rows)
|
||||
]);
|
||||
},
|
||||
|
||||
renderDrainPanel(data) {
|
||||
const drain = data.drain || {};
|
||||
if (!drain.active && !(drain.remaining_connections > 0)) {
|
||||
return E([]);
|
||||
}
|
||||
return E('div', { class: 'zt-drain-panel' }, [
|
||||
E('h4', {}, _('Graceful drain progress')),
|
||||
E('p', {}, _('Draining from %s to %s - %d connection(s) remaining')
|
||||
.format(drain.old_ip || '?', drain.new_ip || '?', drain.remaining_connections || 0)),
|
||||
E('button', {
|
||||
class: 'cbi-button cbi-button-negative',
|
||||
click: ev => this.onCancelDrain(ev)
|
||||
}, _('Cancel drain (force switch)'))
|
||||
]);
|
||||
},
|
||||
|
||||
renderActiveSummary(data) {
|
||||
const region = data.active_gateway || _('none');
|
||||
const ip = data.active_ip || '?';
|
||||
return E('div', { class: 'zt-active-summary' }, [
|
||||
E('strong', {}, _('Active gateway:')),
|
||||
' ',
|
||||
E('span', {}, '%s (%s)'.format(region, ip))
|
||||
]);
|
||||
},
|
||||
|
||||
onSelectRegion(region) {
|
||||
this.state.selectedRegion = region;
|
||||
},
|
||||
|
||||
onSwitchModeChange(ev) {
|
||||
this.state.switchMode = ev.target.value;
|
||||
const timeoutRow = document.querySelector('.zt-drain-timeout-row');
|
||||
if (timeoutRow) {
|
||||
timeoutRow.style.display = (this.state.switchMode === 'graceful') ? '' : 'none';
|
||||
}
|
||||
},
|
||||
|
||||
async onSwitchClick(ev) {
|
||||
const region = this.state.selectedRegion;
|
||||
if (!region) {
|
||||
ui.addNotification(null, E('p', {}, _('Select a gateway first.')));
|
||||
return;
|
||||
}
|
||||
const mode = this.state.switchMode || 'force';
|
||||
const btn = ev.target;
|
||||
btn.disabled = true;
|
||||
const oldLabel = btn.textContent;
|
||||
btn.textContent = _('Switching...');
|
||||
|
||||
try {
|
||||
const res = await ubusSwitch(region, mode);
|
||||
if (res && res.success) {
|
||||
ui.addNotification(null, E('p', {}, res.message || _('Switch complete.')));
|
||||
} else {
|
||||
ui.addNotification(null, E('p', {}, (res && res.message) || _('Switch failed.')));
|
||||
}
|
||||
} catch (err) {
|
||||
ui.addNotification(null, E('p', {}, _('RPC error: ') + (err.message || err)));
|
||||
} finally {
|
||||
btn.disabled = false;
|
||||
btn.textContent = oldLabel;
|
||||
this.refreshState();
|
||||
}
|
||||
},
|
||||
|
||||
async onCancelDrain(ev) {
|
||||
const btn = ev.target;
|
||||
btn.disabled = true;
|
||||
const oldLabel = btn.textContent;
|
||||
btn.textContent = _('Cancelling...');
|
||||
try {
|
||||
const res = await ubusCancelDrain();
|
||||
ui.addNotification(null, E('p', {}, (res && res.message) || _('Cancel issued.')));
|
||||
} catch (err) {
|
||||
ui.addNotification(null, E('p', {}, _('RPC error: ') + (err.message || err)));
|
||||
} finally {
|
||||
btn.disabled = false;
|
||||
btn.textContent = oldLabel;
|
||||
this.refreshState();
|
||||
}
|
||||
},
|
||||
|
||||
async refreshHealth(data) {
|
||||
const gws = data.gateways || [];
|
||||
for (let i = 0; i < gws.length; i++) {
|
||||
const region = gws[i].region;
|
||||
try {
|
||||
const h = await ubusHealth(region);
|
||||
const row = document.querySelector('.zt-gateway-row[data-region="%s"]'.format(region));
|
||||
if (row && h) {
|
||||
const node = row.querySelector('.zt-health');
|
||||
if (node) {
|
||||
node.classList.toggle('zt-health-up', !!h.reachable);
|
||||
node.classList.toggle('zt-health-down', !h.reachable);
|
||||
node.title = h.reachable && h.latency_ms != null
|
||||
? _('UP (%sms').format(h.latency_ms)
|
||||
: (h.reachable ? _('UP') : _('DOWN'));
|
||||
}
|
||||
}
|
||||
} catch (_) {
|
||||
/* health check is best-effort */
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
async refreshState() {
|
||||
try {
|
||||
const data = await ubusStatus();
|
||||
if (data) {
|
||||
this.activeData = data;
|
||||
const container = document.getElementById('zt-gateway-root');
|
||||
if (container) {
|
||||
dom.content(container, this.renderBody(data));
|
||||
}
|
||||
this.refreshHealth(data);
|
||||
}
|
||||
} catch (_) {
|
||||
/* ignore - next poll will retry */
|
||||
}
|
||||
},
|
||||
|
||||
renderBody(data) {
|
||||
const settings = data.settings || {};
|
||||
const mode = settings.switch_mode || 'force';
|
||||
this.state.switchMode = mode;
|
||||
|
||||
return [
|
||||
E('h3', {}, _('ZeroTier Exit Gateway')),
|
||||
this.renderActiveSummary(data),
|
||||
E('div', { class: 'cbi-section' }, [
|
||||
this.renderGatewaysTable(data),
|
||||
E('div', { class: 'cbi-page-actions' }, [
|
||||
E('div', { class: 'zt-mode-group' }, [
|
||||
E('label', {}, _('Switch mode:')),
|
||||
E('select', {
|
||||
class: 'zt-mode-select',
|
||||
change: ev => this.onSwitchModeChange(ev)
|
||||
}, [
|
||||
E('option', { value: 'force', selected: mode === 'force' || null }, _('Force')),
|
||||
E('option', { value: 'graceful', selected: mode === 'graceful' || null }, _('Graceful drain'))
|
||||
]),
|
||||
E('span', {
|
||||
class: 'zt-drain-timeout-row',
|
||||
style: mode === 'graceful' ? '' : 'display:none'
|
||||
}, [
|
||||
' ',
|
||||
_('Drain timeout:'),
|
||||
' ',
|
||||
E('input', {
|
||||
type: 'number',
|
||||
class: 'zt-drain-timeout',
|
||||
value: settings.drain_timeout || 600,
|
||||
min: 30,
|
||||
step: 30
|
||||
}),
|
||||
' s'
|
||||
])
|
||||
]),
|
||||
E('button', {
|
||||
class: 'cbi-button cbi-button-positive',
|
||||
click: ev => this.onSwitchClick(ev)
|
||||
}, _('Switch to selected'))
|
||||
])
|
||||
]),
|
||||
this.renderDrainPanel(data)
|
||||
];
|
||||
},
|
||||
|
||||
render(data) {
|
||||
this.injectCSS();
|
||||
if (!data) {
|
||||
return E('div', { class: 'alert-message error' }, [
|
||||
E('p', {}, _('Could not load gateway state. Verify rpcd is running and the zt-gateway backend is installed.'))
|
||||
]);
|
||||
}
|
||||
this.activeData = data;
|
||||
|
||||
const root = E('div', { id: 'zt-gateway-root' }, this.renderBody(data));
|
||||
|
||||
const self = this;
|
||||
const poll = Poll.create('zt-gateway-drain', function() {
|
||||
return self.refreshState();
|
||||
}, 5);
|
||||
poll.start();
|
||||
|
||||
return root;
|
||||
},
|
||||
|
||||
injectCSS() {
|
||||
if (document.getElementById('zt-gateway-css')) return;
|
||||
const css = '.zt-health{font-size:1.1em;margin-right:4px}' +
|
||||
'.zt-health-up{color:#2e8b57}.zt-health-down{color:#c0392b}' +
|
||||
'.zt-gateways tbody tr{line-height:1.8}' +
|
||||
'.zt-mode-group{display:flex;gap:0.5em;align-items:center;flex-wrap:wrap}' +
|
||||
'.zt-active-summary{margin:0.5em 0 1em 0}' +
|
||||
'.zt-drain-panel{margin-top:1em;padding:0.75em;background:#f8f8f8;border:1px solid #ddd}';
|
||||
const style = document.createElement('style');
|
||||
style.id = 'zt-gateway-css';
|
||||
style.textContent = css;
|
||||
document.head.appendChild(style);
|
||||
}
|
||||
});
|
||||
27
root/etc/config/zt-gateway
Normal file
27
root/etc/config/zt-gateway
Normal file
@@ -0,0 +1,27 @@
|
||||
config global 'global'
|
||||
option active_gateway 'amsterdam'
|
||||
option switch_mode 'force'
|
||||
option drain_timeout '600'
|
||||
option health_interval '60'
|
||||
option auto_failback '0'
|
||||
|
||||
config gateway
|
||||
option region 'amsterdam'
|
||||
option label 'Amsterdam (ocirosea641)'
|
||||
option ip '10.11.12.3'
|
||||
option default '1'
|
||||
option health_check 'ping'
|
||||
|
||||
config gateway
|
||||
option region 'tirunelveli'
|
||||
option label 'Tirunelveli (rpi1000)'
|
||||
option ip '10.11.12.5'
|
||||
option default '0'
|
||||
option health_check 'ping'
|
||||
|
||||
config gateway
|
||||
option region 'bangalore'
|
||||
option label 'Bangalore (sensecap-m4)'
|
||||
option ip '10.11.12.4'
|
||||
option default '0'
|
||||
option health_check 'ping'
|
||||
420
root/usr/sbin/zt-gateway-switch
Executable file
420
root/usr/sbin/zt-gateway-switch
Executable file
@@ -0,0 +1,420 @@
|
||||
#!/bin/sh
|
||||
# shellcheck shell=sh
|
||||
#
|
||||
# zt-gateway-switch — reconfigure which remote ZeroTier node acts as the
|
||||
# internet exit gateway for WIBLAN clients (10.11.13.0/24).
|
||||
#
|
||||
# Usage:
|
||||
# zt-gateway-switch <new_ip> <mode> [drain_timeout]
|
||||
#
|
||||
# Modes:
|
||||
# force - Instant cutover + conntrack flush
|
||||
# graceful - fwmark drain; falls back to force on timeout
|
||||
#
|
||||
# This script is callable directly from SSH for testing; the LuCI rpcd
|
||||
# backend (/usr/share/rpcd/ucode/zt-gateway.uc) wraps it for the UI.
|
||||
#
|
||||
# Exit codes:
|
||||
# 0 success
|
||||
# 1 usage error
|
||||
# 2 pre-flight failed (gateway unreachable)
|
||||
# 3 runtime failure (route/iptables/uci update failed)
|
||||
#
|
||||
# Environment overrides (used by the Docker test harness as much as by
|
||||
# production):
|
||||
# ZTG_BRIDGE bridge device (default: br-zt)
|
||||
# ZTG_WIBLAN_CIDR WIBLAN source subnet (default: 10.11.13.0/24)
|
||||
# ZTG_TABLE_MAIN main policy table (default: 100)
|
||||
# ZTG_TABLE_DRAIN drain policy table (default: 101)
|
||||
# ZTG_TABLE_MWAN mwan3 return-traffic table (default: 1)
|
||||
# ZTG_FWMARK conntrack mark used for drain (default: 0x100)
|
||||
# ZTG_DRAIN_PRIORITY priority of the fwmark drain rule (default: 99)
|
||||
# ZTG_HOTPLUG hotplug script path
|
||||
# ZTG_RCLOCAL rc.local path
|
||||
# ZTG_DRAIN_PIDFILE pidfile for the drain monitor
|
||||
# ZTG_DRAIN_NEWFILE file recording the new gateway IP during drain
|
||||
# ZTG_DRAIN_OLDFILE file recording the old gateway IP during drain
|
||||
# ZTG_SKIP_PERSIST if set to 1, skip persistence (testing)
|
||||
# ZTG_PING_IFACE interface for the pre-flight ping (default: br-zt)
|
||||
#
|
||||
# This script intentionally uses only POSIX sh + busybox-compatible
|
||||
# utilities so the same code path runs on the router and in the Docker
|
||||
# openwrt/rootfs test image.
|
||||
|
||||
set -eu
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Config
|
||||
# ----------------------------------------------------------------------------
|
||||
BRIDGE="${ZTG_BRIDGE:-br-zt}"
|
||||
WIBLAN_CIDR="${ZTG_WIBLAN_CIDR:-10.11.13.0/24}"
|
||||
TABLE_MAIN="${ZTG_TABLE_MAIN:-100}"
|
||||
TABLE_DRAIN="${ZTG_TABLE_DRAIN:-101}"
|
||||
TABLE_MWAN="${ZTG_TABLE_MWAN:-1}"
|
||||
DRAIN_PRIORITY="${ZTG_DRAIN_PRIORITY:-99}"
|
||||
FWMARK="${ZTG_FWMARK:-0x100}"
|
||||
HOTPLUG="${ZTG_HOTPLUG:-/etc/hotplug.d/net/99-zerotier-bridge}"
|
||||
RCLOCAL="${ZTG_RCLOCAL:-/etc/rc.local}"
|
||||
DRAIN_PIDFILE="${ZTG_DRAIN_PIDFILE:-/var/run/zt-gateway-drain.pid}"
|
||||
DRAIN_NEWFILE="${ZTG_DRAIN_NEWFILE:-/var/run/zt-gateway-drain.new}"
|
||||
DRAIN_OLDFILE="${ZTG_DRAIN_OLDFILE:-/var/run/zt-gateway-drain.old}"
|
||||
PING_IFACE="${ZTG_PING_IFACE:-$BRIDGE}"
|
||||
SKIP_PERSIST="${ZTG_SKIP_PERSIST:-0}"
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Logging
|
||||
# ----------------------------------------------------------------------------
|
||||
log() { printf '[zt-gateway-switch] %s\n' "$*" >&2; }
|
||||
die() { rc=$1; shift; log "ERROR: $*"; exit "$rc"; }
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ----------------------------------------------------------------------------
|
||||
valid_ip() {
|
||||
ip=$1
|
||||
# POSIX-only IPv4 validator (no awk regex extensions required).
|
||||
rest=$ip
|
||||
i=0
|
||||
while [ $i -lt 4 ]; do
|
||||
oct=${rest%%.*}
|
||||
if [ "$oct" = "$rest" ]; then
|
||||
# last octet
|
||||
rest=
|
||||
else
|
||||
rest=${rest#*.}
|
||||
fi
|
||||
case "$oct" in
|
||||
''|*[!0-9]*) return 1 ;;
|
||||
esac
|
||||
[ "$oct" -ge 0 ] 2>/dev/null || return 1
|
||||
[ "$oct" -le 255 ] || return 1
|
||||
i=$((i+1))
|
||||
done
|
||||
[ $i -eq 4 ] || return 1
|
||||
[ -z "$rest" ] || return 1
|
||||
[ "$ip" != "$rest" ] || return 1
|
||||
# Reject leading/trailing dots already handled above; accept.
|
||||
return 0
|
||||
}
|
||||
|
||||
preflight_ping() {
|
||||
gw_ip=$1
|
||||
log "pre-flight: ping ${PING_IFACE}->${gw_ip}"
|
||||
if ping -c 2 -W 3 -I "$PING_IFACE" "$gw_ip" >/dev/null 2>&1; then
|
||||
return 0
|
||||
fi
|
||||
# Retry without -I in case the harness lacks the iface binding.
|
||||
if ping -c 2 -W 3 "$gw_ip" >/dev/null 2>&1; then
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
# Replace OLD_IP with NEW_IP in a file. IPv4 only; dots escaped for sed.
|
||||
_replace_ip_in_file() {
|
||||
file=$1
|
||||
old=$2
|
||||
new=$3
|
||||
[ -f "$file" ] || { log "warning: $file not found; skipping"; return 0; }
|
||||
old_re=$(printf '%s\n' "$old" | sed 's/[.]/\\./g')
|
||||
sed -i "s/${old_re}/${new}/g" "$file"
|
||||
}
|
||||
|
||||
set_host_route() {
|
||||
gw_ip=$1
|
||||
ip route replace "$gw_ip" dev "$BRIDGE"
|
||||
}
|
||||
|
||||
set_table_default() {
|
||||
gw_ip=$1
|
||||
table=$2
|
||||
ip route replace default via "$gw_ip" dev "$BRIDGE" table "$table"
|
||||
}
|
||||
|
||||
del_table_default() {
|
||||
table=$1
|
||||
ip route del default dev "$BRIDGE" table "$table" 2>/dev/null || true
|
||||
}
|
||||
|
||||
ensure_mwan_return() {
|
||||
ip route replace "$WIBLAN_CIDR" dev "$BRIDGE" table "$TABLE_MWAN"
|
||||
}
|
||||
|
||||
# Extract the gateway IP from the table N default route, or echo empty.
|
||||
current_gateway_of() {
|
||||
table=$1
|
||||
ip route show table "$table" 2>/dev/null \
|
||||
| awk '/^[[:space:]]*default/ {
|
||||
for (i = 1; i <= NF; i++)
|
||||
if ($i == "via") { print $(i+1); exit }
|
||||
}'
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Persistence
|
||||
# ----------------------------------------------------------------------------
|
||||
persist_all() {
|
||||
new_ip=$1
|
||||
old_ip=${2:-}
|
||||
|
||||
if [ "$SKIP_PERSIST" = "1" ]; then
|
||||
log "ZTG_SKIP_PERSIST=1; skipping persistence"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [ -n "$old_ip" ]; then
|
||||
_replace_ip_in_file "$HOTPLUG" "$old_ip" "$new_ip"
|
||||
_replace_ip_in_file "$RCLOCAL" "$old_ip" "$new_ip"
|
||||
fi
|
||||
|
||||
if command -v uci >/dev/null 2>&1; then
|
||||
if uci -q get network.zt_gateway_host >/dev/null 2>&1; then
|
||||
uci set network.zt_gateway_host.target="$new_ip"
|
||||
else
|
||||
uci -q set network.zt_gateway_host=config route
|
||||
uci -q set network.zt_gateway_host.target="$new_ip"
|
||||
uci -q set network.zt_gateway_host.interface='br-zt'
|
||||
fi
|
||||
|
||||
if uci -q get network.zt_gateway_default >/dev/null 2>&1; then
|
||||
uci set network.zt_gateway_default.gateway="$new_ip"
|
||||
else
|
||||
uci -q set network.zt_gateway_default=config route
|
||||
uci -q set network.zt_gateway_default.target='0.0.0.0'
|
||||
uci -q set network.zt_gateway_default.netmask='0.0.0.0'
|
||||
uci -q set network.zt_gateway_default.gateway="$new_ip"
|
||||
uci -q set network.zt_gateway_default.table="$TABLE_MAIN"
|
||||
fi
|
||||
|
||||
uci -q commit network || true
|
||||
fi
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Force switch
|
||||
# ----------------------------------------------------------------------------
|
||||
do_force() {
|
||||
new_ip=$1
|
||||
old_ip=${2:-}
|
||||
|
||||
preflight_ping "$new_ip" || die 2 "gateway ${new_ip} is unreachable over ${PING_IFACE}"
|
||||
|
||||
set_host_route "$new_ip"
|
||||
set_table_default "$new_ip" "$TABLE_MAIN"
|
||||
ensure_mwan_return
|
||||
|
||||
if command -v conntrack >/dev/null 2>&1; then
|
||||
conntrack -D -s "$WIBLAN_CIDR" 2>/dev/null || true
|
||||
else
|
||||
log "warning: conntrack not present; skipping flush"
|
||||
fi
|
||||
|
||||
persist_all "$new_ip" "$old_ip"
|
||||
|
||||
if ip route show table "$TABLE_MAIN" 2>/dev/null | grep -q "default via ${new_ip}"; then
|
||||
log "force: table ${TABLE_MAIN} default via ${new_ip} confirmed"
|
||||
else
|
||||
die 3 "verification failed: default via ${new_ip} not in table ${TABLE_MAIN}"
|
||||
fi
|
||||
|
||||
printf 'force switch to %s complete\n' "$new_ip"
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Graceful drain switch
|
||||
# ----------------------------------------------------------------------------
|
||||
mangle_rules_install() {
|
||||
# Source-only match: WIBLAN clients reach the router on whatever LAN
|
||||
# interface they live on; binding to -i br-zt would miss the actual
|
||||
# inbound path. We scope by source subnet so the rule fires regardless
|
||||
# of ingress interface, then CONNMARK restores/marks/saves per-flow.
|
||||
iptables -t mangle -A PREROUTING -s "$WIBLAN_CIDR" \
|
||||
-m conntrack --ctstate ESTABLISHED,RELATED -j CONNMARK --restore-mark
|
||||
iptables -t mangle -A PREROUTING -s "$WIBLAN_CIDR" \
|
||||
-m conntrack --ctstate ESTABLISHED,RELATED -m mark --mark 0 -j MARK --set-mark "$FWMARK"
|
||||
iptables -t mangle -A PREROUTING -s "$WIBLAN_CIDR" \
|
||||
-j CONNMARK --save-mark
|
||||
}
|
||||
|
||||
mangle_rules_remove() {
|
||||
iptables -t mangle -D PREROUTING -s "$WIBLAN_CIDR" \
|
||||
-j CONNMARK --save-mark 2>/dev/null || true
|
||||
iptables -t mangle -D PREROUTING -s "$WIBLAN_CIDR" \
|
||||
-m conntrack --ctstate ESTABLISHED,RELATED -m mark --mark 0 -j MARK --set-mark "$FWMARK" 2>/dev/null || true
|
||||
iptables -t mangle -D PREROUTING -s "$WIBLAN_CIDR" \
|
||||
-m conntrack --ctstate ESTABLISHED,RELATED -j CONNMARK --restore-mark 2>/dev/null || true
|
||||
}
|
||||
|
||||
drain_install_rules() {
|
||||
old_ip=$1
|
||||
ip route replace default via "$old_ip" dev "$BRIDGE" table "$TABLE_DRAIN"
|
||||
mangle_rules_install
|
||||
ip rule add fwmark "$FWMARK" table "$TABLE_DRAIN" priority "$DRAIN_PRIORITY" 2>/dev/null || \
|
||||
ip rule replace fwmark "$FWMARK" table "$TABLE_DRAIN" priority "$DRAIN_PRIORITY"
|
||||
}
|
||||
|
||||
drain_cleanup() {
|
||||
log "drain: cleaning up mangle rules, drain table, and pidfiles"
|
||||
mangle_rules_remove
|
||||
ip rule del fwmark "$FWMARK" table "$TABLE_DRAIN" priority "$DRAIN_PRIORITY" 2>/dev/null || true
|
||||
del_table_default "$TABLE_DRAIN"
|
||||
rm -f "$DRAIN_PIDFILE" "$DRAIN_NEWFILE" "$DRAIN_OLDFILE" 2>/dev/null || true
|
||||
}
|
||||
|
||||
drain_count_marked() {
|
||||
# Count conntrack entries carrying the drain mark. /proc/net/nf_conntrack
|
||||
# reports marks in decimal (e.g. mark=256 for 0x100), so convert FWMARK.
|
||||
mark_dec=$(( FWMARK + 0 ))
|
||||
mark_hex="0x$(printf '%x' "$mark_dec")"
|
||||
|
||||
# Preferred: conntrack CLI with -m filter (conntrack-tools >= 1.4.4).
|
||||
if command -v conntrack >/dev/null 2>&1; then
|
||||
count=$(conntrack -L -m "$mark_hex" 2>/dev/null | grep -c . || true)
|
||||
if [ -n "$count" ] && [ "$count" -gt 0 ]; then
|
||||
echo "$count"
|
||||
return
|
||||
fi
|
||||
fi
|
||||
|
||||
# Fallback: parse /proc/net/nf_conntrack. Works without the CLI and
|
||||
# also catches entries the CLI filter might miss on older builds.
|
||||
awk -v mdec="$mark_dec" -v mhex="$mark_hex" '
|
||||
BEGIN { n = 0 }
|
||||
{
|
||||
if (index($0, "mark=" mdec) || index($0, "mark=" mhex)) n++
|
||||
}
|
||||
END { print n }
|
||||
' /proc/net/nf_conntrack 2>/dev/null || echo 0
|
||||
}
|
||||
|
||||
drain_monitor() {
|
||||
new_ip=$1
|
||||
old_ip=$2
|
||||
timeout=$3
|
||||
|
||||
# Grace period before the first count: packets already in flight need
|
||||
# at least one PREROUTING pass through the mangle rules before their
|
||||
# conntrack entry picks up mark=0x100. Without this cushion the very
|
||||
# first count races the mangle hook and reports zero prematurely.
|
||||
DRAIN_INITIAL_GRACE="${ZTG_DRAIN_INITIAL_GRACE:-3}"
|
||||
# Number of consecutive zero-count polls required before we declare
|
||||
# the drain complete. Guards against transient empty reads that race
|
||||
# the conntrack update.
|
||||
DRAIN_MIN_ZERO_POLLS="${ZTG_DRAIN_MIN_ZERO_POLLS:-2}"
|
||||
DRAIN_POLL_INTERVAL="${ZTG_DRAIN_POLL_INTERVAL:-5}"
|
||||
|
||||
sleep "$DRAIN_INITIAL_GRACE"
|
||||
|
||||
end=$(( $(date +%s) + timeout ))
|
||||
consecutive_zeros=0
|
||||
while :; do
|
||||
remaining=$(drain_count_marked)
|
||||
log "drain: ${remaining} marked connection(s) remaining (consecutive_zeros=${consecutive_zeros})"
|
||||
if [ "$remaining" -eq 0 ]; then
|
||||
consecutive_zeros=$(( consecutive_zeros + 1 ))
|
||||
if [ "$consecutive_zeros" -ge "$DRAIN_MIN_ZERO_POLLS" ]; then
|
||||
log "drain: complete (${DRAIN_MIN_ZERO_POLLS} consecutive zero polls)"
|
||||
drain_cleanup
|
||||
return 0
|
||||
fi
|
||||
else
|
||||
consecutive_zeros=0
|
||||
fi
|
||||
if [ "$(date +%s)" -ge "$end" ]; then
|
||||
log "drain: timeout (${timeout}s) reached; falling back to force"
|
||||
set_host_route "$new_ip"
|
||||
set_table_default "$new_ip" "$TABLE_MAIN"
|
||||
ensure_mwan_return
|
||||
if command -v conntrack >/dev/null 2>&1; then
|
||||
conntrack -D -s "$WIBLAN_CIDR" 2>/dev/null || true
|
||||
fi
|
||||
drain_cleanup
|
||||
persist_all "$new_ip" "$old_ip"
|
||||
return 0
|
||||
fi
|
||||
sleep "$DRAIN_POLL_INTERVAL"
|
||||
done
|
||||
}
|
||||
|
||||
do_graceful() {
|
||||
new_ip=$1
|
||||
timeout=${2:-600}
|
||||
|
||||
old_ip=$(current_gateway_of "$TABLE_MAIN")
|
||||
if [ -z "$old_ip" ]; then
|
||||
log "graceful: no current gateway in table ${TABLE_MAIN}; falling back to force"
|
||||
do_force "$new_ip" ""
|
||||
return $?
|
||||
fi
|
||||
|
||||
if [ "$old_ip" = "$new_ip" ]; then
|
||||
log "graceful: new gateway equals current gateway (${new_ip}); nothing to do"
|
||||
return 0
|
||||
fi
|
||||
|
||||
preflight_ping "$new_ip" || die 2 "gateway ${new_ip} is unreachable over ${PING_IFACE}"
|
||||
|
||||
printf '%s\n' "$new_ip" >"$DRAIN_NEWFILE"
|
||||
printf '%s\n' "$old_ip" >"$DRAIN_OLDFILE"
|
||||
|
||||
drain_install_rules "$old_ip"
|
||||
|
||||
set_host_route "$new_ip"
|
||||
set_table_default "$new_ip" "$TABLE_MAIN"
|
||||
ensure_mwan_return
|
||||
|
||||
persist_all "$new_ip" "$old_ip"
|
||||
|
||||
: >"$DRAIN_PIDFILE"
|
||||
(
|
||||
trap 'drain_cleanup; exit 0' TERM INT
|
||||
echo $$ >"$DRAIN_PIDFILE"
|
||||
drain_monitor "$new_ip" "$old_ip" "$timeout"
|
||||
rm -f "$DRAIN_PIDFILE" "$DRAIN_NEWFILE" "$DRAIN_OLDFILE" 2>/dev/null || true
|
||||
) >/tmp/zt-gateway-drain.log 2>&1 &
|
||||
|
||||
printf 'graceful switch to %s started (draining from %s, timeout %ss)\n' \
|
||||
"$new_ip" "$old_ip" "$timeout"
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Entry point
|
||||
# ----------------------------------------------------------------------------
|
||||
main() {
|
||||
if [ $# -lt 2 ]; then
|
||||
cat >&2 <<'USAGE'
|
||||
Usage: zt-gateway-switch <new_ip> <mode> [drain_timeout]
|
||||
|
||||
Modes:
|
||||
force Instant cutover + conntrack flush
|
||||
graceful fwmark drain; falls back to force on timeout
|
||||
|
||||
Options:
|
||||
drain_timeout graceful-mode drain timeout in seconds (default: 600)
|
||||
USAGE
|
||||
die 1 "missing arguments"
|
||||
fi
|
||||
|
||||
new_ip=$1
|
||||
mode=$2
|
||||
timeout=${3:-600}
|
||||
|
||||
if ! valid_ip "$new_ip"; then
|
||||
die 1 "invalid IPv4 address: ${new_ip}"
|
||||
fi
|
||||
|
||||
case "$mode" in
|
||||
force)
|
||||
old_ip=$(current_gateway_of "$TABLE_MAIN")
|
||||
do_force "$new_ip" "${old_ip:-}"
|
||||
;;
|
||||
graceful)
|
||||
do_graceful "$new_ip" "$timeout"
|
||||
;;
|
||||
*)
|
||||
die 1 "unknown mode: ${mode} (use force|graceful)"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
main "$@"
|
||||
14
root/usr/share/luci/menu.d/luci-app-zt-gateway.json
Normal file
14
root/usr/share/luci/menu.d/luci-app-zt-gateway.json
Normal file
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"admin/services/zt-gateway": {
|
||||
"title": "ZeroTier Gateway",
|
||||
"order": 90,
|
||||
"action": {
|
||||
"type": "view",
|
||||
"path": "zt-gateway/overview"
|
||||
},
|
||||
"depends": {
|
||||
"acl": ["luci-app-zt-gateway"],
|
||||
"uci": { "zt-gateway": true }
|
||||
}
|
||||
}
|
||||
}
|
||||
25
root/usr/share/rpcd/acl.d/luci-app-zt-gateway.json
Normal file
25
root/usr/share/rpcd/acl.d/luci-app-zt-gateway.json
Normal file
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"luci-app-zt-gateway": {
|
||||
"description": "ZeroTier Exit Gateway switching",
|
||||
"read": {
|
||||
"uci": ["zt-gateway", "network"],
|
||||
"ubus": {
|
||||
"zt-gateway": ["status", "health", "drain_status"]
|
||||
},
|
||||
"file": {
|
||||
"/etc/hotplug.d/net/99-zerotier-bridge": ["read"],
|
||||
"/etc/rc.local": ["read"]
|
||||
}
|
||||
},
|
||||
"write": {
|
||||
"uci": ["zt-gateway", "network"],
|
||||
"ubus": {
|
||||
"zt-gateway": ["switch", "cancel_drain"]
|
||||
},
|
||||
"file": {
|
||||
"/etc/hotplug.d/net/99-zerotier-bridge": ["write"],
|
||||
"/etc/rc.local": ["write"]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
301
root/usr/share/rpcd/ucode/zt-gateway.uc
Normal file
301
root/usr/share/rpcd/ucode/zt-gateway.uc
Normal file
@@ -0,0 +1,301 @@
|
||||
'use strict';
|
||||
|
||||
import { ubus } from 'ubus';
|
||||
import { uci } from 'uci';
|
||||
import { fs } from 'fs';
|
||||
|
||||
const SWITCH_SCRIPT = '/usr/sbin/zt-gateway-switch';
|
||||
const DRAIN_PIDFILE = '/var/run/zt-gateway-drain.pid';
|
||||
const DRAIN_NEWFILE = '/var/run/zt-gateway-drain.new';
|
||||
const DRAIN_OLDFILE = '/var/run/zt-gateway-drain.old';
|
||||
|
||||
function load_gateways() {
|
||||
const cursor = uci.cursor();
|
||||
const result = [];
|
||||
const sections = uci.sections(cursor, 'zt-gateway', 'gateway');
|
||||
for (let i = 0; i < length(sections); i++) {
|
||||
const s = sections[i];
|
||||
push(result, {
|
||||
region: s.region ?? '',
|
||||
label: s.label ?? s.region ?? '',
|
||||
ip: s.ip ?? '',
|
||||
default: s.default === '1',
|
||||
health_check: s.health_check ?? 'ping'
|
||||
});
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function gateway_by_region(region) {
|
||||
const gws = load_gateways();
|
||||
for (let i = 0; i < length(gws); i++) {
|
||||
if (gws[i].region === region) {
|
||||
return gws[i];
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function read_active_region() {
|
||||
const cursor = uci.cursor();
|
||||
return cursor.get('zt-gateway', 'global', 'active_gateway') ?? '';
|
||||
}
|
||||
|
||||
function read_global_option(option, default_value) {
|
||||
const cursor = uci.cursor();
|
||||
return cursor.get('zt-gateway', 'global', option) ?? default_value;
|
||||
}
|
||||
|
||||
function drain_active() {
|
||||
const st = fs.stat(DRAIN_PIDFILE);
|
||||
return st?.type === 'file';
|
||||
}
|
||||
|
||||
function read_drain_pid() {
|
||||
const data = fs.readfile(DRAIN_PIDFILE);
|
||||
if (!data) {
|
||||
return null;
|
||||
}
|
||||
const s = trim(data);
|
||||
return length(s) > 0 ? +s : null;
|
||||
}
|
||||
|
||||
function read_drain_state() {
|
||||
let new_ip = '';
|
||||
let old_ip = '';
|
||||
const n = fs.readfile(DRAIN_NEWFILE);
|
||||
if (n) {
|
||||
new_ip = trim(n);
|
||||
}
|
||||
const o = fs.readfile(DRAIN_OLDFILE);
|
||||
if (o) {
|
||||
old_ip = trim(o);
|
||||
}
|
||||
return { new_ip, old_ip };
|
||||
}
|
||||
|
||||
function run_switch_script_capture(ip, mode) {
|
||||
const mode_lc = mode ?? 'force';
|
||||
let cmd = `${SWITCH_SCRIPT} ${shell_quote(ip)} ${shell_quote(mode_lc)}`;
|
||||
if (mode_lc === 'graceful') {
|
||||
const to = +read_global_option('drain_timeout', '600');
|
||||
cmd += ` ${to > 0 ? to : 600}`;
|
||||
}
|
||||
const stderr_pipe = '/tmp/zt-gw-stderr';
|
||||
const code = system(`${cmd} 2>${stderr_pipe}`);
|
||||
const err = fs.readfile(stderr_pipe) ?? '';
|
||||
fs.unlink(stderr_pipe);
|
||||
return { code: code ?? 0, stderr: trim(err) };
|
||||
}
|
||||
|
||||
function run_switch_script_force_capture(ip) {
|
||||
const stderr_pipe = '/tmp/zt-gw-stderr';
|
||||
const code = system(`${SWITCH_SCRIPT} ${shell_quote(ip)} force 2>${stderr_pipe}`);
|
||||
const err = fs.readfile(stderr_pipe) ?? '';
|
||||
fs.unlink(stderr_pipe);
|
||||
return { code: code ?? 0, stderr: trim(err) };
|
||||
}
|
||||
|
||||
function system_output(cmd) {
|
||||
const tmp = `/tmp/zt-gw-out-${getpid()}`;
|
||||
system(`${cmd} >${tmp} 2>/dev/null`);
|
||||
const out = fs.readfile(tmp) ?? '';
|
||||
fs.unlink(tmp);
|
||||
return out;
|
||||
}
|
||||
|
||||
function shell_quote(s) {
|
||||
if (!s) {
|
||||
return "''";
|
||||
}
|
||||
if (s ~ /^[A-Za-z0-9_./:@=-]+$/) {
|
||||
return s;
|
||||
}
|
||||
return `'${replace(s, "'", "'\\''")}'`;
|
||||
}
|
||||
|
||||
function rpc_status(req) {
|
||||
const gws = load_gateways();
|
||||
const active_region = read_active_region();
|
||||
|
||||
let active_ip = '';
|
||||
const active_gw = gateway_by_region(active_region);
|
||||
if (active_gw) {
|
||||
active_ip = active_gw.ip;
|
||||
}
|
||||
|
||||
const drain_state = read_drain_state();
|
||||
let remaining = 0;
|
||||
if (drain_active()) {
|
||||
const out = system_output(`conntrack -L -m 0x100 2>/dev/null | wc -l`);
|
||||
remaining = +trim(out) || 0;
|
||||
}
|
||||
|
||||
const cursor = uci.cursor();
|
||||
const settings = {
|
||||
switch_mode: cursor.get('zt-gateway', 'global', 'switch_mode') ?? 'force',
|
||||
drain_timeout: +(cursor.get('zt-gateway', 'global', 'drain_timeout') ?? 600),
|
||||
health_interval: +(cursor.get('zt-gateway', 'global', 'health_interval') ?? 60),
|
||||
auto_failback: cursor.get('zt-gateway', 'global', 'auto_failback') === '1'
|
||||
};
|
||||
|
||||
ubus.reply(req, {
|
||||
active_gateway: active_region,
|
||||
active_ip: active_ip,
|
||||
settings: settings,
|
||||
gateways: gws,
|
||||
drain: {
|
||||
active: drain_active(),
|
||||
new_ip: drain_state.new_ip,
|
||||
old_ip: drain_state.old_ip,
|
||||
remaining_connections: remaining
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
function rpc_switch(req, msg) {
|
||||
const region = msg?.region;
|
||||
const mode = msg?.mode ?? read_global_option('switch_mode', 'force');
|
||||
if (!region || (mode !== 'force' && mode !== 'graceful')) {
|
||||
ubus.reply(req, {
|
||||
success: false,
|
||||
message: 'Missing or invalid argument: region required, mode must be force|graceful'
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
if (drain_active() && mode === 'graceful') {
|
||||
ubus.reply(req, {
|
||||
success: false,
|
||||
message: 'A graceful drain is already in progress. Cancel it first.'
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const gw = gateway_by_region(region);
|
||||
if (!gw) {
|
||||
ubus.reply(req, { success: false, message: `Unknown gateway region: ${region}` });
|
||||
return;
|
||||
}
|
||||
|
||||
const current_region = read_active_region();
|
||||
if (current_region === region && !drain_active()) {
|
||||
ubus.reply(req, { success: true, message: `Already on ${region}.` });
|
||||
return;
|
||||
}
|
||||
|
||||
const result = run_switch_script_capture(gw.ip, mode);
|
||||
if (result.code !== 0) {
|
||||
ubus.reply(req, {
|
||||
success: false,
|
||||
message: result.stderr || `Switch script failed with exit code ${result.code}`
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const cursor = uci.cursor();
|
||||
cursor.set('zt-gateway', 'global', 'active_gateway', region);
|
||||
cursor.commit('zt-gateway');
|
||||
|
||||
ubus.reply(req, {
|
||||
success: true,
|
||||
message: `Switched to ${region} (${gw.ip}) via ${mode}`,
|
||||
active_gateway: region,
|
||||
mode: mode,
|
||||
drain_active: (mode === 'graceful')
|
||||
});
|
||||
}
|
||||
|
||||
function rpc_health(req, msg) {
|
||||
const region = msg?.region;
|
||||
if (!region) {
|
||||
ubus.reply(req, { reachable: false, message: 'Missing region' });
|
||||
return;
|
||||
}
|
||||
|
||||
const gw = gateway_by_region(region);
|
||||
if (!gw) {
|
||||
ubus.reply(req, { reachable: false, message: `Unknown region ${region}` });
|
||||
return;
|
||||
}
|
||||
|
||||
const out = system_output(
|
||||
`ping -c 2 -W 3 -I br-zt ${shell_quote(gw.ip)} 2>/dev/null; echo "EXIT:$?"`
|
||||
);
|
||||
const exit_match = match(out, /EXIT:(\d+)/);
|
||||
const exit_code = exit_match ? +exit_match[1] : 1;
|
||||
|
||||
let latency_ms = null;
|
||||
if (exit_code === 0) {
|
||||
const stats = match(out, /rtt min\/avg\/max[^=]*=\s*[\d.]+\/([\d.]+)\//);
|
||||
if (stats) {
|
||||
latency_ms = +stats[1];
|
||||
}
|
||||
}
|
||||
|
||||
ubus.reply(req, {
|
||||
region: region,
|
||||
ip: gw.ip,
|
||||
reachable: exit_code === 0,
|
||||
latency_ms: latency_ms
|
||||
});
|
||||
}
|
||||
|
||||
function rpc_drain_status(req) {
|
||||
const active = drain_active();
|
||||
const state = read_drain_state();
|
||||
let remaining = 0;
|
||||
if (active) {
|
||||
const out = system_output(`conntrack -L -m 0x100 2>/dev/null | wc -l`);
|
||||
remaining = +trim(out) || 0;
|
||||
}
|
||||
ubus.reply(req, {
|
||||
active: active,
|
||||
new_ip: state.new_ip,
|
||||
old_ip: state.old_ip,
|
||||
remaining_connections: remaining
|
||||
});
|
||||
}
|
||||
|
||||
function rpc_cancel_drain(req) {
|
||||
if (!drain_active()) {
|
||||
ubus.reply(req, { success: false, message: 'No drain in progress' });
|
||||
return;
|
||||
}
|
||||
|
||||
const state = read_drain_state();
|
||||
const pid = read_drain_pid();
|
||||
if (pid) {
|
||||
system(`kill -TERM ${pid} 2>/dev/null`);
|
||||
}
|
||||
|
||||
if (!state.new_ip) {
|
||||
fs.unlink(DRAIN_PIDFILE);
|
||||
ubus.reply(req, {
|
||||
success: true,
|
||||
message: 'Drain cancelled (new gateway unknown; route left as-is).'
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const result = run_switch_script_force_capture(state.new_ip);
|
||||
|
||||
fs.unlink(DRAIN_PIDFILE);
|
||||
fs.unlink(DRAIN_NEWFILE);
|
||||
fs.unlink(DRAIN_OLDFILE);
|
||||
|
||||
ubus.reply(req, {
|
||||
success: result.code === 0,
|
||||
message: result.code === 0
|
||||
? `Drain cancelled; force-switched to ${state.new_ip}.`
|
||||
: (result.stderr || `Cancel failed with code ${result.code}`)
|
||||
});
|
||||
}
|
||||
|
||||
export default {
|
||||
status: rpc_status,
|
||||
switch: rpc_switch,
|
||||
health: rpc_health,
|
||||
drain_status: rpc_drain_status,
|
||||
cancel_drain: rpc_cancel_drain
|
||||
};
|
||||
Reference in New Issue
Block a user