Malar Invention
581d625044
Routing fixes (2026-07-14): - Add missing ip rule 'from 10.11.13.0/24 lookup 100' to hotplug ifup case - Add UCI network rule persistence so netifd restores it on boot - Verify ip rule exists in zt-gateway-switch do_force/do_graceful - Fix BRIDGE_PORTS auto-detect: use /proc/net/dev instead of broken awk-over-ip pipeline (busybox awk mishandles exit in compound if) - Validate bridge port candidate exists as network interface - Fix setup-routing: use dev br-zt not dev ztX (ZT iface has no IP when enslaved to bridge, causing 'Nexthop has invalid gateway') - Replace ip rule replace (GNU-only) with del+add for busybox Infrastructure: - Fix deploy:install stdin starvation: ssh/scp consume pipe data in find|while loop; add </dev/null to prevent truncation - Move luci-dev skill from root/ to skills/ with .agents/skills/ symlink - Add policy routing and busybox gotcha sections to SKILL.md - Add diagnostics doc for the routing fix session
195 lines
11 KiB
Markdown
195 lines
11 KiB
Markdown
# Progress: Playwright E2E Tests for luci-app-zt-gateway
|
|
|
|
## Date: 2026-06-26/2026-06-27
|
|
|
|
## Completed Fixes
|
|
|
|
### 1. `login()` Helper (`e2e/utils/auth.ts`)
|
|
- **Root cause**: LuCI login form lives inside `<section hidden>`; the visible form is rendered by JS after `ui.instantiateView('bootstrap.sysauth')`. The raw `#luci_username` exists in hidden DOM before the view is ready.
|
|
- **Fix**: After `page.goto('/cgi-bin/luci')`, wait for the spinner to disappear:
|
|
```ts
|
|
await page.waitForFunction(() => {
|
|
const view = document.querySelector('#view');
|
|
return view && !view.querySelector('.spinning');
|
|
}, { timeout: 30_000, polling: 500 });
|
|
await page.waitForSelector('#luci_username', { state: 'visible', timeout: 30_000 });
|
|
```
|
|
- **Post-login navigation**: LuCI with empty root password redirects to `/cgi-bin/luci/` (not `/cgi-bin/luci/admin/...`). Changed `waitForURL` to match `/cgi-bin/luci/`, then wait for the admin page spinner to disappear.
|
|
- **`resetToAmsterdam`**: Changed from `page.evaluate(fetch)` to `page.request.post('/ubus')` to avoid "Execution context destroyed" errors when the page navigates during test teardown.
|
|
|
|
### 2. `overview.js` Parenthesis Mismatch
|
|
- **Lines 67 & 208**: `_('UP (%sms').format(latencyMs)` had a missing closing parenthesis.
|
|
- **Fix**: Changed to `_('UP (%sms)').format(latencyMs)` in both locations.
|
|
|
|
### 3. Missing `luci-compat` Package
|
|
- **Symptom**: After clicking login, the page returned a 500 error: `module 'luci.ucodebridge' not found`.
|
|
- **Cause**: Modern OpenWrt snapshots moved `luci.ucodebridge` out of `luci-base` into a separate package `luci-compat`.
|
|
- **Fix**:
|
|
- Updated `Dockerfile.openwrt` to include `luci-compat`.
|
|
- Because `podman build` in this environment cannot fetch packages at build time (network is blocked inside the build container), a new image was produced by:
|
|
1. Running a temporary container with `--network host --entrypoint sh` from the existing image.
|
|
2. Installing `luci-compat` (and its dependencies) inside the temp container using `apk add`.
|
|
3. Committing the temp container to a new image tag.
|
|
4. Replacing the `localhost/zt-gateway-luci:dev` tag with the committed image.
|
|
|
|
### 4. Missing Volume Mounts for Menu & ACL JSONs
|
|
- **Symptom**: Navigating to `/cgi-bin/luci/admin/services/zt-gateway` returned 404: "No page is registered".
|
|
- **Fix**: Added `menu.d` and `acl.d` mounts to `docker-compose.yml`.
|
|
|
|
### 5. Stale ubus Socket on Container Restart
|
|
- **Fix**: Modified `docker/openwrt-luci-entrypoint.sh` to remove stale sockets and lock files before starting `ubusd`.
|
|
|
|
### 6. Network Setup (`docker/openwrt-luci-entrypoint.sh`)
|
|
- **Symptom**: `br-zt` had no IPv4 address because `ZTG_PING_IFACE=br-zt` caused the entrypoint to look for the IP on `br-zt` before it was created.
|
|
- **Fix**: Changed entrypoint to independently detect `SOURCE_IFACE` (the interface holding `10.99.12.x`) and move its IP to `BRIDGE` (`br-zt`), regardless of `ZTG_PING_IFACE`.
|
|
|
|
### 7. UCI Config Not Writable
|
|
- **Symptom**: `uci commit zt-gateway` failed with "I/O error". The backend could not persist `active_gateway` changes.
|
|
- **Root cause**: File-level bind mount (`./root/etc/config/zt-gateway:/etc/config/zt-gateway`) prevents atomic rename on btrfs-backed host filesystem.
|
|
- **Fix**: Changed compose to mount `./root/etc/config:/host-config:ro` and added entrypoint logic to copy `zt-gateway` config from `/host-config` into the container's writable overlay at startup.
|
|
|
|
### 8. UCI Config IPs Updated
|
|
- Updated `root/etc/config/zt-gateway` to use `10.99.12.x` (test network) instead of `10.11.12.x`.
|
|
|
|
### 9. Added State Reset to Test `beforeEach`
|
|
- `drain.spec.ts`, `switch.spec.ts`, and `health.spec.ts` now call `resetToAmsterdam(page)` in both `beforeEach` and `afterEach` to ensure each test starts from a clean state.
|
|
|
|
## Current Blocker: Extreme LuCI Cold-Load Latency
|
|
|
|
After installing `luci-compat`, login and page rendering work correctly, but **first-load times are extremely long** in fresh browser contexts:
|
|
|
|
| Step | Time (fresh context) | Notes |
|
|
|------|---------------------|-------|
|
|
| Login form render | ~11s | Wait for `bootstrap.sysauth` instantiation |
|
|
| Admin overview render | ~54s | `admin_status/index` view compilation |
|
|
| Admin overview (2nd try) | ~37s | Partial cache warm-up |
|
|
| ZT gateway page render | **>120s*** | `zt-gateway/overview` view never renders |
|
|
|
|
*The ZT gateway page spinner (`Loading view…`) persists beyond 120 seconds. Manual `ui.instantiateView('zt-gateway/overview')` returns instantly but does **not** replace the spinner—`#zt-gateway-root` is never created. There are **no JavaScript console errors** (`sessionid` and `token` are correctly set). The `ubus status` RPC returns correct data when called manually. Root cause of the silent render failure is **unknown**—possibly related to `luci-compat`'s Lua compatibility mode altering the `ui.instantiateView` pipeline, or a race condition in the module loader that only manifests on cold loads.
|
|
|
|
## Test Results
|
|
|
|
| Spec | Status | Notes |
|
|
|------|--------|-------|
|
|
| `login.spec.ts` | **PASS** (when run alone on warm container) | Full form submission + admin navigation works. |
|
|
| `overview.spec.ts` | **PASS** (when reached) | Table, radios, active marker assertions pass. |
|
|
| `switch.spec.ts` | **UNKNOWN** | Suite times out before reaching it reliably. |
|
|
| `drain.spec.ts` | **FAIL** | `beforeEach` times out during `login()` cold load. |
|
|
| `health.spec.ts` | **UNKNOWN** | Reaches test body but suite often times out first. |
|
|
|
|
## Infrastructure State
|
|
|
|
- **Running image**: `localhost/zt-gateway-luci:dev` (committed from temp container with `luci-compat`).
|
|
- **Container**: `openwrt-luci` (from compose, uses staging-dir volume mounts).
|
|
- **Networks**: `luci-app-zt-gateway_zt-exit-net` and `luci-app-zt-gateway_zt-gateway-lan` active.
|
|
- **br-zt**: Has `10.99.12.2/24`, routing works, ping to gateways succeeds.
|
|
|
|
## Remaining Work
|
|
|
|
1. **Diagnose ZT page cold-load hang**: Investigate why `ui.instantiateView('zt-gateway/overview')` silently fails to render on fresh browser contexts. Compare with a non-`luci-compat` image or test a vanilla LuCI page to isolate whether `luci-compat` is the cause.
|
|
2. **Evaluate test-timeout strategy**: Options:
|
|
- Increase `playwright.config.ts` timeout to 180s+ and accept 10+ minute suite runs.
|
|
- Implement a `globalSetup` that logs in once via browser to warm the server cache, then share `storageState` across tests.
|
|
- Revert `luci-compat` installation and find an alternative way to provide `luci.ucodebridge` (e.g., copy only the missing file into the image).
|
|
3. **Fix `drain.spec.ts` graceful drain panel**: Once login is stable, verify `zt-gateway-switch` in graceful mode actually creates `/var/run/zt-gateway-drain.pid`; if not, the test expectation may need to be relaxed for the containerized environment.
|
|
4. **Run full suite end-to-end** after the above blocker is resolved.
|
|
5. **Clean up debug scripts**: `debug-pw.js`, `debug-pw2.js`, `debug-pw3.js`, `debug-login-dom.js`.
|
|
6. **Rebuild image properly** once build-time network is restored so `luci-compat` is baked in without manual commits.
|
|
|
|
## Date: 2026-07-13
|
|
|
|
## Setup Wizard Implementation
|
|
|
|
### Completed
|
|
|
|
1. **Created `/usr/sbin/zt-gateway-setup`** (700+ lines) with subcommands:
|
|
- `status`, `setup-bridge`, `setup-routing`, `setup-dhcp`, `setup-wifi-ap`,
|
|
`setup-hotplug`, `setup-persistence`, `setup-all`
|
|
|
|
2. **Added `setup` ubus method** to `zt-gateway.uc` with regex validation
|
|
(ucode lacks `Array.indexOf()`).
|
|
|
|
3. **Added Setup panel** to `overview.js` with buttons for each setup command.
|
|
|
|
4. **Auto-discover deploy task** — `deploy:install` now discovers files from
|
|
`root/` and `htdocs/` instead of listing them individually.
|
|
|
|
### Bugs Found and Fixed During Setup
|
|
|
|
1. **`_cidr_to_mask` produced 3 octets** for /24 (`255.255.255` instead of
|
|
`255.255.255.0`). Fixed by padding to 4 octets.
|
|
|
|
2. **UCI values had embedded quotes** — `uci set "proto='static'"` stored
|
|
`'static'` instead of `static`. Fixed by removing shell quotes.
|
|
|
|
3. **Bridge ports hardcoded to `ztabc0`** — ZeroTier interface names are
|
|
randomized. Fixed by auto-detecting `zt*` interfaces.
|
|
|
|
4. **DHCP/WiFi AP referenced device name instead of interface name** — dnsmasq
|
|
and hostapd bind to interfaces, not devices. Fixed to use `zt_wiblan`.
|
|
|
|
5. **Firewall zone missing** — nftables fw4 has `policy drop`. Fixed by adding
|
|
`zt_wiblan` to the LAN zone.
|
|
|
|
6. **ZT interface lost IP when added to bridge** — ARP responses failed. Fixed
|
|
by ensuring ZT-assigned IP stays on the interface.
|
|
|
|
7. **Table 100 routed to self** — Default route pointed to `WIBLAN_GW` (local)
|
|
instead of exit gateway. Fixed by detecting exit gateway from UCI/routes.
|
|
|
|
8. **Bridge netmask /24 instead of /23** — Couldn't reach ZT subnet. Fixed by
|
|
defaulting to `/23` (255.255.254.0).
|
|
|
|
9. **ZeroTier Ethernet Bridging disabled** — L2 frames couldn't traverse
|
|
tunnel. Fixed by enabling in ZT network controller.
|
|
|
|
10. **DHCP range wrong in /23** — Clients got `10.11.12.x` instead of
|
|
`10.11.13.x`. Fixed by calculating correct offset (356 for /23 base).
|
|
|
|
### Key Learnings
|
|
|
|
- OpenWrt UCI uses **interface names** (not device names) for DHCP and WiFi
|
|
- Bridge netmask must be `/23` to cover both ZT (10.11.12.x) and WIBLAN (10.11.13.x)
|
|
- ZeroTier requires "Allow Ethernet Bridging" for L2 traffic
|
|
- nftables fw4 zones must explicitly include bridge interfaces
|
|
- Policy routing table 100 must route via exit gateway, not local IP
|
|
## Date: 2026-07-14
|
|
|
|
## Post-Reboot Routing Fixes
|
|
|
|
### Problem
|
|
|
|
After rebooting the production OpenWrt router, WiBLAN clients
|
|
(10.11.13.x) could not route traffic through the active exit
|
|
gateway (Amsterdam, 10.11.12.3). The `ip rule` directing WIBLAN
|
|
traffic to policy table 100 was missing entirely.
|
|
|
|
### Root Cause
|
|
|
|
Three gaps in the hotplug/setup-routing/switch pipeline:
|
|
1. Hotplug `ifup` case never installed the `from 10.11.13.0/24` ip rule
|
|
2. UCI persistence wrote routes but not the `network rule` section
|
|
3. `zt-gateway-switch` never verified the rule existed
|
|
|
|
### Fixes
|
|
|
|
1. Added ip rule to hotplug heredoc (`zt-gateway-setup`)
|
|
2. Added `network.zt_wiblan_rule` UCI section to `setup-routing` persistence
|
|
3. Added rule verification to `do_force()`/`do_graceful()` in `zt-gateway-switch`
|
|
|
|
### Additional fixes during the same session
|
|
|
|
- **BRIDGE_PORTS auto-detect**: Replaced broken awk-over-ip pipeline
|
|
with UCI lookup + `/proc/net/dev` fallback + interface existence
|
|
validation (busybox compatibility)
|
|
- **Route dev param**: Changed `dev $BRIDGE_PORTS` to `dev $BRIDGE`
|
|
in `setup-routing` (ZT interface has no IP when enslaved to bridge)
|
|
- **`ip rule replace`**: Replaced GNU-only `ip rule replace` with
|
|
`ip rule del` + `ip rule add` for busybox compatibility
|
|
- **Deploy stdin starvation**: Added `</dev/null` to `ssh`/`scp` in
|
|
`deploy:install` pipe loop
|
|
- **Deploy overwriting UCI**: Noted that `deploy:install` overwrites
|
|
`/etc/config/zt-gateway` with repo version, clobbering production
|
|
customizations (e.g. real Amsterdam IP 10.11.12.3 vs placeholder
|
|
10.99.12.3)
|