luci-app-zt-gateway/docs/PROGRESS.md

Progress: Playwright E2E Tests for luci-app-zt-gateway

Date: 2026-06-26/2026-06-27

Completed Fixes

1. login() Helper (e2e/utils/auth.ts)

• Root cause: LuCI login form lives inside <section hidden>; the visible form is rendered by JS after ui.instantiateView('bootstrap.sysauth'). The raw #luci_username exists in hidden DOM before the view is ready.
• Fix: After page.goto('/cgi-bin/luci'), wait for the spinner to disappear:

  await page.waitForFunction(() => {
    const view = document.querySelector('#view');
    return view && !view.querySelector('.spinning');
  }, { timeout: 30_000, polling: 500 });
  await page.waitForSelector('#luci_username', { state: 'visible', timeout: 30_000 });
  

• Post-login navigation: LuCI with empty root password redirects to /cgi-bin/luci/ (not /cgi-bin/luci/admin/...). Changed waitForURL to match /cgi-bin/luci/, then wait for the admin page spinner to disappear.
• resetToAmsterdam: Changed from page.evaluate(fetch) to page.request.post('/ubus') to avoid "Execution context destroyed" errors when the page navigates during test teardown.

2. overview.js Parenthesis Mismatch

• Lines 67 & 208: _('UP (%sms').format(latencyMs) had a missing closing parenthesis.
• Fix: Changed to _('UP (%sms)').format(latencyMs) in both locations.

3. Missing luci-compat Package

• Symptom: After clicking login, the page returned a 500 error: module 'luci.ucodebridge' not found.
• Cause: Modern OpenWrt snapshots moved luci.ucodebridge out of luci-base into a separate package luci-compat.
• Fix:
  • Updated Dockerfile.openwrt to include luci-compat.
  • Because podman build in this environment cannot fetch packages at build time (network is blocked inside the build container), a new image was produced by:
    1. Running a temporary container with --network host --entrypoint sh from the existing image.
    2. Installing luci-compat (and its dependencies) inside the temp container using apk add.
    3. Committing the temp container to a new image tag.
    4. Replacing the localhost/zt-gateway-luci:dev tag with the committed image.

4. Missing Volume Mounts for Menu & ACL JSONs

• Symptom: Navigating to /cgi-bin/luci/admin/services/zt-gateway returned 404: "No page is registered".
• Fix: Added menu.d and acl.d mounts to docker-compose.yml.

5. Stale ubus Socket on Container Restart

• Fix: Modified docker/openwrt-luci-entrypoint.sh to remove stale sockets and lock files before starting ubusd.

6. Network Setup (docker/openwrt-luci-entrypoint.sh)

• Symptom: br-zt had no IPv4 address because ZTG_PING_IFACE=br-zt caused the entrypoint to look for the IP on br-zt before it was created.
• Fix: Changed entrypoint to independently detect SOURCE_IFACE (the interface holding 10.99.12.x) and move its IP to BRIDGE (br-zt), regardless of ZTG_PING_IFACE.

7. UCI Config Not Writable

• Symptom: uci commit zt-gateway failed with "I/O error". The backend could not persist active_gateway changes.
• Root cause: File-level bind mount (./root/etc/config/zt-gateway:/etc/config/zt-gateway) prevents atomic rename on btrfs-backed host filesystem.
• Fix: Changed compose to mount ./root/etc/config:/host-config:ro and added entrypoint logic to copy zt-gateway config from /host-config into the container's writable overlay at startup.

8. UCI Config IPs Updated

• Updated root/etc/config/zt-gateway to use 10.99.12.x (test network) instead of 10.11.12.x.

9. Added State Reset to Test beforeEach

• drain.spec.ts, switch.spec.ts, and health.spec.ts now call resetToAmsterdam(page) in both beforeEach and afterEach to ensure each test starts from a clean state.

Current Blocker: Extreme LuCI Cold-Load Latency

After installing luci-compat, login and page rendering work correctly, but first-load times are extremely long in fresh browser contexts:

Step	Time (fresh context)	Notes
Login form render	~11s	Wait for bootstrap.sysauth instantiation
Admin overview render	~54s	admin_status/index view compilation
Admin overview (2nd try)	~37s	Partial cache warm-up
ZT gateway page render	>120s*	zt-gateway/overview view never renders

*The ZT gateway page spinner (Loading view…) persists beyond 120 seconds. Manual ui.instantiateView('zt-gateway/overview') returns instantly but does not replace the spinner—#zt-gateway-root is never created. There are no JavaScript console errors (sessionid and token are correctly set). The ubus status RPC returns correct data when called manually. Root cause of the silent render failure is unknown—possibly related to luci-compat's Lua compatibility mode altering the ui.instantiateView pipeline, or a race condition in the module loader that only manifests on cold loads.

Test Results

Spec	Status	Notes
login.spec.ts	PASS (when run alone on warm container)	Full form submission + admin navigation works.
overview.spec.ts	PASS (when reached)	Table, radios, active marker assertions pass.
switch.spec.ts	UNKNOWN	Suite times out before reaching it reliably.
drain.spec.ts	FAIL	beforeEach times out during login() cold load.
health.spec.ts	UNKNOWN	Reaches test body but suite often times out first.

Infrastructure State

• Running image: localhost/zt-gateway-luci:dev (committed from temp container with luci-compat).
• Container: openwrt-luci (from compose, uses staging-dir volume mounts).
• Networks: luci-app-zt-gateway_zt-exit-net and luci-app-zt-gateway_zt-gateway-lan active.
• br-zt: Has 10.99.12.2/24, routing works, ping to gateways succeeds.

Remaining Work

1. Diagnose ZT page cold-load hang: Investigate why ui.instantiateView('zt-gateway/overview') silently fails to render on fresh browser contexts. Compare with a non-luci-compat image or test a vanilla LuCI page to isolate whether luci-compat is the cause.
2. Evaluate test-timeout strategy: Options:
   • Increase playwright.config.ts timeout to 180s+ and accept 10+ minute suite runs.
   • Implement a globalSetup that logs in once via browser to warm the server cache, then share storageState across tests.
   • Revert luci-compat installation and find an alternative way to provide luci.ucodebridge (e.g., copy only the missing file into the image).
3. Fix drain.spec.ts graceful drain panel: Once login is stable, verify zt-gateway-switch in graceful mode actually creates /var/run/zt-gateway-drain.pid; if not, the test expectation may need to be relaxed for the containerized environment.
4. Run full suite end-to-end after the above blocker is resolved.
5. Clean up debug scripts: debug-pw.js, debug-pw2.js, debug-pw3.js, debug-login-dom.js.
6. Rebuild image properly once build-time network is restored so luci-compat is baked in without manual commits.

Date: 2026-07-13

Setup Wizard Implementation

Completed

1. Created /usr/sbin/zt-gateway-setup (700+ lines) with subcommands:

   • status, setup-bridge, setup-routing, setup-dhcp, setup-wifi-ap, setup-hotplug, setup-persistence, setup-all

2. Added setup ubus method to zt-gateway.uc with regex validation (ucode lacks Array.indexOf()).

3. Added Setup panel to overview.js with buttons for each setup command.

4. Auto-discover deploy task — deploy:install now discovers files from root/ and htdocs/ instead of listing them individually.

Bugs Found and Fixed During Setup

1. _cidr_to_mask produced 3 octets for /24 (255.255.255 instead of 255.255.255.0). Fixed by padding to 4 octets.

2. UCI values had embedded quotes — uci set "proto='static'" stored 'static' instead of static. Fixed by removing shell quotes.

3. Bridge ports hardcoded to ztabc0 — ZeroTier interface names are randomized. Fixed by auto-detecting zt* interfaces.

4. DHCP/WiFi AP referenced device name instead of interface name — dnsmasq and hostapd bind to interfaces, not devices. Fixed to use zt_wiblan.

5. Firewall zone missing — nftables fw4 has policy drop. Fixed by adding zt_wiblan to the LAN zone.

6. ZT interface lost IP when added to bridge — ARP responses failed. Fixed by ensuring ZT-assigned IP stays on the interface.

7. Table 100 routed to self — Default route pointed to WIBLAN_GW (local) instead of exit gateway. Fixed by detecting exit gateway from UCI/routes.

8. Bridge netmask /24 instead of /23 — Couldn't reach ZT subnet. Fixed by defaulting to /23 (255.255.254.0).

9. ZeroTier Ethernet Bridging disabled — L2 frames couldn't traverse tunnel. Fixed by enabling in ZT network controller.

10. DHCP range wrong in /23 — Clients got 10.11.12.x instead of 10.11.13.x. Fixed by calculating correct offset (356 for /23 base).

Key Learnings

• OpenWrt UCI uses interface names (not device names) for DHCP and WiFi
• Bridge netmask must be /23 to cover both ZT (10.11.12.x) and WIBLAN (10.11.13.x)
• ZeroTier requires "Allow Ethernet Bridging" for L2 traffic
• nftables fw4 zones must explicitly include bridge interfaces
• Policy routing table 100 must route via exit gateway, not local IP

Date: 2026-07-14

Post-Reboot Routing Fixes

Problem

After rebooting the production OpenWrt router, WiBLAN clients (10.11.13.x) could not route traffic through the active exit gateway (Amsterdam, 10.11.12.3). The ip rule directing WIBLAN traffic to policy table 100 was missing entirely.

Root Cause

Three gaps in the hotplug/setup-routing/switch pipeline:

1. Hotplug ifup case never installed the from 10.11.13.0/24 ip rule
2. UCI persistence wrote routes but not the network rule section
3. zt-gateway-switch never verified the rule existed

Fixes

1. Added ip rule to hotplug heredoc (zt-gateway-setup)
2. Added network.zt_wiblan_rule UCI section to setup-routing persistence
3. Added rule verification to do_force()/do_graceful() in zt-gateway-switch

Additional fixes during the same session

• BRIDGE_PORTS auto-detect: Replaced broken awk-over-ip pipeline with UCI lookup + /proc/net/dev fallback + interface existence validation (busybox compatibility)
• Route dev param: Changed dev $BRIDGE_PORTS to dev $BRIDGE in setup-routing (ZT interface has no IP when enslaved to bridge)
• ip rule replace: Replaced GNU-only ip rule replace with ip rule del + ip rule add for busybox compatibility
• Deploy stdin starvation: Added </dev/null to ssh/scp in deploy:install pipe loop
• Deploy overwriting UCI: Noted that deploy:install overwrites /etc/config/zt-gateway with repo version, clobbering production customizations (e.g. real Amsterdam IP 10.11.12.3 vs placeholder 10.99.12.3)