Device control channel + remote support cast (2026-09-13, fw 0.8.499 / 0.8.500)
Design record for two things shipped together on 2026-09-13 (firmware 0.8.499 to staging; foogolf-ota, foogolf-dashboard and foogolf-mirror-syd stacks updated by review-first deploys; dashboard frontend redeployed):
- A device control channel - the dashboard writes a device-row field and the device notices within 30 s instead of at its next boot.
- The remote support cast, rebuilt on the LAN cast - the dashboard's Support page shows a device's screen and presses its buttons, started from the dashboard itself.
The cost side (per-device budget, retention) is
cloud-cost-budget-per-device.md.
The retired predecessor ("Remote support cast", the cloud relay of
0.8.186-0.8.492) is recorded in
serverless-screen-mirror-design.md
and foodoublebassesp32/docs/removals-2026-09-11.md; its LAN section 9
is what this design reuses.
1. Why a control channel
Before 0.8.499 nothing polled the cloud unless Online play was hosting or
joined: the OTA check-in is deterministic (reset, WiFi connect, manual
check - never a timer, ota-stuck-2026-08-postmortem.md), the waveform
uploader fetches its config per shot only while uploads are ON (a device
switched OFF never fetches, so OFF -> ON waited for a reboot), and the OTA
channel was adopted at the boot check-in. "Start remote support from the
dashboard, immediately if possible" needs a channel that exists whenever
the device is powered.
2. The control poll
- Device -> cloud:
POST /device/poll, plain http throughshare.foogolf.com(/device/*behavior -> the foogolf-ota HttpApi), HMAC-signed like every device request, from the SimShare core-0 task (SimShare::ctlPoll): every 30 s while WiFi is up, whatever the Online play role (Off: it is the only traffic; Host / Guest: it runs between the long polls), and at once when the boot gate opens or 12+ telemetry events are queued. Nothing before the boot check-in (SimShare::openBootGate), nothing while a main-loop TLS call runs (NetActivity::busy()), and never an OTA offer - the check-in stays deterministic. - Body:
{"cv": <ctl version last applied>, "fw": "0.8.499", "ev": [ ...telemetry events... ]}. - Response:
{"status":"ok", "ctl": {cv, cast, uploads, levels, channel, label}, "acked": n}-ctlis the set of dashboard-owned device-row fields the firmware acts on, under one versionctl_verthat the dashboard'supdate_devicebumps on every write (ADD ctl_ver :one). The same object rides in/sim/polland/sim/joinresponses when the body'scvis stale, so a host or guest already holding a long poll needs no extra request; the ctl builder is a copy indevice_controlandsim_share(the OTA stack has no shared layer - keep them identical). - Device side:
SimShare::takeControlhands a differing object toApp::serviceDeviceControl(main loop, in the idle housekeeping next toserviceSimShareHost;SimShare::controlVersion()is inscanShouldYield), which applies:ShotUploader::applyRemoteConfig(uploads gate + levels, exactly whatget-upload-urlswould say at the next shot),Provisioning::adoptChannelwhen the channel differs (no check-in triggered), and the cast flag (section 4). The label is not applied live (cosmetic; the boot check-in caches it) - the library keeps the setter private, revisit if it matters. - Cost: ~$0.35 per device-year at 3 h powered a day (API Gateway + CloudFront requests + a 50 ms Lambda; no DynamoDB write unless events arrived). A 20 s held long poll would have cost ~$5 a year, which is why this is a short poll.
3. Telemetry events
Best-effort usage record for every customer's device, the thing the cost
budget explicitly allows: SimShare::noteEvent(json) queues a compact
JSON object (<= 120 chars, ring of 24, oldest dropped when full) and the
next control poll carries up to 20; the cloud acks how many it stored and
the device drops exactly those (a failed poll retries them). Kinds:
| k | when | fields |
|---|---|---|
boot |
the boot gate opens | fw |
game |
Menu_StartExperience (every Play / New / Resume / Basic view, a guest following its host) |
m = gameMode |
shot |
ShotProcessor::beginShot (a good swing) |
mps, face, rot, aoa |
Every event carries t (epoch seconds, 0 before SNTP - the cloud stores
"now" for a stamp more than a week off). Rows land in foogolf-events
(device_id, ts = ms + index) with a one-year TTL (expires); numeric
fields are clamped and rounded to 3 dp, unknown fields dropped. No
dashboard view yet - the table is for Athena-style analysis later.
4. Remote support cast
What is reused, and what the old relay got wrong
The browser cannot reach a device's :9202 server behind NAT, so the
device pushes. Reused verbatim: the PSRAM ring and batch packer
(serveBatched), the RECT / HEARTBEAT / AUDIO / AUDIO_CUT wire packets,
the whole cast-viewer.html (decode, re-timing, xBR, audio, shield,
keypad) and NET/RemoteKeys. The retired relay reused these too; what
broke it was the transport around them - a hand-rolled overlapped HTTP
client on the streamer task, an ingest Lambda, an API Gateway WebSocket
per region, base64 text frames and a viewer bucket. This design keeps the
streamer task on the ordinary HTTPClient + Signing::addSignedHeaders
path that SimShare has run for months, and uses the dashboard's EXISTING
WebSocket (the one that pushes new shots to open browsers).
Data path
device (cast task) --POST /cast/ingest (batch, HMAC, plain http)--> share.foogolf.com
^ | /cast/* -> foogolf-dashboard HttpApi
| {viewers, keys[], next_seq, stop, kf} v
| cast_ingest Lambda
| | post_to_connection {"type":"cast","d":base64}
| v
| dashboard WebSocket (foogolf-dashboard-ws)
| | connection rows with cast_device == D
| v
+--- cast-inputs FIFO <-- ws_default "key" <-- Support page (cast-viewer.html?cloud=1) keypad
- Firmware (
src/NET/ScreenCast.{h,cpp}): aCloudSinknext to theLanSink, both inside oneSessionSink; one streaming session serves whichever sinks are on (section 5). The cloud sink POSTs each batch to/cast/ingestwithX-Cast-Since(keys drained) andX-Cast-Wait(0 for content, 500 ms for a heartbeat - the cloud holds an idle poll that long for a key press, so a press on a static screen lands within about half a second); the response'skeysgo toRemoteKeysexactly as the LAN server feeds them,viewersdrives the cadence (300 ms idle polls with a viewer, one empty poll every 2 s with none - content is discarded while nobody watches, a join keyframes),stopends the sink,kfrequests a keyframe. Rate is bounded by the existing batch policy (a batch per frame-last flush, >= 50 ms apart, 200 ms age cap): at most ~4-5 POSTs a second while the screen changes.slotFree()is!NetActivity::busy()and the busy hook now also waits onScreenCast::cloudWaitIdle, so a main-loop TLS call never overlaps the task's request. The cast task stack grew 16 -> 24 KB (PSRAM) for HTTPClient + signing. Started / stopped ONLY byApp::serviceDeviceControlfrom thecastflag; never persisted (the flag re-arms it after a reboot via the first control poll); a session that ended itself (10 min without a viewer, 30 consecutive failed POSTs) stays ended until the dashboard asks again (its next PATCH bumpsctl_ver). The device flashes "Remote support cast started / ended" and its cast LED is green while any sink streams; Device status showsSupport cast: <state>, N viewer(s) / posts fails keys. - Cloud (foogolf-dashboard stack):
CastIngestFnon the dashboard HttpApi atPOST /cast/ingestwith the JWT authorizer overridden to NONE (HMAC inside), reached from the device through a newdashboard-apiorigin +/cast/*behavior on the Sydney distribution;WsDefaultFnon the WebSocket$defaultroute (sub/unsub/key, browser-side);CastInputsTable(foogolf-dash-cast-inputs, device_id + seq, TTL 60 s);ws_disconnectclears the device row'scastflag (and bumpsctl_ver) when the LAST subscriber of that device disconnects;update_deviceacceptscast;cast_ingeststampscast_last_at/cast_viewerson the device row at most every 5 s. The WebSocket deployment resource was renamedWsDeployment2so the new route went live (a Deployment only redeploys when its logical id or properties change). - Dashboard: the Devices page has a Support column -
Open, or◐ requestedwhile the flag is set, or● livewhen the device posted within 15 s. Opening it navigates toSupportView, which PATCHes{cast:true}, embeds/cast-viewer.html?cloud=1&ws=...&token=...&dev=...(the firmware's page copied bydashboard/scripts/sync-cast-viewer.pyintofrontend/public/- the firmware copy is canonical; run the script after editing it and commit both) and shows the state the viewer reports bypostMessage(requesting / waiting for the device / live). Stop support cast PATCHes{cast:false}; the device ends the session at its next ingest (stop). Cloud mode in the viewer: connect to the dashboard WebSocket with the Cognito access token, send{"action":"sub","device_id"}, decode{"type":"cast","d":base64}text frames (API Gateway's WebSocket is text-only), send keys as{"action":"key","device_id","k":"kd:N"}; everything else is the LAN page unchanged.
Several devices at once (the UI question)
Each Support tab is its own WebSocket subscription to one device, so the
support team can open a tab per device; the Devices page shows every
device's state (requested / live) in one column. Leaving a Support page
does NOT clear the flag by itself - the $disconnect Lambda clears it
only when the last viewer of that device is gone, so a second person
watching the same device keeps it streaming, and a tab that merely
reloads reconnects, re-subscribes and re-arms within seconds. The
explicit Stop button clears it for everyone. On the device side the
session also ends itself after 10 minutes without any viewer.
5. Support cast vs. LAN cast - does one disable the other?
No. They are two sinks of ONE streaming session (SessionSink): one
ring, one batch, delivered to every active sink, each started and
stopped live from its own command (Controls > Screen casting for the LAN
sink, the dashboard flag for the cloud sink); the session ends when
neither is on. What you should expect while both run:
- A cloud POST is synchronous on the shared streamer task (~200-300 ms round trip from New Zealand through CloudFront Sydney to us-east-1), so LAN viewers get the cloud's pace during a support session; the LAN viewer's adaptive re-timing absorbs that.
- Cloud failures never end the LAN sink and vice versa (each sink's give-up counter is its own; the composite reports none to the loop).
- WiFi loss ends the LAN sink (its session is over) but the cloud sink waits for WiFi to return (the dashboard's flag outlives a blip).
- Keys from both viewers feed the same
RemoteKeys, indistinguishable from the switches, as before.
6. Cost of a support session
Roughly $0.06-$0.08 an hour per session with audio (4-5 POSTs a second while the screen changes: API Gateway $1/M + CloudFront requests + a short Lambda + WebSocket messages per viewer + data), less while the screen is static (300 ms idle polls held 500 ms are cheap), and about $0.004 an hour while the flag is set but nobody watches (one empty poll every 2 s). Sessions are explicit and end themselves, so there is no always-on cost. Borne by the supporter / coach, outside the per-device budget.
7. Verification
- Cloud:
python scripts/device-smoke.py [--share](foogolf-ota-cloud):/device/pollanswers a ctl object and acks an event (row seen infoogolf-events),/cast/ingestidle poll and a 60 KB batch answer ok - both directly and throughshare.foogolf.com(2026-09-13, all pass). - Firmware: builds, native tests pass, published to staging as 0.8.499,
then 0.8.500 the same day: 0.8.499's viewer page shipped with a
broken regex literal in the new cloud-mode code (a whole-script
SyntaxError - the LAN viewer would not have run either); caught by
node --checkon the page's script, which is now the rule before publishing a viewer change. Bench verification pending (2026-09-13): install 0.8.500 on the bench unit; on the dashboard's Devices page press Support > Open for it; expect "Remote support cast started" on the device within 30 s, the picture on the Support page, keypad presses moving the menu; Stop ends it; also flip Uploads off/on and a level on the Devices page and confirm Device status / the next shot follows within 30 s; then a LAN cast at the same time as a support cast.
8. Open items
- Label changes are not applied live (needs a library setter).
- The events table has no dashboard view; the per-shot DynamoDB
metarow is still 3 KB (see the cost note's "what to watch"). - The dashboard passes the Cognito access token to the viewer iframe as a
query parameter (the WebSocket
$connectalready takes it that way). - The support cast has no on-device opt-out yet ("Allow remote support" setting) - the device shows a flash and a green cast LED instead.