FooGolf developers
Documentation › docs/waveform-capture.md

Waveform capture → cloud (design)

Status: shipped, firmware v0.8.44 (cross-repo: foodoublebassesp32 + foogolf-ota-cloud).

v3 (fw 0.8.166, 2026-08-03) — the URL batch is gone

The device now fetches ONE presigned S3 URL per shot, on demand, in ShotUploader::enqueue() on the main loop. No batch, no ring, no prefetch, and no cap on shots per power cycle.

The v2 batch below prefetched 48 URLs in a single boot-time TLS call. That imposed a hard 48-shot limit per power cycle and made "why did uploads stop?" needlessly hard to answer — the state that explained it lived in a ring buffer filled minutes earlier. It existed only because per-shot TLS was unaffordable (mbedTLS needs ~30–40 KB contiguous internal heap; the device had ~43 KB), and v0.8.163 removed exactly that pressure by moving LVGL's 48 KB pool out of internal RAM into PSRAM (static RAM 210128 → 160976).

Trade-off accepted deliberately: each upload now blocks the main loop for a TLS handshake. It must — the NetTask worker's stack is in PSRAM and can't run mbedTLS. It lands between scans, right after a shot, never during capture, and uploading is a diagnostic mode. A heap gate (kMinTlsMaxAlloc, 40 KB) drops the shot rather than attempt a handshake that would fail and fragment the heap on its way out — so the failure mode that motivated v2 is guarded, not ignored.

Notable consequences: - The per-device upload-level mask and the uploads_disabled toggle now refresh on every shot instead of at boot, so a dashboard change takes effect on the next shot. - uploads_disabled still latches until restart, so a disabled device doesn't pay a pointless handshake per shot. - PUT retries reuse the same URL (valid 12 h), so a transport blip can't consume a budget. - The "URLs left" line is gone from the Wi-Fi status page — there's no remaining budget to report. - PSRAM use drops: the response buffer goes ~88 KB → 4 KB, the URL store ~81 KB → 3.4 KB.

The boot OTA check-in runs before any shot uploads, so these handshakes cannot break that boot's update path; the next boot starts from a fresh heap regardless.

CURRENT ARCHITECTURE (v2 — supersedes the "Architecture" section below)

The original design had the device POST signed metadata to /record-shot (TLS) which returned a presigned PUT URL. That doesn't work on this hardware: the device has only ~43 KB contiguous internal heap, an mbedTLS handshake needs ~that, and a per-shot TLS upload fragments the heap until even OTA check-in fails (-17040 RSA/BIGNUM memory allocation failed). See [[esp32-internal-heap-tls-constraint]].

v2 path (no per-shot TLS): 1. Device fetches a BATCH of presigned S3 PUT URLs via one TLS call to POST /get-upload-urls (HMAC-signed), done at idle on the main loop, gated on ESP.getMaxAllocHeap() >= 40 KB + throttled. URLs held in PSRAM (ring buffer, batch of 8). 2. On a GOODSWING, the firmware serializes the host frame and PUTs it to the next presigned URL over plain HTTP (no TLS, light on heap — safe even right after a shot). 3. index_shot Lambda (S3 ObjectCreated on foogolf-waveforms/shots/) reads the frame's JSON header (Range request) and writes the foogolf-shots row — replaces /record-shot.

Because uploads no longer use TLS, the heap doesn't fragment and OTA/check-in stays healthy. record_shot Lambda was removed from the stack on 2026-09-11 (superseded; last invoked June 2026). The S3 object format and the foogolf-shots/dashboard schema are unchanged.

SELECTABLE UPLOAD LEVELS (added firmware v0.8.134, 2026-06-25)

Originally only GOODSWING swings uploaded (hard-coded in ShotProcessor::beginShot). The dashboard can now opt a device into uploading other message levels too — chiefly to diagnose hardware (e.g. turn on BADSWING uploads to inspect a suspected miswired sensor).

The sections below are the original v1 design, kept for history.

Cross-repo feature spanning foodoublebassesp32 (firmware) and foogolf-ota-cloud (backend).

Goal

Capture per-swing waveform data from the device and store it in the cloud for later analysis. A UI for browsing/visualising waveforms will come later — this phase only lands the data.

Decisions (made 2026-06-14)

Architecture

Mirrors the existing OTA pattern: small signed JSON through API Gateway + Lambda, large binary blob direct to S3 via a presigned URL (never through Lambda — avoids base64 inflation, the HTTP API payload ceiling, and Lambda transfer cost).

ShotProcessor::beginShot (firmware)
  ├─ forward to integration (existing)
  ├─ send USB JSON+binary frame (existing)
  ├─ if GOODSWING && WiFi connected: enqueue cloud upload   ◄── NEW (background task)
  └─ UI hand-off (existing)

cloud upload (background FreeRTOS task, best-effort):
  1. POST /record-shot  (signed JSON metadata, ~3-5 KB)
         → Lambda verifies HMAC, writes foogolf-shots row, returns presigned S3 PUT URL
  2. HTTP PUT waveform blob → S3 (foogolf-waveforms bucket), direct, plain HTTP

The upload runs on a background task so a slow/absent Wi-Fi link never stalls the shot-result page or the UI hand-off.

Auth

Reuses the existing device HMAC scheme verbatim — X-Device-Id / X-Timestamp / X-Signature headers, signature = HMAC-SHA256(secret, f"{ts}." + raw_body), secret looked up from the foogolf-devices table. See _verify_signature in check_for_update/get_firmware_download. The /record-shot Lambda duplicates that function (each Lambda is its own deployment package, matching the repo's existing no-shared-layer convention).

Storage

S3 — foogolf-waveforms (new bucket)

Separate from foogolf-firmware (different lifecycle — firmware is versioned+retained forever; waveforms may get a retention/expiry policy later). Encrypted (AES256), public access blocked.

DynamoDB — foogolf-shots (new table)

PAY_PER_REQUEST, encrypted, PITR on. Holds the queryable metadata so the future UI can list/filter shots without fetching blobs.

Per-swing size

~100–150 KB typical, ~490 KB worst case (4 sensors × up to 10 000 samples × 12 B/sample + peaks + JSON). Fine for S3; the blob never touches DynamoDB (400 KB item limit) or Lambda.

Cloud changes (foogolf-ota-cloud)

Firmware changes (foodoublebassesp32, after cloud is live)