Skip to main content
Synced from the repo — do not edit here

Canonical source: docs/claude/one-true-source-plan.md. This page is generated by docs/scripts/sync-handbook.mjs. Edit the source file in the repo; changes appear here on the next build.

One True Source — End-to-End Build Plan

Status: DEFINITIVE build contract (2026-07-15). Synthesized from 5 parallel code audits + the June discovery doc (plan-metrics-data-contract.md) + the July divergence audit (data-source-of-truth-audit.md). Nothing gets built until this is signed off.

Scope: make every value byte-identical across all four surfaces — basisflow-web (provider), basisweb (client portal), basishybrid (mobile), and Atlas (AI) — for four domains: biomarkers/ wearables · labs · protocols · benchmarks (verdicts).

0. Why the prior attempt failed, and the one rule that makes this correct​

The June investigation was a read-side contract audit — it aimed to point every reader at the same field (dailyDigest.zoneNMinutes). That can't produce 1:1 because the divergence is at write/compute, not read: the "same" number is (a) computed 4+ different ways, (b) double-counted on web, and (c) holds an arbitrary source's value (last-write-wins between Oura and Apple).

THE RULE (applies to all four domains):

One value, computed once, at write; every surface reads it verbatim; delete every downstream re-computation; and prove 1:1 with the parity harness (tools/parity_harness.js) before and after each change.

This only works if the fix is COMPLETE. If any recompute survives (web's, mobile's, Terra's) or the source-precedence isn't deterministic, it silently fails to 1:1 again — exactly like last time. Every domain below has an explicit DELETE-LIST that must ship, not just an ADD-LIST.

1. WEARABLES / BIOMARKERS (30+ metrics — steps, exercise, maxHr, rhr, hrv, sleep, zones, calories, …)​

1.1 Canonical source​

users/{uid}/dailyDigest/{localDate} — per-day scalars, derived from the extracted map on every users/{uid}/healthSummaries doc (already the "single-source-of-truth biomarker values keyed by canonical field names", per plan-metrics-data-contract.md §0/§5.2). Clinic path is a server mirror via functions_health_mirror.py. Field names: biomarker-utils.ts:167-231 (EXTRACTED_KEY_MAP).

1.2 The real problem is MULTI-SOURCE, not zones-only​

The same metric arrives from Apple HealthKit, Terra→{Oura/WHOOP/Garmin/Fitbit/Withings/Polar/Samsung}, soon Google Fit, and manual — and today last-write-wins silently drops all but the last writer (proven: same-day rhr Oura 49 vs Apple 71 → digest kept whichever synced last; steps recorded by both, dropped). _sources[field]={device,writer,at} provenance IS stamped (service_health_firestore_writer.dart:2102-2110) but never read before the overwrite. The user's heartDataPreferences.devicePreferences (an ordered priority list, model_user.py:327 / preferences.dart:1982) is consulted only on read (motion_ffi.dart:150), never at write.

1.3 Resolution model — DECIDED 2026-07-15 + REFINED by real-data preview (strength-default authority)​

Original Option A was "the user's devicePreferences order is the winner." The #381 dry-run preview (tools/resolve_preview.js, run on George's real data) falsified that: the stored legacy prefs are uncurated + polluted (his heartDataPreferences = [Apple Watch, Hidrate] — a water bottle in the heart list; "Apple Watch first" for RHR resolves to a physiologically implausible 71/79 bpm vs Oura's 49/62). Honoring them verbatim would degrade clinical data. Refined + confirmed model:

Resolution pipeline (per metric, per day) — this is the validated #381 logic:

  1. VENDOR-COLLAPSE — group candidate readings by normalized CATEGORY (Dart _normalizeSourceCategory port; "Oura via Terra" + "Oura via Apple Health" → one Oura). Within a vendor prefer Terra-direct (metaSource=='t') over Apple-relayed, then recency. (Kills the [49,49,49] dup churn.)
  2. ELIGIBILITY GATE — a source is eligible for a metric only if it's a known producer of it (in the metric's strength list). Excludes a water bottle from heart. If none eligible (phone-only/odd device), fall back to all candidates by recency so the user still gets a value.
  3. AUTHORITY = clinical STRENGTH-DEFAULT (below). The legacy display-cycling devicePreferences are NOT trusted as clinical truth. A user preference overrides ONLY when it's a deliberate, metric-valid choice (survives the gate + carries a future "curated" marker set via the cycle-to-drill UI). Recency breaks ties only within a tier.
  4. Winner = the one stored value; losers → _shadow. Every surface reads the headline + keeps cycle-to-drill.

Proven on real data: rhr 0 regressions (3 multi-source days → Oura), steps 0 regressions (15 multi-source days → Apple Watch over iPhone), all deterministic. Sequencing guard: resolved value must be correct BEFORE any UI hides sources.

Two identity/keying rules that make it robust:

  • Key on stable CATEGORY, not raw sourceId. _normalizeSourceCategory (service_health_firestore_writer.dart:4393) already yields "Apple Watch"/"iPhone"/"Oura"/… BUT DeviceInfo equality/hashCode is keyed on sourceId (device_info.dart:183) → an Apple UUID re-issue reads as a new device and breaks preference-matching. Fix: match preferences + dedup on the normalized category. (This is the Apple-identifier-instability fix.)
  • Google Fit forward-compat: one more category (enums.py:1110 MetaSource.google defined) — slots into _normalizeSourceCategory + becomes a selectable device in the preference list. Zero rework.

STRENGTH-DEFAULT = the authority (not just the unset fallback), CONFIRMED 2026-07-15: rank by category strength; recency breaks ties only WITHIN a tier (never recency-first — that flips every sync). Overridden ONLY by a deliberate + metric-valid curated preference; losers → _shadow.

MANDATORY pre-step — collapse same-vendor multi-path to ONE candidate. Dedup by vendor, not ingestion path: "Oura via Terra" and "Oura via Apple Health" = one Oura. Within a vendor prefer the richer path: Terra-direct > Apple-Health-relayed (Terra carries structured scores/zones/stages the Apple Health relay flattens). This is the biggest stability win — kills the Oura-vs-Oura sync flip.

Metric familystrength order (recency breaks ties WITHIN a tier)
rhr · hrv · sleep · spo2 · recovery/sleep scoresring (Oura) → watch/strap → tracker
stepsApple Watch / wearable → dedicated tracker → iPhone LAST (fallback only; not excluded, so phone-only users still get steps)
exercise mins · zones · active-calories · workouts · maxHrApple Watch → Garmin/WHOOP → tracker
weight · body-composition · BP · waistmanual / smart scale → tracker
glucoseCGM (Dexcom) → manual
nutrition · mindfulness · fastingmanual / app

1.4 The FOUR max-HR / zone computations that must collapse to ONE​

  1. app UI analyzer_hr.dart:97-101,185 — maxHR = measured ?? 220−age(default 35); interval-walk; weighted scalar
  2. app digest writer service_health_firestore_writer.dart:6290,6399-6419 — DOB 220−age clamp(100,210) fb 185; interval-walk
  3. web recompute MetricsTab.tsx:131-181 — 220−age else absolute 120/140/160/180; proportional-by-count; then SUMS onto digest zones (double-count) :1278-1308
  4. Terra per-workout service_health_firestore_writer.dart:6296 — vendor's own zones (each vendor differs)

Same metric class for exercise/rhr/sleep: web=latest · Atlas=avg · digest=keep-max/sum · mobile=local sum — four aggregations of one number.

1.5 Writers (18 total)​

16 client-side in service_health_firestore_writer.dart (_digestAppend:2085 / _digestAppendMulti:2119 — merge=true, last-write-wins) + add_manual_metric (functions_clinic.py:13880) + no server writer today (Terra → mirror_firestore.py:41 writes healthSummaries only; digest populates only when the app runs → web-only/backgrounded clients get nothing).

1.6 Readers​

  • basisflow-web MetricsTab dailyDigest primary (:871) + recompute (:1071-1231); PlanTab both paths.
  • basisweb portal/health/page.tsx:305 reads healthSummaries, recomputes, shows no wearables (gap).
  • basishybrid recomputes from local SQLite (Rust FFI) — never reads the server digest for display.
  • Atlas _read_metric_from_daily_digest (functions_ai_agent.py) — now has zones (#382 shipped); aggregation/tz still diverges from web.

1.7 DELETE-LIST (must ship or 1:1 fails)​

  • MetricsTab.tsx:131-181 computeZonesFromHrSamples + :1071-1231/:1278-1308 the recompute + double-count merge → read digest verbatim (#394).
  • mobile local zone/metric recompute-for-display → read the server digest (wt/hybrid).
  • redundant per-zone hrzone* healthSummaries docs + hrDaily.z1..z5 once server-derived (P1, cost).
  • last-write-wins in _digestAppend (replace with precedence, #393).

1.8 ADD-LIST​

  • Server-side per-day digest writer (#381) — new functions_digest_writer.py (or extend mirror_firestore/terra_adapter): read a day's healthSummaries, group by metric by normalized category, resolve via the user's per-metric devicePreferences order (default table §1.3 only when unset), write ONE value + _sources winner + _shadow losers to dailyDigest/{localDate}. Batch per-DAY, debounced — NOT per-summary (the §9 cost multiplier: per-summary writes already fire mirror + RAG embedding at 4-5×). Callsites: post Terra webhook, post add_manual_metric, + a daily backfill cron. Reads devicePreferences from the user doc server-side (today it's Dart-only + display-only).
  • One shared HR/zone/aggregation resolver module (shared_hr_resolver) — single maxHR resolution + zone edges + per-field aggregation + tz — used by the server writer AND Atlas AND any surface that still computes.
  • Preference-at-write, category-keyed (#393) — resolve the winner from the user's devicePreferences order matched on normalized category (NOT raw sourceId — the Apple-UUID-instability fix, device_info.dart:183 equality must move to category); loser → _shadow. Applies in BOTH the server writer and the app _digestAppend. (Groundwork shipped: add_manual_metric now stamps _sources.)
  • Cycle-to-drill on every surface — basisflow-web + portal gain the source-cycle affordance basishybrid already has (route_summary.dart:707), reading _sources + _shadow so a provider can inspect Oura-vs-Apple. Headline stays the resolved value.
  • Persist selection as preference (optional nicety) — when a user cycles + pins a source, write it into devicePreferences so the choice propagates to the write + all surfaces (today the cycle arrows are ephemeral widget state).

1.9 1:1 proof​

parity_harness.js per metric across {digest, flow-web, atlas, basisweb, mobile*} → GREEN. --sources mode shows winner vs _shadow losers. Mobile is a device-bridge blind spot until it reads the digest.

2. LABS (value · unit · verdict)​

2.1 Canonical source​

users/{uid}/labs/{docId} + clinic mirror written in the same batch. Shape: {analyteKey, value(canonical unit), unit(canonical), originalValue/originalUnit, source, collectedAt, min/maxRangeValue(normalized), verdict(new), verdictSource(new)}.

2.2 Six ingestion doors → three code paths​

  • import_lab_values (functions_ai_documents.py:2336) — provider manual + PDF scan + general manual. ✅ _canonicalize + _normalize_unit + deterministic ids + atomic mirror.
  • Junction webhook (functions_junction.py:1794) — ✅ now normalized (#383 shipped); preserves ranges/interpretation normalized.
  • Mobile direct Dart writes — ❌ scan (modal_activity_details.dart:2084) + manual typed (modal_add_summary.dart:157) write raw units, random ids, user-path only (#385).

2.3 The unit-registry problem (3-4 disagreeing authorities)​

lab_analytes.json (784, display+ranges) · lab_unit_conversions.json (222, the one used at write) · Dart BasisLabType (hardcoded units) · TS lab-analytes-generated.ts (generator generate_lab_analytes_ts.py missing → hand-drifted). → #386: one generated registry from lab_analytes.json + CI drift check.

2.4 Verdicts — none persisted today → Atlas can't match​

Every surface computes the verdict on read (LabsTab.tsx:1944, analyzer_biomarkers.dart:402, basisweb own DEFAULT_LAB_BENCHMARKS), and Atlas has no engine — it only echoes stored flags (functions_ai_agent.py:2569) which don't exist for most labs. Fix = #392 persist the computed verdict at write (both import_lab_values

  • Junction) so Atlas reads what the UIs show.

2.5 DELETE-LIST / ADD-LIST​

  • DELETE: mobile raw writes (modal_activity_details.dart:2084, modal_add_summary.dart:157) → route through import_lab_values.
  • ADD: restore generate_lab_analytes_ts.py + CI drift; persist-verdict hook in both ingestion paths; a shared Python verdict evaluator.

2.6 1:1 proof​

Same analyte imported via PDF vs Junction vs mobile → identical value/unit; verdict identical on flow-web/basisweb/mobile/Atlas for ≥20 common analytes.

3. PROTOCOLS (assignment · schedule/instances · notes · recurrence)​

3.1 Canonical representation​

Pick users/{uid}.recurringEvents as canonical (mobile already reads only it; has full context). Today assign_protocol (functions_clinic.py:1015) quadruple-writes — recurringEvents (full) + clinic mirror (_habit_min, drops defaultSupplements/description) + user mirror + expected-* fan-out. #387: one serialize_protocol_habit() used by assign + update so all copies carry identical fields, written atomically.

3.2 Collapse THREE schedule generators to ONE​

Backend fan-out (functions_clinic.py:1325/2188) · mobile Rust engine (service_recurring_events.dart:1394) · web materializeProtocolActivities (PlanTab.tsx:2918). KEEP the backend fan-out all surfaces read; DELETE web materialize + retire mobile instance-generation for protocol habits (keep it for completion tracking only) (#395). Also unify assign(90d, raw type) vs update(14-180d, resolved type) inconsistency.

3.3 Recurrence + per-instance exceptions (#397)​

Rule (freq/byWeekDays/until) is interpreted consistently by all 3 generators (good) — but there is NO exception model, and update_protocol delete-all-regenerates, WIPING per-instance edits AND medication-completion marks (functions_clinic.py:2107-2116; completions at rp_activity_route.dart:133). ADD rule.exceptions[] (date-keyed override + completion) + edit scopes (this/this-and-future/all) + reconcile-not-delete + extend/end-early.

3.4 Notes (#390)​

Notes live in ≥4 places; UpdateHabitInput has no notes field (functions_clinic.py:1528) so the patch path can't edit notes; QuickActivityModal writes notes directly to the event doc (PlanTab.tsx:428) bypassing everything. ONE notes field on the canonical habit, plumbed via the serializer; route QuickActivityModal through update_protocol.

3.5 DELETE-LIST / 1:1 proof​

DELETE web materializedProtocolActivities, QuickActivityModal direct write, stale activities field. Proof: assign → edit → identical instance list + notes + presets on all 4 surfaces (harness protocol check). #388 (expected-* mirror-leak) ✅ shipped.

4. BENCHMARKS / VERDICTS (labs + the 15 wearable metrics)​

4.1 Canonical registry​

Global /benchmarks/{code} default tier seeded from lab_analytes.json (784) + the 16 wearable METRIC_BENCHMARKS (benchmark-defaults.ts:48-254); clinics fork via existing clinicsv2/{clinic}/benchmarks

  • clinic_users/{uid}/benchmarks (unlimited age/gender/scheme variants). Resolution: personal → clinic → global.

4.2 Evaluators are already aligned; the bug is units​

Web clinic-benchmark-evaluator.ts:141 and mobile clinic_benchmarks.dart:13 ClinicPolicyEvaluator are byte-identical. The bug: raw value compared to a benchmark in a different unit with no conversion (LabsTab.tsx:1944, analyzer_biomarkers.dart:402) → "HDL 3.9 mmol/L vs 60 mg/dL → out-of-range". Fix = a unit-normalize-before-compare wrapper (convert value → benchmark unit; guard ratio analytes which have no conversion). Atlas has no evaluator → #392 persist verdict at write so it reads the same answer.

4.3 DELETE-LIST / ADD-LIST​

  • DELETE: benchmark-defaults.ts LAB_BENCHMARKS (43) + LabsTab DEFAULT_LAB_BENCHMARKS + trend_type.dart hardcodes (after readers switch).
  • ADD: seed /benchmarks; unit-normalization middleware (web + mobile); persist-verdict-at-write; the shared Python verdict evaluator (used by #392 + Atlas); Mayo+CLIA range backfill (ref_ranges, common labs first, age/sex variants).

4.4 1:1 proof​

Same analyte+value → identical verdict on web/mobile/Atlas for ≥20 common analytes + the 15 metrics.

5. Build sequence (dependency-ordered, cross-worktree)​

PhaseDomainIssueWhere
0 Measureall#384 parity harness (extend to labs+protocols)backend tools/
1 Authoritative writewearables#381 server digest writer + shared resolver (keystone)backend
wearables#393 write-time precedence (app _digestAppend + manual)wt/hybrid + backend
labs#385 route mobile scan+manual through import_lab_values; #386 registry generator+CIwt/hybrid + backend
protocols#387 serializer; #390 notes; #389 datesbackend (+ wt/hybrid #397)
benchmarksseed /benchmarks; unit-normalize wrapperbackend + web + wt/hybrid
2 Converge readerswearables#394 web verbatim (delete recompute); #391 basisweb reads digestbasisflow-web + feat/basisweb
protocols#395 one generator (retire web materialize + mobile dup)backend + web + wt/hybrid
labs/bench#392 persist verdict at write (Atlas parity)backend
3 Proveallharness GREEN on every metric/lab/protocol across all surfaces—

Worktrees: backend + basisflow-web here (master); mobile (#393 Dart, #385, #397, mobile reads) → wt/hybrid; basisweb (#391) → feat/basisweb.

6. Definition of done​

Ask any surface and Atlas for the same patient/window and get byte-identical answers, provable by the harness:

  1. rhr 2026-03-15 → 49 everywhere, with _sources=Oura and _shadow={Apple:71} auditable.
  2. zone 2 minutes last week → identical 7-day sum on web/mobile/Atlas; basisweb renders it.
  3. a lab value+unit+verdict → identical on all four.
  4. a protocol's schedule + notes → identical instances on all four; per-instance edits survive protocol edits.