Canonical source: docs/claude/plan-metrics-data-contract.md. This page is generated by docs/scripts/sync-handbook.mjs. Edit the source file in the repo; changes appear here on the next build.
Plan & Metrics Tabs — Data Contract
This file documents exactly what Firestore data the basisflow-web Plan tab and Metrics tab read, including paths, field names, and the JS/TS types the read code assumes. It is the source of truth we will compare against how basishybrid writes the same data, so we can collapse divergent shapes into a single trusted source per field.
Evidence is cited as file:line against the basisflow-web repo (/Users/G/basis/hybrid/basisflow-web/). All paths assume the multi-tenant model — clinicId is the active clinic, clientId is the patient/user being viewed.
0. Glossary
| Term | Meaning |
|---|---|
| User path | Root-level users/{clientId}/... collections — written by basishybrid as the authoritative source |
| Clinic path | clinicsv2/{clinicId}/clinic_users/{clientId}/... — for health summaries, this is a scheduled mirror of the user path (see basis-functions/functions/src/functions_health_mirror.py); for events/protocols/prescriptions/etc. it is the primary source |
| localDate | Date-only string "YYYY-MM-DD" in the client's local timezone, never a Firestore Timestamp |
| extracted | Map written by basishybrid onto every healthSummaries doc — single-source-of-truth biomarker values keyed by canonical field names (see §5.2) |
| type / vt | Short metric-type code on a healthSummary doc (e.g. 'rhr', 'hrvs', 'sleep'). type is canonical; vt is a legacy alias also read in some places |
| safeOnSnapshot | Wrapper around Firestore onSnapshot that survives permission errors (@/lib/firebase) |
1. PlanTab — Surfaces & Reads
File: app/(main)/clients/[id]/components/PlanTab.tsx (~9500 lines)
1.1 Week view & Month view (calendar grid)
The calendar populates from a merged stream of:
- Events (scheduled/booked activities)
- Health summaries (Apple Health / Terra / Oura / manual entries)
- Protocols (defines recurring habits)
| # | Path | Listener | Query | Expected fields | Notes |
|---|---|---|---|---|---|
| A | clinicsv2/{clinicId}/clinic_users/{clientId}/events | safeOnSnapshot | where('start','>=', sinceTs), orderBy('start','desc'), limit(500) | start: Timestamp, end?: Timestamp, type: string, eventType?: string, name/title: string, status: string, completionStatus?: string, source?: string, protocolId?: string | PlanTab.tsx:1697; mapped by mapEventToActivity at 2320 |
| B1 | clinicsv2/{clinicId}/clinic_users/{clientId}/healthSummaries | safeOnSnapshot | where('end','>=', sinceTs), orderBy('end','desc'), limit(200) | see §5 | 1710–1717 |
| B2 | same path | safeOnSnapshot | where('start','>=', sinceTs), orderBy('start','desc'), limit(100) | 1730–1737 — duplicate fetch by alternate index | |
| B3 | same path | safeOnSnapshot | where('type','==','wor'), where('start','>=', …), orderBy('start','desc'), limit(500) | 1750–1758 | |
| B4 | same path | safeOnSnapshot | where('type','==','sleep'), where('start','>=', …), orderBy('start','desc'), limit(200) | 1772–1780 | |
| C1 | users/{clientId}/healthSummaries | safeOnSnapshot | where('end','>=', sinceTs), orderBy('end','desc'), limit(150) | 1794–1801 | |
| C2 | same path | safeOnSnapshot | where('type','==','wor'), …, limit(500) | 1862–1870 | |
| C3 | same path | safeOnSnapshot | where('type','==','sleep'), …, limit(200) | 1884–1892 | |
| D | users/{clientId}/events | getDocs | where('start','>=', …), where('start','<=', …), orderBy('start','asc') | docs whose ID begins with expected- are skipped (1847) | 1837–1844 |
| E | clinicsv2/{clinicId}/clinic_users/{clientId}/protocols | safeOnSnapshot | where('status','in', ['active','ended','stopped','completed','paused','cancelled']) | see §3 | 1906–1911 |
Cleanup signal #1 — B1/B2/B3/B4 query the same collection four ways. B1+B2 differ only by which timestamp field they index on; B3+B4 add type filters. Pick one shape (probably the type-less B1 plus client-side filtering, or strict per-type queries) and remove the rest.
Cleanup signal #2 — Both C-series (user path) and B-series (clinic path) load health summaries. Per
functions_health_mirror.py:1the clinic path is a mirror of the user path. PlanTab is reading both halves of a mirror, so duplicates are deduped client-side. The mirror exists for security-rule access; PlanTab could read only the user path when the clinician already has access, or only the clinic path and trust the mirror is current.
Mappers
mapEventToActivity (PlanTab.tsx:2320) and mapSummaryToActivity (PlanTab.tsx:2377) normalize both shapes into a single PlanActivity interface declared at PlanTab.tsx:218. Both call .toDate() on start/end defensively (x?.toDate?.() ?? new Date(x)).
1.2 Side panel / day-detail view
When a day is clicked, the side panel pulls HR samples and glucose samples to draw the vitals chart.
| Path | Listener | Query | Fields |
|---|---|---|---|
users/{clientId}/healthSummaries | getDocs | where('start','>=',dayStart), where('start','<=',dayEnd), orderBy('start','asc') | 2866–2871, 3041 |
clinicsv2/{clinicId}/clinic_users/{clientId}/healthSummaries | getDocs | same as above | 2873–2878, 3053 (conditional) |
HR sample extraction (PlanTab.tsx:2886–2996) — the code walks ~20 possible nested paths to find samples:
summary.heartRateData.detailed.hrSamples
summary.heart_rate_data.detailed.hr_samples
summary.hrSamples
deviceData.heartRateData.detailed.hrSamples
… and 16 more
Per sample, the timestamp may be on s.timestamp | s.time | s.t and the value may be on s.hr | s.hrFrequency | s.bpm | s.hr_bpm | s.heartRate | s.heart_rate. Range filter: 30 <= hr <= 250.
Glucose sample extraction (3004–3038) — paths payload.glucoseSamples, summary.glucoseSamples, payload.body.glucoseSamples. Per-sample value on s.value | s.glucose | s.glucoseValue | s.mg_dl | s.mgDl.
Cleanup signal #3 — Two-script samples should land in one canonical path. Currently basisflow-web reads up to 20 path variants because basishybrid (or its integrations) writes them differently per provider. The minimum viable cleanup:
summary.heartRateSamples: Array<{ t: Timestamp | string, hr: number }>andsummary.glucoseSamples: Array<{ t: Timestamp | string, value: number }>.
1.3 Event detail view (single activity)
Opening an activity tries up to six fallback reads in order (PlanTab.tsx:7012–7425):
users/{clientId}/healthSummaries/{activityId}—getDoc(7124)users/{clientId}/healthSummaries—getDocswithwhere('start',…)window,limit(50), client-side filter on type containing"workout"(7143)clinicsv2/{clinicId}/clinic_users/{clientId}/healthSummaries/{activityId}—getDoc(7165)- Day-scan with HR-sample overlap scoring across both user + clinic paths (7234–7289). Indexed query first; on missing-index error, retries without
orderBy(7244–7249). clinicsv2/{clinicId}/clinic_users/{clientId}/events/{activityId}—getDoc(7338)- For protocol-source activities:
events/{activityId}doc fetched once more (7023) to load coach edits likeworkoutPreset
1.4 Protocol tracking
| Surface | Reads via | Path | Notes |
|---|---|---|---|
| Protocol list & active habits | direct (1906–1911) | clinicsv2/{clinicId}/clinic_users/{clientId}/protocols | See §3 for Protocol shape. Both habits and activities arrays are read and merged-by-id (1918–1923) — backend writes both with identical content |
| Daily completion state | direct (1990–1995) | users/{clientId}/dailyDigest orderBy('localDate','desc'), limit(30) | Reads localDate and completions: { [eventId]: status } |
| Biomarker baseline (per binding) | calculateBaseline() from lib/biomarker-utils.ts | dailyDigest then healthSummaries fallback | see §4 |
| Biomarker current value | getCurrentValue() from same | dailyDigest then healthSummaries fallback | see §4 |
| Biomarker history (sparkline) | getHistoricalValues() from same | both user + clinic healthSummaries, 90-day window | see §4 |
| Adherence % | calculateAdherence(completed, expected) | pure function, no Firestore |
1.5 Other reads on PlanTab
| Surface | Path | Listener | Fields |
|---|---|---|---|
| Fullscript "connected" badge | clinicsv2/{clinicId}/config/fullscript | safeOnSnapshot (2016) | connected: boolean |
| DoseSpot "connected" badge | clinicsv2/{clinicId}/config/dosespot | safeOnSnapshot (2039) | connected: boolean |
| Current user DoseSpot enabled | clinicsv2/{clinicId}/clinicians/{uid} | safeOnSnapshot (2061) | dosespotEnabled: boolean |
| Prescriptions list | clinicsv2/{clinicId}/clinic_users/{clientId}/prescriptions | safeOnSnapshot (2142) | orderBy('writtenDate','desc'); full doc spread |
| Supplement plans | clinicsv2/{clinicId}/clinic_users/{clientId}/supplementPlans | safeOnSnapshot (2235) | orderBy('createdAt','desc'); reads status, full doc |
| Staff name resolution | clinicsv2/{clinicId}/clinicians | getDocs (2162) | firstName, lastName, displayName, name, email keyed by uid |
| Client demographics (clinic) | clinicsv2/{clinicId}/clinic_users/{clientId} | safeOnSnapshot (2186) | dateOfBirth | birthday (string "YYYY-MM-DD"), email, gender: string | 1 | 2 (numeric coercion at 2195–2196) |
| Client demographics (user) | users/{clientId} | safeOnSnapshot (2201) | same |
Cleanup signal #4 — gender as a number. Web code maps
gender === 1 → 'Male',gender === 2 → 'Female'(CLAUDE.md acknowledges this). But the Dart enumBasisGenderatbasiscore/lib/src/models/user/constants/gender.darthas the orderingfemale (0), male (1), other (2), unknown (3). So2should be "other", not "female" — unless basishybrid is writing a different mapping than the enum's.index. This is a likely bug worth verifying when you share the writer code. Single-source fix: writegenderas the lowercase string"male" | "female" | "other"and migrate existing docs.
1.6 AssignProtocolModal reads (components/protocol/AssignProtocolModal.tsx)
Opened from PlanTab. Loads catalog data and client sleep:
| Path | Listener | Fields |
|---|---|---|
clinicsv2/{clinicId}/exercises | getDocs (245) | name, force, level, mechanic, equipment, primaryMuscles, secondaryMuscles, instructions, category, images |
clinicsv2/{clinicId}/supplements | getDocs (267) | name, category, typicalDosage, dosageUnit, typicalDoseMin, typicalDoseMax, benefits, timing, forms |
clinicsv2/{clinicId}/medications | getDocs (289) | name, brandNames, category, type, typicalDosage, dosageUnit, schedule, prescriptionRequired, deaSchedule, notes |
clinicsv2/{clinicId}/clinic_users/{clientId} and users/{clientId} | getDoc (326, 344) | typicalWake (string "HH:mm" or number-of-minutes — coerced at 331–333), sleepMetrics.typicalWake |
clinicsv2/{clinicId}/clinic_users/{clientId}/healthSummaries | getDocs (364) | Fallback wake-time estimation — circular mean of last N sleep end/wakeTime timestamps (374–409) |
clinicsv2/{clinicId}/protocols (clinic-level template library, not per-client) | getDocs (430) | habits | activities array (same shape as §3); used as templates |
clinicsv2/{clinicId}/mealTemplates (fallback to global mealTemplates) | getDocs | meal preset data |
Cleanup signal #5 —
typicalWakepolymorphism. Field is read as both"HH:mm"string and as a number-of-minutes from midnight. Pick one.
2. MetricsTab — Surfaces & Reads
File: app/(main)/clients/[id]/components/MetricsTab.tsx (~3400 lines)
2.1 Main metrics grid (tiles)
Driven entirely by two streams merged client-side:
| Path | Listener | Query | Role |
|---|---|---|---|
users/{clientId}/dailyDigest | safeOnSnapshot | orderBy('localDate','desc'), limit(365) (871–880) | Primary — aggregated daily values, basishybrid writes one doc per localDate |
users/{clientId}/healthSummaries | safeOnSnapshot | where('end','>=', oneYearAgoTs), orderBy('end','desc') (886–894) | Fallback — for metrics not in the digest, or raw HR samples / zone synthesis |
Per (date, type), digest wins; matching healthSummary docs are filtered out (MetricsTab.tsx:860). Merge policies (1278–1308): mhr keeps the highest, rhr keeps the lowest non-zero, ste/exe keep highest, hrzone* SUM, default = latest by rank (updatedAt > ingestedAt > end).
MetricsTab reads ONLY the user path for healthSummaries — it does not touch the clinic mirror. PlanTab reads both. This asymmetry is intentional but undocumented.
2.2 Detail drawer / chart
Selecting a tile opens a detail drawer with a recharts area chart. No additional Firestore reads — chart is built from the same in-memory MetricItem.series already computed for the tile (1695–1790). Series points are { t: Date, v: number }; date axis is rendered with t.toLocaleDateString('en-US', { month:'short', day:'numeric' }) (1700–1710).
2.3 Detail grids (zones, HR samples, sleep stages)
For the HR-zones tile, MetricsTab synthesizes per-day zone minutes from up to three sources (1071–1231):
hrDailyhealthSummary docs — fieldsz1..z5(1071–1122) plus optionallocalDate,tzOffsetMinutesdailyDigest— fieldszone1Minutes..zone5Minutes- Non-Apple workout docs — computed from raw
summary.heartRateData.detailed.hrSamplesby counting samples per zone, divided byhrFrequency/60 (1124–1231). Apple workouts are skipped because Apple Health writes pre-computed zones.
Sleep stages: prefer (end - start)/60000 to compute total sleep over doc.value (1401–1406); fields start/end expected as Timestamps via .toDate?.().
2.4 Benchmarks
Three layers, evaluated by getApplicableBenchmark → evaluateBenchmark:
| Layer | Source | Path / module |
|---|---|---|
| Default (hardcoded) | METRIC_BENCHMARKS in lib/benchmark-defaults.ts | bundled; ~20 metrics with { scheme, min, inMin, optMin, optMax, inMax, max, variants[] } and gender/age variants |
| Clinic override | clinicsv2/{clinicId}/benchmarks/{metricCode} | MetricsTab.tsx:638–654 — safeOnSnapshot on the whole collection; doc id is metric code |
| Personalized (per-user) | BenchmarkSources.personalized accepted by evaluateWithSources in lib/clinic-benchmark-evaluator.ts | Not read by MetricsTab today — interface exists but MetricsTab never wires it in. Personalized benchmarks (variance-based for HRV etc.) are a planned-but-unused codepath |
BenchmarkPolicy shape (lib/clinic-benchmark-evaluator.ts:29–55):
{ scheme: 'a'|'b'|'c'|'d',
variants?: BenchmarkVariant[], // { gender, ageFrom, ageTo, values: { min, inMin, optMin, optMax, inMax, max } }
min?, inMin?, optMin?, optMax?, inMax?, max?: number,
unit?, displayName?, notes?: string,
source?: 'default'|'clinic'|'personalized'|'ai-copilot',
varianceBased?: boolean, // only used by personalized
stdDevMultiplier?: number,
baseline?: { value, method, sampleSize, startDate, endDate, stdDev? }
}
Coercion: toNumber() (clinic-benchmark-evaluator.ts:65–72) is applied to every numeric policy field — i.e. the code is defending against Firestore returning strings instead of numbers here. That's a smell on the writer side.
2.5 Other reads on MetricsTab
| Path | Listener | Fields |
|---|---|---|
clinicsv2/{clinicId}/customMetrics | safeOnSnapshot (660–676) | name: string, unit: string — populates the "Add value" modal |
clinicsv2/{clinicId}/preferences/metricsFavorites | safeOnSnapshot (682–695) + getDoc on toggle (1995) | codes: string[], order: string[] |
clinicsv2/{clinicId}/preferences/units | safeOnSnapshot (701–728) | weightUnit: 'kg'|'lb', distanceUnit: 'km'|'mi', temperatureUnit: 'c'|'f', volumeUnit: 'l'|'oz', glucoseUnit: 'mmol'|'mgdl', timeFormat: '12h'|'24h' — strict-equality checks with imperial fallback |
3. Shape contracts (per collection)
3.1 users/{clientId}/dailyDigest/{docId}
Doc ID convention: typically localDate (one doc per day). Single doc per day per user.
Required:
| Field | Type | Notes |
|---|---|---|
localDate | string | "YYYY-MM-DD" in client TZ. Used as join key. Code regex-checks /^\d{4}-\d{2}-\d{2}$/ in some paths (MetricsTab.tsx:1079). |
Optional metric fields (from EXTRACTED_KEY_MAP in lib/biomarker-utils.ts:167–231):
| Field | Type | Domain |
|---|---|---|
rhr | number | bpm — resting HR |
hr | number | bpm — daily avg HR |
hrvSdnn | number | ms |
hrvRmssd | number | ms |
maxHr | number | bpm |
walkingHr | number | bpm |
vo2Max | number | ml/kg/min |
steps | number | count |
activeCalories | number | kcal |
restingCalories | number | kcal |
calories | number | kcal total |
exerciseMinutes | number | minutes |
floorsClimbed | number | count |
distanceMeters | number | meters |
mindfulnessMinutes | number | minutes |
sleepMinutes | number | minutes (total) |
deepSleepMinutes | number | minutes |
remSleepMinutes | number | minutes |
lightSleepMinutes | number | minutes |
sleepEfficiency | number | percent (0–100 OR 0–1 — see cleanup signal #6) |
glucose | number | mg/dL |
glucoseTIR | number | percent — time in range |
glucoseVariability | number | CV % |
bmi | number | kg/m² |
weight | number | kg (canonical) |
bodyFatPct | number | percent |
spo2 | number | percent |
proteinG | number | grams |
carbsG | number | grams |
fatG | number | grams |
waterMl | number | mL |
fastingMinutes | number | minutes |
height | number | cm |
waistCm | number | cm |
bpSystolic | number | mmHg |
bpDiastolic | number | mmHg |
activityScore | number | Oura-style 0–100 |
recoveryScore | number | Oura-style 0–100 |
sleepScore | number | Oura-style 0–100 |
zone1Minutes … zone5Minutes | number | minutes in each HR zone |
avgHr | number | bpm |
tzOffsetMinutes | number | client TZ offset for reconstruction |
completions | { [eventId: string]: status } | per-day protocol activity completion map (read by PlanTab at 2001) |
Reader rule: code uses typeof val === 'number' && val > 0 (MetricsTab.tsx:826). Zero is treated as missing. Negative values are silently dropped.
Cleanup signal #6 —
sleepEfficiencyunit ambiguity. MetricsTab.tsx:1448–1450 multiplies value by 100 ifvalue < 1andunit === '%'. Pick one form (0–100) and stop the coercion.
3.2 users/{clientId}/healthSummaries/{docId}
Authoritative source written by basishybrid. Mirrored to clinicsv2/{clinicId}/clinic_users/{clientId}/healthSummaries by functions_health_mirror.py.
Required:
| Field | Type | Notes |
|---|---|---|
type | string | Short code — see §3.3. Canonical. |
start | Timestamp | Reader calls .toDate?.(); falls back to new Date(x) |
Common optional:
| Field | Type | Notes |
|---|---|---|
end | Timestamp | for ranged events (sleep, workouts) |
vt | string | legacy alias for type; some readers prefer vt (PlanTab.tsx:1807 lowercases both) |
localDate | string | "YYYY-MM-DD" — preferred over deriving from start (avoids TZ math) |
tzOffsetMinutes | number | TZ offset for start reconstruction when localDate missing |
value | number | primary numeric value for single-valued types (glucose reading, body weight, etc.) |
aggSum / aggCount / aggMax | number | aggregations (sleep minutes typically in aggSum) |
source | string | provider id — "apple", "com.apple…", "oura", "terra", "manual", etc. PlanTab checks for 'calendar'/'calendarDevice' (2339–2340); MetricsTab checks for Apple via isAppleSource() (120–124) |
updatedAt, ingestedAt | Timestamp | used for merge-rank ordering |
extracted | { [canonicalKey]: number } | Preferred biomarker map — see §3.5 |
summary | object | provider-specific nested payload — see §3.4 |
payload | object | raw vendor blob; some readers fall back to payload.body.glucoseSamples etc. |
3.3 healthSummary type values that the web actually reads
From getExactQueryTypes (lib/biomarker-utils.ts:101–151) plus PlanTab/MetricsTab literal comparisons:
type value | Means | Read by |
|---|---|---|
rhr | resting HR | biomarker-utils, MetricsTab |
hrvs | HRV SDNN | biomarker-utils |
hrvr | HRV RMSSD | biomarker-utils |
hrDaily | per-day HR aggregate + zone minutes | biomarker-utils, MetricsTab (1242) |
mhr | max HR | biomarker-utils, MetricsTab |
whr | walking HR | biomarker-utils |
vo2 | VO₂ max | biomarker-utils |
ste | steps | biomarker-utils, MetricsTab |
aca | active calories | biomarker-utils |
bca | resting/basal calories | biomarker-utils |
exe | exercise minutes | biomarker-utils, MetricsTab |
fli | floors climbed | biomarker-utils |
dru | distance (meters) | biomarker-utils |
min | mindfulness minutes | biomarker-utils |
sleep | sleep session | biomarker-utils, PlanTab (1774, 1888), MetricsTab |
glu | glucose reading | biomarker-utils, PlanTab (3032) |
bmi | BMI | biomarker-utils |
wei | weight | biomarker-utils |
bf | body fat % | biomarker-utils |
oxy | SpO₂ | biomarker-utils |
cal, pro, car, fat, wat | calories, protein, carbs, fat, water | biomarker-utils |
wor | workout | PlanTab (1752, 1864), MetricsTab |
body | generic body composition doc | biomarker-utils (line 252) |
Inconsistencies caught by readers:
- PlanTab.tsx:1809–1810 accepts
vt === 'wo' || vt === 'sl'andtype === 'wor' || type === 'sle'. So writers are or were producing both'wo'and'wor'for workouts, and'sl'/'sle'/'sleep'for sleep. - PlanTab.tsx:3032 accepts
'glu'or'glucose'.
Cleanup signal #7 — canonicalize
typestrings. Pick one form per concept (recommend the 3-letter short codes already used bygetExactQueryTypes:wor,sleep— notesleepis the exception that's already long-form —glu, etc.). The reader's fallback paths are evidence of past inconsistency.
3.4 healthSummary summary nested fields
Heterogeneous; depends on type and provider:
| For type | Path readers walk | Expected leaf |
|---|---|---|
hrvs / hrvr | summary.hrvSdnn | summary.hrv_sdnn | hrSummary.avgHrvSdnn | hrSummary.avgHrvRmssd (biomarker-utils.ts:269–273) | number |
rhr | summary.restingHeartRate | summary.rhr | hrSummary.restingHrFrequency (278–281) | number |
sleep | sleepDurations.asleep.durationAsleotStat.s (seconds — line 297) | summary.asleepMinutes | doc.value in [30,1440] (303) | number |
glu | summary.avgGlucose | summary.glucose | number |
vo2 | summary.vo2Max | bodyData.vo2Max | number |
hrDaily | summary.zone1Minutes..zone5Minutes | number |
wor (HR zones) | summary.heartRateData.detailed.hrSamples[].{hrFrequency | hr | bpm} (1143–1156) | array, each sample has number HR + Timestamp/ISO/number timestamp | time | t |
wor (steps/calories) | summary.steps, summary.activeCalories, summary.maxHr | number |
| Any type | summary.workoutActivityType, summary.workoutType (PlanTab.tsx:2392–2394) | string |
3.5 healthSummary extracted map — the contract we should align on
lib/biomarker-utils.ts:233–241 declares preference: if extracted exists, use it. Otherwise fall back to nested paths. That makes extracted the de-facto single source of truth — except there is no schema artifact (no Pydantic model, no Dart class) defining its keys anywhere in basis-functions/functions/src/ or basiscore/lib/src/models/. The keys are inferred only from the consumer's EXTRACTED_KEY_MAP.
Recommended canonical keys (the values column in EXTRACTED_KEY_MAP):
hrvSdnn, hrvRmssd, rhr, hr, maxHr, walkingHr, vo2Max, steps, activeCalories, restingCalories, calories, exerciseMinutes, floorsClimbed, distanceMeters, mindfulnessMinutes, sleepMinutes, deepSleepMinutes, remSleepMinutes, lightSleepMinutes, sleepEfficiency, glucose, glucoseTIR, glucoseVariability, bmi, weight, bodyFatPct, spo2, proteinG, carbsG, fatG, waterMl, fastingMinutes, height, waistCm, bpSystolic, bpDiastolic, activityScore, recoveryScore, sleepScore, zone1Minutes…zone5Minutes.
Cleanup signal #8 — formalize
extracted. Define this as a Pydantic model (ExtractedBiomarkers) inbasis-functions/functions/src/, generate a matching Dart class inbasiscore, and have basishybrid write it. Once that ships, retire all thesummary.*nested-path fallbacks inbiomarker-utils.ts:244–352.
3.6 clinicsv2/{clinicId}/clinic_users/{clientId}/events/{eventId}
PlanTab calendar source. Mapped through mapEventToActivity (PlanTab.tsx:2320).
| Field | Type | Notes |
|---|---|---|
start | Timestamp | required |
end | Timestamp | optional |
type | string | required; e.g. 'workout', 'supplement', 'meal', 'meeting', 'booking' |
eventType | string | alt field — read alongside type |
name | title | string | required (one or the other) |
status | string | 'completed'/'done'/'missed'/'skipped'/'not_applicable' (PlanTab.tsx:2346–2357) |
completionStatus | string | alt to status |
source | string | 'calendar'/'calendarDevice' recognized specially |
protocolId | string | links event back to a protocol |
| (full doc) | spread into .payload | reader holds the entire doc for later inspection |
Cleanup signal #9 —
statusvscompletionStatus. Same concern as type/vt. Pick one and drop the other.
3.7 clinicsv2/{clinicId}/clinic_users/{clientId}/protocols/{protocolId}
Protocol shape per lib/protocol-data.ts:208–258, but reader (PlanTab.tsx:1918–1957) is much looser:
| Field | Reader expects | Notes |
|---|---|---|
title | name | string | either is accepted |
description | string | |
status | string | filter is in ['active','ended','stopped','completed','paused','cancelled'] |
startDate | scheduledFrom | assignedAt | Timestamp or ISO string | parsed via parseDate() (1935–1942) — val.toDate?.() then new Date(val) |
endDate | scheduledUntil | same | |
progress | number | optional |
habits | Activity[] | see below — same data as activities |
activities | Activity[] | merged with habits by id (1918–1923) |
biomarkers | array | legacy |
biomarkerBindings | BiomarkerBinding[] | preferred shape — { code, displayName, category, unit, baselineValue?, baselineDate?, baselineSource?, targetDirection?, targetValue?, expectedOutcome? } per protocol-data.ts:50–65 |
durationWeeks | number | preferred |
duration | string | legacy ("6 weeks") |
clinicalEvidence, evidenceGrade, expectedOutcomes | optional |
Each entry of habits / activities (CustomProtocolActivity, protocol-data.ts:123–206) is another shape rife with legacy aliases: duration: number (legacy) + durationMin?: number (preferred); frequency: string | ActivityFrequency; time: string + optional timeWindow: ActivityTimeWindow; optional embedded workoutPreset, supplementPreset, medicationPreset.
Cleanup signal #10 —
habitsvsactivities. Backend writes both arrays with identical content; client merges by id. Pick one, write it once, delete the other.
3.8 Clinic-level catalogs (read by AssignProtocolModal)
| Path | Fields (as read by web) |
|---|---|
clinicsv2/{clinicId}/exercises/{id} | name, force, level, mechanic, equipment, primaryMuscles, secondaryMuscles, instructions, category, images |
clinicsv2/{clinicId}/supplements/{id} | name, category, typicalDosage, dosageUnit, typicalDoseMin, typicalDoseMax, benefits, timing, forms |
clinicsv2/{clinicId}/medications/{id} | name, brandNames, category, type, typicalDosage, dosageUnit, schedule, prescriptionRequired, deaSchedule, notes |
clinicsv2/{clinicId}/protocols/{id} (clinic template library) | same as §3.7 |
clinicsv2/{clinicId}/mealTemplates/{id} | meal preset blob |
clinicsv2/{clinicId}/benchmarks/{metricCode} | BenchmarkPolicy (§2.4) |
clinicsv2/{clinicId}/customMetrics/{id} | name, unit |
clinicsv2/{clinicId}/preferences/metricsFavorites | codes: string[], order: string[] |
clinicsv2/{clinicId}/preferences/units | unit pref enums (§2.5) |
clinicsv2/{clinicId}/config/fullscript and …/dosespot | connected: boolean |
3.9 clinicsv2/{clinicId}/clinic_users/{clientId} and users/{clientId} (client demographics)
| Field | Type as read | Notes |
|---|---|---|
dateOfBirth | birthday | string | "YYYY-MM-DD" per CLAUDE.md rule. Never a Timestamp. |
email | string | |
gender | string | 1 | 2 | See cleanup signal #4 |
typicalWake | string "HH:mm" | number (minutes) | See cleanup signal #5 |
sleepMetrics.typicalWake | same | nested alt |
4. Cross-cutting reader expectations
4.1 Timestamp handling
Every Firestore Timestamp read goes through one of these forms:
x?.toDate?.()— defensive (most readers)x?.toDate?.() ?? new Date(x)— handles already-deserialized DatesTimestamp.fromDate(jsDate)— for writing query bounds
If basishybrid writes start as anything but a Firestore Timestamp, the .toDate?.() returns undefined and the doc silently disappears from filters.
4.2 Numeric handling
typeof v === 'number' && v > 0— zero is treated as missinglib/clinic-benchmark-evaluator.ts:65–72runstoNumber()on every policy value because the writer sometimes produces strings — fix on the writer sideisFinite(v)guard not always present — NaN can leak throughNumber(x)calls
4.3 Date-only strings
- Demographics:
dateOfBirth/birthdayalways"YYYY-MM-DD"(CLAUDE.md hard rule) - Daily digest:
localDatealways"YYYY-MM-DD" - HealthSummary
localDateoptional but preferred
4.4 The two-path mirror
| Collection | User path | Clinic path | Authoritative |
|---|---|---|---|
healthSummaries | yes | yes (mirror via functions_health_mirror.py) | User path |
dailyDigest | yes | no | User path |
events | yes (limited) | yes | Clinic path |
protocols | no | yes | Clinic path |
prescriptions, supplementPlans | no | yes | Clinic path |
| Client demographics | yes | yes | Either — written by different surfaces (mobile vs admin) |
Cleanup decision needed: should clinicians read the mirror or read the user path directly with stronger rules?
5. Unified action plan
Re-ranked 2026-05-18 after folding in the writer-side audit (§6–§8). Each issue now states the writer (hybrid), reader (basisflow-web), and backend actions required, plus its severity:
- BUG — current behaviour is incorrect for real users
- COST — wasted Firestore writes / RAG embeddings / function invocations
- CORRECT — data-correctness or race-condition risk
- DEBT — schema clutter, no immediate user impact
Order is impact-first: cost-of-doing-nothing × ease-of-fix.
Tier 1 — Ship this week
1. RAG embedding trigger on every healthSummary write [COST, top leverage]
- Evidence: §6.6.
functions_rag.py:674-686fires on everyusers/{uid}/healthSummaries/{summaryId}write — one Vertex AItext-embedding-004call (768d) per write, doubled by the mirror atfunctions_health_mirror.py. Every redundant write (idempotent backfill re-sets, per-sample_digestAppendfan-out, the hrb pipeline) pays this multiplier ~4–5x. - Backend: gate the trigger to high-value types only (
wor,lab, dailysleep); add a 30s per-user batch window; short-circuit when the new doc bytes equal the prior doc bytes. - Hybrid: none directly, but every other write-side fix in this list compounds here.
- Web: none.
2. Gender numeric mapping is a real bug [BUG]
- Evidence: §7.3. basishybrid writes
BasisGender.indexperbasiscore/lib/src/models/user/constants/gender.dart— order isfemale(0), male(1), other(2), unknown(3). basisflow-web atPlanTab.tsx:2195-2196and2212-2215maps1 → 'Male'(accidentally correct) and2 → 'Female'(wrong — should be 'other'). Everyotheruser has been mis-rendered as Female since this code shipped. Affects benchmark variant selection (clinic-benchmark-evaluator.ts:97-128) —other-gender users getfemalevariants applied to their ranges. - Hybrid: stop writing numeric
gender. Write the lowercase string"male" | "female" | "other" | "unknown"(matches the'm'/'f'/'o'/'u'FFI letters already in the enum). - Web: delete the
g === 1 / g === 2branches; treatgenderas a string only. - Backend: one-shot migration over
users/*andclinicsv2/*/clinic_users/*mapping0→female, 1→male, 2→other, 3→unknown. Run it before deploying the writer change so old docs don't disappear from gender-aware variants.
3. Delete the hrb (hrBatch) pipeline [COST, zero risk] — ✅ SHIPPED 2026-05-18
- Evidence: §7.1. Writer at lines 6315-6382, call site at 1177. Zero readers across basisflow-web, basisweb, basis-functions, and basishybrid itself. Fires every 10s during HR streaming → hundreds of writes per active-HR hour, each one paying the RAG-trigger tax.
- What it does NOT touch:
processTerraWebhook(functions_terra.py:859). That function is upstream ofhrb— Terra →hook→processTerraWebhook→ writes healthSummaries → mobile syncs them → mobile re-emitshrb. Killinghrbwon't reduceprocessTerraWebhookinvocations or its per-invocation CPU cost (which is driven by Terra-side webhook volume, especially initial-connect backfills, and themax_concurrent_dispatches=4cap at functions_terra.py:853 that saturates fast). - What it DOES amplify: for every active HR-streaming hour per user, ~360 invocations each of
index_user_health_summary(functions_rag.py:673 — one Vertex AI 768d embedding per write) andmirror_user_health_summary_to_clinic(functions_health_mirror.py:371). The mirror's clinic-side doc then fireshealth_summary_workout_mirror_handler(functions_notes.py:206) which early-exits but still costs an invocation. - Hybrid: ✅ Deleted in commit on
feat/extracted-health-data. Removed_hrBuffer,_hrFlushTimer,_bufferHrForHourlyBatch,_flushHrBatches,_writeHrBatch, and the call site inside theheartRatecase ofonSummaryCreated.BasisHealthType.hrBatchenum entry kept (still referenced bymodal_add_summary.dart:888UI case + thebasis_health_type.dartdefinition itself; no Firestore docs of this type will be produced going forward).flutter analyzeclean. - Web: PlanTab.tsx:1345
'hrb'exclude-list entry is a passive filter (returns "this type belongs in Metrics not Plan"). Now matches nothing — keep as defense-in-depth or delete in a cosmetic pass later. - Backend: optional cleanup migration to remove orphan docs in
users/{uid}/healthSummariesand the clinic mirror (type == 'hrb').
4. Strip summary.hypnogramSamples from sleep writes [COST, zero risk]
- Evidence: §7.2. 8K–32K entries per sleep doc. Fetched at
PlanTab.tsx:7730but the sleep UI at 8486-8522 only renderssummary.sleepDurationsDatatotals. Pattern matches the GPS strip at writer:2885 (already shipped on the current branch). - Hybrid: in
_upsertAppleSleepSession(~3089-3176), striphypnogramSamplesbefore the finalset(). KeepsleepDurationsDataand per-stage totals. - Web: none.
5. Remove duplicate healthSummaries queries on PlanTab [COST, web-only]
- Evidence: §1.1, queries B1 + B2. Same collection, same window, one indexed on
start, one onend. Both produce overlapping result sets; client dedupes. - Web: keep B1 (
end >= since, used for time-bounded scrolls), drop B2. C-series user-path remains because of the mirror policy in #6. - Hybrid / backend: none.
6. Decide one side of the user↔clinic mirror [COST / DEBT]
- Evidence: §4.4 + §7.3. basishybrid writes only to
users/{uid}/healthSummaries;functions_health_mirror.pymirrors to the clinic path. PlanTab reads BOTH halves (B-series clinic + C-series user); MetricsTab reads only the user path. Every mirrored doc is a second Firestore write event + a second RAG embedding call. - Backend: keep the mirror only for collections clinicians need security-rules-isolated access to — if rules allow direct user-path reads with role check, kill the mirror entirely. If we keep it, exempt the mirror from the RAG trigger registration (the embedding has already been generated on the user-path write).
- Web: PlanTab should mirror MetricsTab and read only the user path. Drop the B-series listeners (1710-1780) once the rules update lands.
Tier 2 — Multi-day cleanups
7. Add extracted map to per-day aggregator docs [DEBT, retires 100 lines of reader fallback]
- Evidence: §3.5 + §7.3.
_buildExtractedMapruns only inside_normalize(writer:2899) for per-sample docs. The per-day aggregators (_upsertDailyLatestDocat 4541,_upsertDailyHrSummaryAppleat 2500,_upsertDailyHrSummaryTerraat 2669,_upsertDailyBloodPressureDocat 4599,_upsertDailyBodyDocat 4626) do not includeextracted. Web'sbiomarker-utils.ts:244-352exists almost entirely to walk nestedsummary.*paths because of this gap. - Hybrid: add
extracted: _buildExtractedMap(...)to each per-day aggregator write. Use the canonical keys from §3.5. - Web: once shipped and backfilled, retire the nested fallback cascade in
biomarker-utils.ts(lines 244-352) — preserveextractedlookup only. - Backend (optional): one-shot backfill to populate
extractedon historic per-day docs.
8. Three read-modify-write aggregators race [CORRECT]
- Evidence: §6.4.
_updateDailySteps(writer:5245),_updateDailySleep(5308),_upsertDailyMean(6255) all read existing doc → increment → write back withoutrunTransaction. Concurrent inflows (Apple Watch sync + Terra webhook + on-launch backfill) interleave; the second writer overwrites the first's aggregate. - Hybrid: wrap each in
FirebaseFirestore.instance.runTransaction((tx) => …). OR, conditional on #9, delete the per-day healthSummaries doc entirely and rely ondailyDigest(which already accumulates the same fields via_digestAppend). - Web: none.
9. Per-day healthSummary docs duplicate dailyDigest fields [COST / DEBT]
- Evidence: §7.3. The same writer path produces both:
_upsertDailyLatestDocwrites a per-day healthSummary doc formhr/rhr/hrvrAND calls_digestAppendto write the same field on dailyDigest. Two docs, two RAG triggers, identical content. - Hybrid: for any field that already lives on
dailyDigest, skip the per-day healthSummaries doc. If readers need provenance (per-source/per-device split), keep the per-day doc and gate the dailyDigest write instead — pick one. - Web: confirm no MetricsTab read paths require the per-day doc for fields already in the digest (currently the merge policy at
MetricsTab.tsx:1278-1308accepts either).
10. _digestAppend fires per sample [COST]
- Evidence: §6.2. Every per-sample writer (
_updateDailyMaxHeartRate,_upsertDailyMean,_updateDailySteps, …) calls_digestAppendimmediately after writing its own doc. A 200-sample HealthKit sync produces ~200 merge-writes to the same daily digest doc — each one a separate RAG-trigger-firing event. - Hybrid: add a per-(uid, localDate, field) debounce / flush window (e.g. 30s) before the digest write. Aggregate samples in-memory, write once per window.
- Backend: dailyDigest does not currently fire the RAG trigger; if that changes, this becomes Tier 1.
11. Unconditional re-issue of identical writes on backfill [COST]
- Evidence: §6.3.
backfillWorkouts(writer:3799) re-firesonSummaryCreated(items)for every workout in the last 14 days on every app launch → the workout-header write at 3977 re-issuesset(header, merge: true)even when bytes are identical. Each re-issue is a Firestore write event + RAG embedding call. - Hybrid: compare new doc bytes to last-known-written bytes (cache locally per id) and short-circuit before
set()when equal. Same applies to_backfillMhrLastDays(5134) and_backfillRmssdLastDays(5196).
12. Workout HR enrichment 30s retry [COST]
- Evidence: §6.1.
_enrichWorkoutDocWithHeartRate(writer:3992) has an unconditional 30s retry at 3981-3983 that re-issues the sameset(). Idempotent in content, but counts as a separate write. - Hybrid: make the retry conditional — only retry when the first attempt fails or when the LocalDB sample count grew. Drop the unconditional path.
13. HR sample cap is too generous [COST]
- Evidence: §6.1. 10K-sample cap = 100-200KB per workout doc. A 60-min workout sampled every 7s is 514 samples — sufficient for time-in-zone bucketing.
- Hybrid: lower the cap at writer:4052 and 6424 from 10000 to 1000. Web zone-synthesis at
MetricsTab.tsx:1124-1231is proportional-by-count — denser sampling doesn't change the output materially.
Tier 3 — Schema canonicalization
14. Drop vt from healthSummary writes [DEBT]
- Evidence: §3.3, §7.3. basishybrid writes
vt='wo'alongsidetype='wor'(writer:2922-2939) for legacy compat; sleep writesvt='terraSleep'alongsidetype='sleep'(writer:3118). - Hybrid: remove the
vtfield assignment from_normalizeand_upsertAppleSleepSession. - Web: once shipped, remove
vtchecks atPlanTab.tsx:1807-1810and thevt || valueTyperead atMetricsTab.tsx:1248. - Backend: no migration required — readers fall back to
type.
15. Canonicalize type strings [DEBT]
- Evidence: §3.3. Writer is already canonical (writes
wor,sleep,glufromBasisHealthType.{name}.stringType). The reader-side fallbacks ('wo','sl','sle','glucose') cover only legacy docs from prior writers. - Web: keep fallbacks for now (cheap, defensive). Only strip after a backfill audit confirms no docs with legacy types remain.
16. Drop duplicate habits / activities arrays on protocols [DEBT]
- Evidence: §3.7. Backend writes both arrays (in
functions_protocols.py). basishybrid is not involved. - Backend: pick
activities(matchesprotocol-data.ts:208-258preferred shape), deletehabitswrites. - Web: stop merging the two at
PlanTab.tsx:1918-1923.
17. Drop completionStatus on events [DEBT]
- Evidence: §3.6. PlanTab accepts both
statusandcompletionStatus. - Backend / Hybrid: audit which surface writes
completionStatusand switch tostatusonly. - Web: remove the second check at
PlanTab.tsx:2347.
18. typicalWake polymorphism [DEBT]
- Evidence: §1.6. Read as both
"HH:mm"string and as number-of-minutes-from-midnight. - Hybrid / backend: pick
"HH:mm"string. Audit every writer ofclinic_users.typicalWakeandusers.typicalWake(mobile profile screen + backend onboarding). - Web: drop the numeric coercion in
AssignProtocolModal.tsx:331-333once writers are clean.
19. sleepEfficiency 0–100 vs 0–1 [DEBT]
- Evidence: §3.1.
MetricsTab.tsx:1448-1450multiplies by 100 if value < 1 and unit is%. - Hybrid: standardize on 0–100 across
_buildExtractedMapand_upsertAppleSleepSession. Audit Terra path which may pass 0–1 through. - Web: drop the coercion.
20. HR / glucose sample path proliferation [DEBT, migration-bound]
- Evidence: §1.2, §7.3. basishybrid is canonical (
summary.heartRateData.detailed.hrSamples). Legacy Terra and other-provider payloads landed in 19 other paths in old docs. - Web: keep fallbacks until a backfill normalizes old docs. After backfill, retire fallback cascade at
PlanTab.tsx:2896-2921. - Backend: one-shot migration to move samples into the canonical path on legacy docs.
21. BenchmarkPolicy numeric fields sometimes arrive as strings [DEBT]
- Evidence: §2.4.
clinic-benchmark-evaluator.ts:65-72runstoNumber()on every policy value. - Backend / Web (admin UI): audit benchmark write paths — the clinic benchmark editor in MetricsTab likely converts form inputs incorrectly. Write numbers as numbers.
- Web: keep
toNumber()defensively for one release after the fix.
22. val > 0 filter drops legitimate zeros [DEBT, small]
- Evidence: §4.2.
MetricsTab.tsx:826and several biomarker-utils sites usetypeof v === 'number' && v > 0— silently drops zero-valued metrics (zero drinks, zero floors climbed, zero workouts). - Web: change to
typeof v === 'number' && Number.isFinite(v) && v >= 0. Verify no downstream code divides by these values.
Tier 4 — Multi-week project
23. HR zone-minutes computed five ways [BUG, complex]
- Evidence: §8. Five surfaces, three computation methods (interval-walk vs proportional-by-count vs vendor-defined), two maxHR derivations, two zone-breakpoint schemes (
[0.5, 0.6, 0.7, 0.8, 0.9, 1.0]×maxHRvs[0.6, 0.7, 0.8, 0.9]×maxHR). Plus thehrDaily.z1..z5field is per-source, so an Apple+Oura user gets two docs per day that MetricsTab.tsx:1278-1308 sums, double-counting any overlapping minute. - Hybrid: pick
[0.5, 0.6, 0.7, 0.8, 0.9, 1.0]×maxHR(existing basishybrid choice), interval-walk computation inclassifyHRReadings. Write the result once. Stop suppressinghrzone*docs on Apple days (writer:6108-6110) — either always write them or always skip; the half-orphaned state is the problem. - Hybrid: for multi-source days, pick one writer (e.g. wrist-worn beats phone-derived) and skip the others rather than letting both write.
- Hybrid: for Terra vendor zones, re-bucket from raw HR samples; do not trust
hrZoneDatadirectly. - Web: delete
computeZonesFromHrSamples(MetricsTab.tsx:128-178); read zones only fromdailyDigest.zoneNMinutes. Delete the workout-fallback path at MetricsTab.tsx:1124-1231. - Backend: if any zone re-computation happens in cloud functions, align it on the same definition.
24. Backend _synthesize_sleep_gap_filler is a second writer [DEBT, must include in migration]
- Evidence: §6.5.
functions_health_mirror.py:331writes syntheticsleephealthSummaries when event clusters are incomplete. Any "single source of truth" migration must cover both basishybrid and this synthesizer. - Backend: confirm the synthesizer writes match the canonical schema chosen in #7 and #15. Add a
synthetic: trueflag so the web reader can distinguish it from device-derived data.
25. Formalize the extracted schema in code [DEBT]
- Evidence: §3.5. No Pydantic model, no Dart class — keys are inferred only from the web consumer's
EXTRACTED_KEY_MAP. - Backend: declare
ExtractedBiomarkersas a Pydantic model inbasis-functions/functions/src/. - Basiscore: generate the matching Dart class so basishybrid imports the canonical key set instead of duplicating field names in
_buildExtractedMap. - Web: import the generated TypeScript type if we add a tsgen step; otherwise mirror it manually.
Quick reference: who owns what
| Side | Tier 1 | Tier 2 | Tier 3 | Tier 4 |
|---|---|---|---|---|
| basis-functions (backend) | #1, #2 migration, #6 | (#9 if reader confirms) | #16, #21 | #24, #25 |
| basishybrid (writer) | #2, #3, #4 | #7, #8, #9, #10, #11, #12, #13 | #14, #17, #18, #19 | #23, #25 |
| basisflow-web (reader) | #2, #5, #6 | #7 retire fallbacks, #9 confirm | #14, #15, #17, #18, #19, #20, #22 | #23 |
Generated 2026-05-18 from a four-agent reader audit + three-agent writer audit. §6–§8 contributed by the hybrid-side agent. §5 unified action plan updated 2026-05-18 to integrate writer-side findings — every issue now has explicit owners and concrete file:line citations.
6. Writer-side audit (basishybrid)
Compiled 2026-05-18 from a three-agent audit (schema, reader, adversarial) of /Users/G/basis/hybrid/basishybrid/lib/services/service_health_firestore_writer.dart (~6500 lines). All citations are file:line in that writer unless otherwise noted. Type values are BasisHealthType.{name}.stringType taken from /Users/G/basis/hybrid/basiscore/lib/src/models/summary/constants/basis_health_type.dart.
6.1 Distinct doc shapes written to users/{uid}/healthSummaries
16 distinct type values, produced by the writer. Sorted alphabetically; "Reader" column references §3.3.
type | BasisHealthType | Primary write site | Trigger | Idempotency | Read by web? |
|---|---|---|---|---|---|
aca | activeEnergyBurned | _normalize (2833) → set (826/892/1768) | per-sample listener | deterministic ID (sha1 of uid+source+type+start+end+unit+value) | ✓ §3.3 |
ao | bloodOxygen | _normalize (2833) | per-sample listener | deterministic ID | ✓ as oxy / spo2 |
bca | basalEnergyBurned | _normalize (2833) | per-sample listener | deterministic ID | ✓ §3.3 |
bp | bloodPressure | _upsertDailyBloodPressureDoc (4599) | per-day Apple backfill | deterministic ID (uid+t:bp+date+source) | not directly — extracted as bpSystolic/bpDiastolic |
body | body (Terra) | _upsertDailyBodyDoc (4626) | per-day Apple backfill | deterministic ID | ✓ §3.3 |
glu | bloodGlucose | _normalize (2833) | per-sample listener | deterministic ID | ✓ §3.3 (PlanTab 3032 also accepts 'glucose') |
hrb | hrBatch | _writeHrBatch (6340), buffered via _bufferHrForHourlyBatch (6319) | n/a — no longer written | n/a | ✗ NEVER READ anywhere — was confirmed zero readers before delete |
hrDaily | (synthetic — no enum) | _upsertDailyHrSummaryApple (2500) + _upsertDailyHrSummaryTerra (2669) | per-day cardio aggregator | deterministic ID (uid+t:hrDaily+date+source) | ✓ MetricsTab 1242, 947 (zone synthesis) |
hrvr | heartRateVariabilityRMSSD | per-day aggregator (4437) + per-sample (2833) | per-sample listener + daily | deterministic ID | ✓ §3.3 |
hrvs | heartRateVariabilitySDNN | per-day aggregator (4437) + per-sample (2833) | per-sample listener + daily | deterministic ID + read-modify-write on daily | ✓ §3.3 |
lab | labs | labsCol.add() (2704) — also mirrors to clinic path at 2709 | per-lab result | .add() auto-id (no collision) | (read elsewhere — labs UI, not in this doc) |
mhr | maxHeartRate | _upsertDailyLatestDoc (4541) at line 4262 | per-day Apple backfill | deterministic ID, keep-max | ✓ §3.3 |
rhr | restingHeartRate | _upsertDailyLatestDoc (4228) | per-day Apple backfill + per-sample (2833) | deterministic ID, keep-latest | ✓ §3.3 |
sleep | sleep | _upsertAppleSleepSession (3089-3176) — header (3139) + summary (3176) | per-Apple-sleep-session | deterministic ID (uid+t:sleep+source+start+end) | ✓ §3.3 (PlanTab 1774/1888, MetricsTab) |
ste | steps | _normalize (2833) + _updateDailySteps (5245) | per-sample listener | deterministic ID + read-modify-write daily | ✓ §3.3 |
wei | weight | _normalize (2833) + _upsertDailyLatestDoc (4330) | per-sample + per-day | deterministic ID | ✓ §3.3 |
wor | workout | _normalize (2833) with legacy compat (2922-2939) + _upsertWorkoutHeaderFromEvent (3977) | per-sample listener / habit event | deterministic ID, header re-written every backfill | ✓ §3.3 |
Some types in §3.3 (e.g.
whr,vo2,fli,dru,min,cal,pro,car,fat,wat,bmi,bf) are produced via the same_normalizepath (2833) but with their respectiveBasisHealthType.{name}.stringType. They are not enumerated individually here — they share the per-sample shape from §3.2.
Workout HR enrichment (separate from the 'wor' row): _enrichWorkoutWithHeartRate (6385) and _enrichWorkoutDocWithHeartRate (3992) merge summary.heartRateData.detailed.hrSamples onto the workout doc. Capped at 10,000 samples per workout via the LocalDB query limit: at 4052/6424 — but a 2-hour Apple Watch workout still routinely produces 500-2000 samples × ~50 bytes ≈ 100-200KB per doc. The 3992 path has an unconditional 30s retry at 3981-3983 that re-issues the same set() (idempotent in content but a second Firestore write event). The 6385 path does not retry.
6.2 Daily digest writes
_digestAppend (1808) writes a single field-merge to users/{uid}/dailyDigest/{localDate} per metric. Fires per sample — every _updateDailyMaxHeartRate, _upsertDailyMean, _updateDailySteps, etc. calls _digestAppend after writing its own doc. A HealthKit sync delivering 200 HR samples produces ~200 digest writes to the same daily doc with merge=true. Range guards exist (1813-1816). _digestAppendMulti (1842) is used for batched fields like sleep stages.
6.3 On-launch backfills (writer:1013-1019)
Five functions fire 10s after app launch on every session:
| Function | Window | Notes |
|---|---|---|
_repairCardioMetricsIfNeeded (2373) | last 14 days | gated by users/{uid}/maintenance/cardioRepairV2 flag (2381) — runs once ever per user, then idempotent |
runSmartBackfill (430) | last 14 days | lease per (uid, date) at 518-523 — won't repeat the same date in 24h |
backfillWorkouts (3799) | last 14 days | re-issues onSummaryCreated(items) for each workout — re-fires the workout-header write at 3977 |
_backfillMhrLastDays (5134) | last 14 days | one mhr doc per day per source |
_backfillRmssdLastDays (5196) | last 14 days | one hrvr doc per day per source |
The header-write at 3977 is unconditional — every backfill pass re-issues set(header, merge: true) for every workout in the window, even if the bytes are identical. Each write is a Firestore trigger event regardless of byte equality.
6.4 Read-modify-write paths (race-prone)
Three daily aggregators read existing doc, increment, write back without runTransaction:
| Path | Writer line | What it accumulates |
|---|---|---|
_updateDailySteps | 5245 | aggSum, aggCount for steps |
_updateDailySleep | 5308 | aggSum, aggCount for sleep minutes |
_upsertDailyMean | 6255 | aggSum, aggCount, derived mean for rhr/hrvs/hrvr/spo2 |
Under concurrent inflow (Apple Watch sync + Terra webhook + on-launch backfill) the read-then-write window can interleave; second writer overwrites first writer's partial aggregate.
6.5 Hidden writers (sources outside basishybrid that write the same collections)
| Writer | Where | Trigger | Note |
|---|---|---|---|
| Backend sleep gap-filler | basis-functions/functions/src/functions_health_mirror.py:331 _synthesize_sleep_gap_filler() | on_document_written on users/{uid}/events/{eventId} | Writes synthetic sleep docs into users/{uid}/healthSummaries when clusters are incomplete. The user is not the only writer. Possible feedback if basishybrid's sleep sync triggers this which writes back. |
| Backend mirror | basis-functions/functions/src/functions_health_mirror.py (top-level trigger) | on_document_written on users/{uid}/healthSummaries | Mirrors to clinicsv2/{clinicId}/clinic_users/{clientId}/healthSummaries (§4.4). The mirror itself counts as a write event from Firestore's POV — fires downstream triggers on the clinic-side collection. |
6.6 Hidden cost: RAG embedding trigger on every healthSummaries write
basis-functions/functions/src/functions_rag.py:674-686 registers an on_document_written Firestore trigger on users/{userId}/healthSummaries/{summaryId}. Every write generates a Vertex AI text-embedding-004 embedding (768 dimensions). Combined with the mirror chain:
basishybrid writes 1 healthSummary doc
→ mirror writes 1 clinic doc (2 write events total)
→ RAG trigger fires on both (2 embedding API calls)
→ derived-metrics triggers if any
Per-write cost multiplier: ~4-5x what a Firestore-only view suggests. Every redundant write — every idempotent re-set during a backfill, every _digestAppend call from a 200-sample HR sync — pays this multiplier. This is the biggest single piece of evidence that "kill unnecessary writes" is the right framing rather than "scale up the function memory."
7. Reader / writer comparison
Pairing §6 (what basishybrid writes) against §1, §2, §3 (what basisflow-web reads).
7.1 Types written but never read (delete candidates)
type | Where written | Read sites found |
|---|---|---|
hrb (hrBatch) | Zero across basisflow-web, basisweb, basis-functions, basishybrid client reads. Was pure dead weight; pipeline now removed. |
That was the only confirmed-orphan. Other under-read types (whr, vo2, fli, dru, min) appear in getExactQueryTypes (lib/biomarker-utils.ts:101-151) so removing them risks breaking biomarker fallbacks. Keep them.
7.2 Embedded fields written but never rendered
| Field | Write site | Reader status |
|---|---|---|
summary.hypnogramSamples (8K-32K entries per sleep) | _upsertAppleSleepSession (writer:3140-3176) | Fetched at PlanTab.tsx:7730 but never rendered — sleep UI at 8486-8522 uses only summary.sleepDurationsData totals. Strip before write — pattern matches the GPS strip at writer:2885 (already shipped). |
summary.positionData + summary.polylineMapData (Terra GPS track) | _normalize (2885) | No polyline decoder in basisflow-web/basisweb. Stripped at write site as of 2026-05-18 — see commit on feat/extracted-health-data. |
summary.heartRateData.detailed.hrSamples | _enrichWorkoutWithHeartRate (6385) + _enrichWorkoutDocWithHeartRate (3992) — capped 10K | Actively used — MetricsTab.tsx:1164 calls computeZonesFromHrSamples(hrSamples, durationMinutes, clientAge); PlanTab.tsx:1447-1458 checks presence for activity inclusion. Keep but cap lower — ~500 samples = 1 per ~7s for a 60-min workout, sufficient for zone time-in-range. |
7.3 Read sites the writer doesn't satisfy cleanly
Tracing readers from §1/§2 back to writer paths in §6:
| Reader expectation | Writer reality | Action |
|---|---|---|
| PlanTab walks 20 paths for HR samples (§1.2 cleanup signal #3) | Writer writes to summary.heartRateData.detailed.hrSamples (canonical, writer:6411). The other 19 paths exist because legacy Terra/provider payloads landed there before basishybrid normalized them. | Writer is canonical for new writes. Legacy data in old docs persists. One-shot migration to drop fallback paths is a backfill problem, not a writer problem. |
MetricsTab merge policy treats per-day healthSummaries doc + dailyDigest doc as equivalent for the same (date, type) (§2.1) | Writer produces BOTH for the same metric: _upsertDailyLatestDoc for mhr/rhr/hrvr writes a per-day healthSummaries doc, AND the same writer path calls _digestAppend to update the matching field on the dailyDigest. Same data, two docs. | If dailyDigest carries the field, the per-day healthSummaries doc is redundant — and each one fires the RAG trigger. Per-aggregator decision: which fields does basisflow-web read ONLY from the per-day doc? If none, delete the per-day write. |
MetricsTab reads extracted map preferentially (§3.5) | Writer produces extracted via _buildExtractedMap (called at 2899 from _normalize) only on per-sample docs. Per-day aggregator docs (mhr, rhr, hrvr, body, bp) do not include extracted based on the schema audit's table — the per-day shape is {id, uid, type, start, end, tz, unit, value, source, ingestedAt, updatedAt} without it. | Web reader code falls back to summary.* nested paths for these (§3.4). Adding extracted to per-day docs would close the gap and let basisflow-web retire the fallback paths in biomarker-utils.ts:244-352 (cleanup signal #8). |
§3.6 readers expect type canonical but PlanTab.tsx:1809 accepts vt='wo'/vt='sl' + type='wor'/type='sle'/type='sleep' | Writer produces type='wor' (basis_health_type.dart workout stringType) and vt='wo' (legacy compat, writer:2922-2939). Sleep writes type='sleep' with vt='terraSleep' (writer:3118). | Writer is honest about the canonical names; the vt legacy alias is the noise. Drop vt from writer output once we confirm no reader requires it. |
| §4.4 mirror direction — web reads BOTH user and clinic paths for healthSummaries | basishybrid writes only the user path (writer comment at 1774: "Client writes only to user path; backend mirror handles clinicsv2"). The backend mirror in functions_health_mirror.py propagates to clinic path. | Writer-side fine. Reader-side has the duplication (PlanTab B-series + C-series). Cleanup signal #1 in §1.1 covers this. |
§3.7 protocols habits vs activities — both arrays read | basishybrid does not write protocols — backend writes both (functions_protocols.py and similar). Out of scope for writer audit. | Backend-side fix. |
§1.5 gender numeric (cleanup signal #4) | basishybrid writes gender as BasisGender.index per basiscore/lib/src/models/user/constants/gender.dart — enum order: female(0), male(1), other(2), unknown(3). So basishybrid writes gender: 1 for male, not female. | Web's mapping at PlanTab.tsx:2195-2196 is wrong. Web reads 1 → Male (correct by accident from a different convention?) and 2 → Female (wrong — should be "other"). Verify in production data before "fixing" either side. Recommended endpoint: migrate writer to string "male"/"female"/"other"/"unknown" and delete the numeric branches — covered by §5 item 1 but writer-side action is required. |
7.4 Net new entries to add to §5 "Top issues to fix"
Insert at the top of the ranked list:
- Investigate the RAG embedding trigger (
basis-functions/functions/src/functions_rag.py:674-686). It fires on everyusers/{uid}/healthSummarieswrite — every write costs 1 embedding API call, doubled by the mirror. This is the single largest piece of leverage on the cost side. Tasks: (a) gate to specific high-value types (workouts, labs) instead of every doc; (b) batch embeddings within a 30s window per user; (c) skip on idempotent re-writes via prior-bytes comparison.
0a. Delete the ✅ SHIPPED 2026-05-18 on hrb (hrBatch) write pipeline.feat/extracted-health-data. Writer 6315-6382 plus call site at 1177 removed. Saves hundreds of writes per active-HR hour per user, plus the same number of mirror writes and RAG embedding calls.
0b. Strip summary.hypnogramSamples from sleep writes. Fetched but never rendered. Same surgical pattern as the GPS strip at writer:2885. Sleep UI is fed by summary.sleepDurationsData totals.
13a. Backend _synthesize_sleep_gap_filler writes to users/.../healthSummaries. A second writer beyond basishybrid. Any single-source-of-truth migration must include it. basis-functions/functions/src/functions_health_mirror.py:331.
13b. Three read-modify-write aggregators, not just one. Steps + sleep + mean — wrap all three in runTransaction OR delete the per-day healthSummaries doc and rely on dailyDigest if reader audit confirms parity. Writer lines 5245, 5308, 6255.
13c. extracted map missing from per-day aggregator docs. Per-day mhr/rhr/hrvr/bp/body/wei/vo2/sleep docs don't carry extracted. Web falls back to nested summary.* paths because of it. Either add extracted everywhere or stop writing the per-day doc when digest covers the field.
Updated 2026-05-18 with writer-side findings from three-agent audit (schema/reader/adversarial). Verification pass: hrb stringType confirmed against basiscore enum; PlanTab.tsx:1817 hrSamples filter confirmed; functions_rag.py:674-686 trigger registration confirmed. Open work: §5 items 0, 0a, 0b, 13a-c queued as Task IDs #80-85 in the session backlog.
8. Case study: HR zone-minutes mismatch between platforms
When the same workout shows different zone-minute totals on basishybrid and basisflow-web, the cause is not a sync bug — it's that the same number is computed five different ways across the system, and the formulas disagree.
8.1 The five representations of "zone minutes"
For a given user-day, zone time-in-zone data may exist in any of these forms simultaneously:
| # | Where | Doc / field | Written by | Read by |
|---|---|---|---|---|
| 1 | users/{uid}/healthSummaries | doc with type='hrDaily', fields z1..z5 (in minutes) | _upsertDailyHrSummaryApple (writer:2500), _upsertDailyHrSummaryTerra (writer:2669) — once per day per source | MetricsTab.tsx:947, 1071-1122 (zone-tile synthesis) |
| 2 | users/{uid}/healthSummaries | separate docs with type='hrzone1'..'hrzone5', plus combos 'hrzone34', 'hrzone45', 'hrzone234'. value field is in minutes | _updateHrZoneDurations (writer:6104-6142) — per workout or per HR-summary trigger | not explicitly read in MetricsTab.tsx zone synthesis (which only filters type='hrDaily') |
| 3 | users/{uid}/dailyDigest/{localDate} | fields zone1Minutes..zone5Minutes (and possibly zone34Minutes etc.) in minutes | _updateHrZoneDurations digest append (writer:6147-6149+) | MetricsTab.tsx (digest primary path, §2.1) |
| 4 | users/{uid}/healthSummaries workout doc | summary.heartRateData.detailed.hrSamples array (raw HR samples, capped 10K) | _enrichWorkoutWithHeartRate (writer:6385), _enrichWorkoutDocWithHeartRate (writer:3992) | MetricsTab.tsx:1124-1231 → computeZonesFromHrSamples (line 128-178) — used as fallback for non-Apple workouts |
| 5 | clinicsv2/{clinicId}/clinic_users/{clientId}/healthSummaries mirror | same shapes as 1 + 2 + 4 above, mirrored by functions_health_mirror.py | backend mirror | PlanTab.tsx (B-series, §1.1) — sums independently from the user path |
Five surfaces, three computation methods, two coverage windows. Mismatch is the default state.
8.2 Three computation methods that disagree
Method A — basishybrid classifyHRReadings (basiscore/lib/src/analytics/analyzer_hr.dart:185)
Used by _updateHrZoneDurations (writer:6089). True time-in-zone: walks the sorted (timestamp, hr) readings, treats consecutive readings as intervals, sums interval durations per zone bucket.
Zone breakpoints (writer:6085): [0.5, 0.6, 0.7, 0.8, 0.9, 1.0] × maxHR — six breakpoints, five zones, HR below 50% maxHR is excluded entirely.
MaxHR derivation (writer:6066-6084): reads users/{uid}.dateOfBirth, computes (220 - ageYears).clamp(100, 210). Falls back to 185 bpm when DOB missing.
Method B — basisflow-web computeZonesFromHrSamples (MetricsTab.tsx:128-178)
Used as fallback when only raw HR samples are available and the workout source isn't Apple (line 1167). Proportional-by-count:
z1: (z1Count / total) * durationMinutes
Zone breakpoints: [0.60, 0.70, 0.80, 0.90] × maxHR — four breakpoints, five zones, HR below 60% maxHR is Z1. (No lower bound on Z1.)
MaxHR derivation: Math.min(Math.max(220 - clientAge, 100), 210). Fallback when age missing: absolute thresholds cutZ2=120, cutZ3=140, cutZ4=160, cutZ5=180 — no relative scaling.
Method C — Terra hrZoneData (writer:5998-6013)
When the source is Terra, basishybrid trusts the device's manufacturer-computed zones — reads terra.heartRateData.summary.hrZoneData and uses whatever zone definition Garmin/WHOOP/Polar/Oura/etc. used internally. Each vendor uses different breakpoints, different maxHR estimates, different zone counts. Then projects them into the 5-zone schema by zone integer field.
Apple Health pre-computed zones — when Apple's HealthKit reports HKQuantityTypeIdentifierActiveEnergyBurned per-zone, basishybrid writes them straight through into hrDaily.z1..z5 (writer:2500). Apple's zone definitions are user-configurable in iOS Health settings — they can be % of max HR, % of HR reserve, or custom user-set BPM ranges. Three more sources of variance.
8.3 Five concrete disagreements you will see in production
-
Below 50% maxHR readings. A resting heart rate of e.g. 65 bpm for a user with maxHR=185 sits at ~35% maxHR → counted as Z1 in basisflow-web, excluded entirely in basishybrid. Visible as: basishybrid says "0 min Z1 today" while basisflow-web shows "300 min Z1". The discrepancy is real-data not bug, but the formulas are inconsistent.
-
Apple workout proportional vs interval. basishybrid Apple-fallback path (writer:6063-6094) computes time-in-zone using
classifyHRReadingsintervals — exact time. basisflow-web'scomputeZonesFromHrSamplesdoes(count / total) * duration— proportional. For a workout with bursty HR sampling (Apple Watch samples HR more densely during high-intensity intervals), proportional over-counts high-zone time and under-counts low-zone time. Same raw data, different totals. -
Apple skip path: which doc wins? basisflow-web MetricsTab line 1167 skips Apple workouts because "Apple Health writes pre-computed zones". So for Apple workouts:
- basishybrid produces a per-workout
'wor'doc with HR samples → web ignores zone-wise. - basishybrid ALSO produces
hrDaily.z1..z5for that day → web reads this for the daily tile. - But basishybrid's hrDaily is filled from… (writer:2500) the daily Apple HR data, not the workout-only samples. Daily totals include resting HR drift over 24h, not just the workout. Workout tile vs daily-zones tile disagree.
- basishybrid produces a per-workout
-
The
hrDailywrite is per-source. ID at writer:2519/2683 issha1(uid + t:hrDaily + date + source). So an Apple+Oura user gets twohrDailydocs per day. basisflow-web's MetricsTab merge policy (MetricsTab.tsx:1278-1308) sums them, doubling zone minutes for any minute both sources captured. -
Terra vendor-defined zones. Two different Garmin watches can use different zone definitions if the user customized one. basishybrid stores both as
hrZoneDatawithzone: int, and basisflow-web reads them through the samehrDaily.z1..z5field — no way to know the underlying breakpoints differed. Comparing a Garmin user's Z3 minutes to an Apple user's Z3 minutes is comparing apples to oranges silently. -
hrzone1..hrzone5docs vshrDaily.z1..z5field — separate code paths._updateHrZoneDurationswrites thehrzone*docs (writer:6135-6142)._upsertDailyHrSummaryApple/TerrawriteshrDaily.z1..z5(writer:2500/2669). The two run in different triggers from different inputs and can produce different values for the same day. basisflow-web readshrDailyfor zone synthesis but doesn't readhrzone*docs — so thehrzone*write is half-orphaned (used for combined-zone queries likehrzone34but not for the main tile). -
Apple-skip in basishybrid for
hrzone*docs (writer:6108-6110):if (preferApple && typeStr.startsWith('hrzone')) return;— when an ApplehrDailyexists for the day, basishybrid suppresses thehrzone*doc write. Sohrzone*docs only exist on Terra-only days. If basisflow-web ever reads fromhrzone*instead ofhrDaily, the answer changes depending on whether Apple wrote that day.
8.4 What "fixing" zone minutes looks like
A real fix has to pick:
- One zone definition — recommend
[0.5, 0.6, 0.7, 0.8, 0.9, 1.0] × maxHR, the existing basishybrid choice, including <50% as outside zones. UpdatecomputeZonesFromHrSamplesto match. Migrate Terra vendor zones into this scheme by re-bucketing from raw HR samples when available, or by interpolating zone boundaries. - One computation method — true time-in-zone via interval walking, not proportional-by-count. Run it ONCE in basishybrid, write the result, never recompute client-side.
- One maxHR derivation — same formula with the same fallback. Recommend
220 - ageclamped, no absolute-bpm fallback (use 185 if age missing, on both sides). - One source of truth doc —
dailyDigest.zone1Minutes..zone5Minutesas canonical. Delete the per-zonehrzone*healthSummaries docs. Delete thehrDaily.z1..z5redundant copy. basisflow-web reads only fromdailyDigest. - Per-source disambiguation — when multiple devices are active, decide a priority order (e.g., wrist-worn > phone-derived) and write only the winner's zones. Avoid summing two sources for the same minute.
- Terra vendor zones — re-bucket or annotate. Either re-classify Terra's raw HR samples into the canonical zones in basishybrid (and discard the vendor's pre-computed
hrZoneData), or include azoneSchema: 'manufacturer'flag so basisflow-web can warn users that Garmin's Z3 isn't necessarily Apple's Z3.
This is a multi-week project, not a one-line fix — and it ties into §5 issues 2 (formalize extracted) and 9 (HR sample paths).
Section 8 added 2026-05-18 with verification: classifyHRReadings confirmed in basiscore/lib/src/analytics/analyzer_hr.dart:185; basisflow-web formula confirmed in MetricsTab.tsx:128-178; basishybrid Apple-skip guard confirmed at writer line 6108.
9. Multi-clinic mirror fan-out — the dominant cost multiplier
Verified 2026-05-19 via Firestore query (basis-functions/tools/_find_user_clinics.js) against UID GoOU3d2QcyRZMD83IhbQeIclpwh2 (the project owner's account):
The user has clinic_users/{uid} documents in 17 distinct clinics, and every one of those clinics has ≥2000 healthSummaries docs mirrored into it:
CLINIC PAK6ZL Front Door Labs healthSummaries=2000+
CLINIC axuk-khwf-prkr Ready Practice healthSummaries=2000+
CLINIC boxt-imjb-jvlk Apex Integrative Medicine healthSummaries=2000+
CLINIC bvrl-xmod-llza Basis Demo Clinic healthSummaries=2000+
CLINIC dhvn-msxg-lcks DSCF healthSummaries=2000+
CLINIC edke-ynvm-dgep Hatter Labs healthSummaries=2000+
CLINIC euvo-cyim-iiyr Monarch healthSummaries=2000+
CLINIC fdnz-wuew-aewu Pitaya Health healthSummaries=2000+
CLINIC fgpo-sazh-enuy Bondi Longevity Lab healthSummaries=2000+
CLINIC fivr-bdiq-beaf 7Longevity healthSummaries=2000+
CLINIC overland-wellness Overland Wellness healthSummaries=2000+
CLINIC ovld-wlns-care Overland Wellness healthSummaries=2000+ ← possible dup
CLINIC wamf-loge-csik Front Door Labs healthSummaries=2000+ ← possible dup
CLINIC wsqz-ncnv-uosr Viamed Salud healthSummaries=2000+
CLINIC wxvi-udwq-gtal Echo Center healthSummaries=2000+
CLINIC xdoz-pjao-zxvm Monarch Clubs healthSummaries=2000+
CLINIC xdrq-cxdq-rbjt Sunshine Medical Services healthSummaries=2000+
USER-SIDE users/{uid}/healthSummaries count=2000+
9.1 The arithmetic
Per the writer-side audit (§6) basishybrid writes only to users/{uid}/healthSummaries. The mirror function in basis-functions/functions/src/functions_health_mirror.py fans out to clinic_users/{uid}/healthSummaries paths.
For a multi-clinic user, one basishybrid write produces 18 Firestore write events (1 user + 17 clinic) and 18 RAG embedding calls (because the trigger fires on each mirror). For an Apple Watch user generating ~50 healthSummary writes per hour during normal activity, that's ~900 write events and ~900 Vertex AI calls per active hour — for one user.
Prior estimate of "4-5x per-write multiplier" was based on the implicit assumption of a single clinic membership. For a multi-clinic user it's ~18-35x depending on which downstream triggers fire per clinic doc (mirror handler, derived-metrics, RAG, data warehouse sync).
9.2 Why this matters
This is the single largest cost lever in the system — bigger than the RAG trigger discussion in §6.6. The RAG trigger is part of the multiplier; the fan-out itself is what amplifies it. Killing the RAG trigger but keeping the 17-way fan-out still leaves 17 redundant Firestore write events per source write.
It also reframes the storm postmortem: a multi-clinic active user generates write volume the prior single-user analysis can't account for, and disproportionate fraction of the daily Cloud Functions budget hits on a handful of multi-clinic users (founders, staff, demo accounts).
Two of the 17 clinics appear to be duplicate setups for the same logical org (Overland Wellness × 2, Front Door Labs × 2). Worth investigating separately whether one is a stale shadow that can be deleted to immediately drop the multiplier from 17 to 15.
9.3 Architectural options
(See also §6.5 — backend _synthesize_sleep_gap_filler is a second writer to the same user-path that fires the same fan-out.)
Option A — Eliminate the mirror, grant clinics user-path read access via rules.
- Firestore rules engineered so a clinician at clinic X can read
users/{patientUid}/healthSummariesifpatientUidis inclinic_usersof clinic X. - Drops the multiplier from 18 to 1 immediately.
- One source of truth. No drift.
- Requires significant rules engine work + reader migration in basisflow-web (PlanTab today reads both halves, MetricsTab only reads the user-path).
- Permission/HIPAA review needed: the user path becomes the access point for every clinic, so the row-level grant model is critical.
Option B — Single shared "patient health" bucket per user, granted to N clinics.
- New collection like
patient_health/{uid}/...owned by the patient, with explicit ACLs for each authorized clinic. - Intermediate between A and C. More moving parts.
Option C — Keep the mirror but make it selective.
- Mirror only "view-essential" recent data (last 90 days? last 30 days?) per clinic, not full history.
- Read-on-demand from user-path for historical lookups.
- Drops the multiplier somewhat but doesn't eliminate it.
- Lower migration risk than A. Could be an interim while A is built.
Option D — Mirror gated on clinician activity.
- Only mirror to clinics where a clinician has read the patient's data in the last N days.
- "Cold" clinic memberships (signed up, never visited) stop receiving mirrors.
- Hybrid of B and C.
Recommend A for the long term, D as the interim if rules work is too big to ship immediately. Either way, the duplicate clinics (Overland × 2, Front Door × 2) should be reconciled to a single canonical clinic per logical org — that's a 12% multiplier reduction with zero code change.
10. basisflow-web write surface — migration scope is bounded
Audit 2026-05-19 against app/(main)/clients/[id]/components/PlanTab.tsx, MetricsTab.tsx, the Labs tab, and components/protocol/AssignProtocolModal.tsx.
25 distinct write sites total: 17 direct Firestore writes, 8 callables.
10.1 Already callable-routed (8 sites)
| Surface | Callable | Notes |
|---|---|---|
| Protocol assignment | clinic_service with request_type: 'assign_protocol' (AssignProtocolModal.tsx:826) | The cleanest example — single call handles habits/recurring events/biomarker bindings |
| Lab document analysis | analyze_document_summary_v2 (LabsTab:921), analyze_document_summary fallback (LabsTab:930) | Backend handles all writes |
| Lab values import | import_lab_values (LabsTab:1059, 3313) | Two call sites — same callable |
| Junction order cancel | junction_cancel_order (LabsTab:1712) | |
| Junction create user | junction_create_user (LabsTab:3541) | |
| Junction order test | junction_order_test (LabsTab:3699) | |
| Junction catalog/area/availability reads | junction_get_test_catalog, junction_get_area_info, junction_get_phlebotomy_availability | Read-only, but listed for completeness |
10.2 Direct writes to migrate (17 sites)
Grouped by what they actually do:
| Group | Sites | Operation | Target |
|---|---|---|---|
| Activity status changes | PlanTab:3619 (complete), 3638 (N/A), 6977 (generic update) | updateDoc | clinicsv2/{c}/clinic_users/{u}/events/{id} |
| Workout preset save | PlanTab:6773 (protocol), 6778 (event + protocol dual-write), 6797 (non-protocol event) | updateDoc + setDoc(merge) | events + protocols |
| Strength session create | PlanTab:3667 | setDoc | clinicsv2/{c}/clinic_users/{u}/events/{id} |
| Coach notes save | PlanTab:6542 | setDoc(merge) | events or healthSummaries |
| Session notes | EventDetailsDrawer:1440 | setDoc | clinicsv2/{c}/sessionNotes/{id} |
| Meeting URL set/clear | EventDetailsDrawer:2021, 2023 | updateDoc + deleteField | clinicsv2/{c}/scheduled/{id} |
| Metric favorites toggle | MetricsTab:2003, 2008 + LabsTab:1821, 1826 | setDoc(merge) | clinicsv2/{c}/clinic_users/{u}/favorites |
| Custom benchmark CRUD | MetricsTab:2759 (delete), 3219 (save) + LabsTab:2579 (delete) | deleteDoc, setDoc | clinicsv2/{c}/benchmarks/{code} |
| Document summary | LabsTab:898 (user), 901 (clinic mirror) | setDoc dual-write | both user + clinic paths |
10.3 Migration shape
The 17 direct writes collapse to ~5 canonical callables:
update_activity_status— covers PlanTab:3619, 3638, 6977save_workout_preset— covers PlanTab:6773, 6778, 6797toggle_favorite— covers all 4 favorites sites ({collection, code}payload)manage_benchmark— covers MetricsTab:2759, 3219 + LabsTab:2579 (save/delete)save_event_metadata— covers session notes, meeting URL, coach notes, strength session create
LabsTab document upload (898/901) is a special case — it's the only dual-path direct write and should go through a save_lab_document callable that handles user + clinic write atomically.
Net result: ~17 client-side direct writes → ~5 server callables. Smaller migration than feared.
11. Junction integration — backend mostly built, basishybrid is the gap
Audit 2026-05-19 against basis-functions/functions/src/functions_junction.py (1341 lines).
11.1 What exists today
18 callable Cloud Functions + 1 HTTP webhook, all using x-vital-api-key against https://api.sandbox.tryvital.io (Junction is built on Vital's API — the env says sandbox).
Catalog: junction_get_test_catalog (line 382), junction_get_area_info (575), junction_get_psc_info (616), junction_get_phlebotomy_availability (637), junction_get_psc_availability (658).
Patient: junction_create_user (531) — writes users/{patientId}.junctionUserId.
Orders: junction_order_test (777), junction_order_from_package (910), junction_get_orders (956), junction_cancel_order (976), junction_get_results (1021).
Package management: junction_create_lab_package (419), junction_update_lab_package (461), junction_delete_lab_package (494), junction_list_lab_packages (510).
Clinic enablement: junction_get_status (304), junction_enable (329), junction_disable (360).
Webhook: junction_webhook (1056) — handles labtest.order.created, labtest.order.updated, labtest.result.created, labtest.result.updated. Routes via top-level junction_order_index/{junctionOrderId} lookup since Junction's payloads don't carry clinic context.
11.2 Firestore write paths from Junction integration
| Path | Trigger | Shape |
|---|---|---|
clinicsv2/{c}/config/junction | junction_enable / junction_disable | {enabled, enabled_at} |
clinicsv2/{c}/lab_packages/{id} | junction_create_lab_package | {packageId, name, testIds[], tests[], totalCost, totalPrice, markupPercentage, ...} |
clinicsv2/{c}/lab_orders/{id} | junction_order_test (initial), webhook (status updates), junction_cancel_order | {orderId, junctionOrderId, patientId, testIds[], status, appointment?, ...} |
clinicsv2/{c}/lab_orders/{id}/results/{rid} | labtest.result.* webhook | raw payload |
junction_order_index/{junctionOrderId} | order creation | {clinicId, orderId, patientId, createdAt} — webhook routing index |
users/{patientId}.junctionUserId | junction_create_user | scalar |
users/{patientId}/labs/{markerDocId} | webhook result ingestion (line 1249) | per-analyte: {analyteKey, displayName, value, unit, source:{type='junction'}, ...} — deterministic doc id from junction provider+timestamp |
clinicsv2/{c}/clinic_users/{patientId}/labs/{markerDocId} | webhook (line 1250) | clinic mirror of above |
11.3 What's NOT built
- basishybrid has zero Junction integration. No client-side callable invocations, no catalog UI, no zip-prompt, no TOS-consent flow, no order placement, no Junction order list display. Per
project_junction_integration.md: zip-first UX, TOS-only consent, team-per-clinic were design decisions. None implemented client-side. - Per-clinic Junction team model is NOT in code. Currently uses one global
JUNCTION_API_KEYenv var. The team-per-clinic model would require storing per-clinic Vital team IDs and credentials inclinicsv2/{c}/config/junction. - Lab results don't roll into
healthSummaries. Results only land inusers/{u}/labs/{id}— they don't appear in the daily-digest aggregator, biomarker history sparklines, or Atlas AI biomarker context unless those readers queryusers/{u}/labsseparately. Task #85 (canonical biomarker registry) needs to cover this. - Numeric-only auto-ingest. Comment markers ("positive"/"negative") and coded markers land in the raw results subcollection but never get structured into the labs collection.
- Order status state machine is implicit. Mirrors Junction's status field directly —
pending/submitted/in-progress/completedetc. — no explicit Basis-side mapping. - Cancellation has no pre-check.
junction_cancel_orderdoesn't check whether sample has already been collected; relies on Junction to reject.
11.4 What basishybrid needs to add for end-to-end Junction
- Catalog browser (calls
junction_get_test_catalog) - Zip-first area lookup (calls
junction_get_area_info→ shows availability) - TOS-only consent modal
- Phlebotomy or PSC selection
- Order placement (calls
junction_order_test) - Order status tracking UI (subscribes to
clinicsv2/{c}/lab_orders/{id}reads) - Result display (already exists for lab UI in general at
route_my_health.dart→ labs tab; needs to know to queryusers/{u}/labsfor Junction-sourced results too)
All of these are net-new client UI; none are backend changes. Per project memory the planned UX is zip-first.
12. End-to-end write pipeline — basishybrid + backend → Firestore (as of 2026-06-21)
This section captures the full set of writers that land data in users/{uid}/healthSummaries and the clinic-path mirror. §6 documented the basishybrid Dart writer in detail; this section adds the backend writers + the recent June changes that resolved several gaps.
12.1 The four write paths
┌───────────────────────────────────────────┐
│ SOURCE DEVICES │
├───────────────────────────────────────────┤
│ Apple Health Terra (Oura/Whoop/ │
│ (HealthKit) Garmin/Fitbit/etc.) │
└────────┬──────────────────────┬───────────┘
│ │
│ HealthKit │ Webhook POST
│ samples │ (HMAC signed)
▼ ▼
┌────────────────────────────┐ ┌─────────────────────┐
│ basishybrid (Dart) │ │ hook (Cloud Run) │
│ service_health_firestore_ │ │ - HMAC verify │
│ writer.dart │ │ - ACK 200 in <1s │
│ │ │ - Enqueue Cloud │
│ Triggered by: │ │ Task │
│ - per-sample listener │ └──────────┬──────────┘
│ - on-launch backfill x5 │ │
│ - habit event upsert │ ▼
│ - Terra-bridge FFI calls │ ┌─────────────────────┐
│ │ │ processTerraWebhook │
│ Writes: │ │ - Per-uid lock │
│ (A) Dart live writer → │ │ - Download DuckDB │
│ _normalize() │ │ - Per-type adapter │
│ sets vt='tes'/'wo'/ │ │ process_sleep │
│ etc. │ │ process_activity │
│ Embeds full │ │ process_body │
│ summary.toJson() │ │ process_daily │
│ conditionally │ │ - INSERT/REPLACE │
│ │ │ into DuckDB │
│ (B) Dart Terra-bridge │ │ - Upload DuckDB │
│ path (write same │ │ - Mirror to │
│ _normalize() shape │ │ Firestore (C) │
│ when Rust bridge │ └──────────┬──────────┘
│ hands Terra data │ │
│ back to Dart) │ │ writes
│ │ ▼
└────────────┬───────────────┘ ┌──────────────────────────┐
│ │ (C) terra_adapter │
│ writes │ mirror_doc │
│ │ vt='terraSleep' / │
│ │ vt='terraActivity' │
│ │ Doc ID: s:UUID / │
│ │ a:UUID │
│ │ Includes summary. │
│ │ heartRateData. │
│ │ detailed.hrSamples │
│ │ (as of 2026-06-21) │
│ └──────────┬───────────────┘
│ │
└──────────────┬───────────────┘
│
▼
┌────────────────────────────────────┐
│ users/{uid}/healthSummaries/{id} │
│ (Firestore — authoritative) │
└─────────┬──────────────────────────┘
│ on_document_written trigger
▼
┌────────────────────────────────────┐
│ (D) user_summary_clinic_id_ │
│ stamper │
│ Fan-out to every clinic the │
│ user belongs to │
│ (functions_health_mirror.py) │
└─────────┬──────────────────────────┘
│
▼
┌────────────────────────────────────┐
│ clinicsv2/{clinic}/clinic_users/ │
│ {uid}/healthSummaries/{id} │
│ (Per-clinic mirror) │
└─────────┬──────────────────────────┘
│
▼
┌────────────────────────────────────┐
│ basisflow-web (PlanTab + Metrics) │
│ reads BOTH user-path + clinic- │
│ path, dedupes client-side │
└────────────────────────────────────┘
12.2 Writers summary
| ID | Writer | Where | Triggers | Doc ID scheme | vt value(s) | Note |
|---|---|---|---|---|---|---|
| A | basishybrid _normalize() → live | service_health_firestore_writer.dart:3110 | HealthKit per-sample listener; some Terra-bridge paths | sha1(uid+source+type+start+end+unit+value) hex | 'tes' / 'wo' / etc. (short form — BasisValueType.X.stringType) | Embeds full s.summary.toJson() when shouldEmbed matches (line 3153-3175) |
| B | basishybrid _upsertAppleSleepSession | service_health_firestore_writer.dart:3293 | HealthKit-mediated sleep containers | sha1(uid+'sleep'+source+start+end) | 'terraSleep' (long form) | Header write at 3404 + summary write at 3464 (two-step, see "empty shell" issue) |
| C | backend terra_adapter.process_sleep / process_activity | terra_adapter.py:866 / :353 | Terra webhook → Cloud Task | s:<summary_id> / a:<summary_id> | 'terraSleep' / 'terraActivity' | As of 2026-06-21 includes summary.heartRateData.detailed.hrSamples |
| D | backend user_summary_clinic_id_stamper | functions_health_mirror.py | on_document_written on users/{uid}/healthSummaries | Same ID as source | Same as source | Fans out to every clinic the user belongs to — the multi-clinic cost multiplier from §9 lives here |
12.3 Reader expectations (recap from §1-4)
basisflow-web PlanTab reads from BOTH users/{uid}/healthSummaries AND clinicsv2/{clinic}/clinic_users/{uid}/healthSummaries (8 queries total, see §1.1 B1-B4 + C1-C3 + D). Client-side dedup. For HR time-series specifically: reader checks 6 paths in priority order (PlanTab.tsx:3325-3343):
summary.heartRateData.detailed.hrSamples← writers A and C land heresummary.hrSamples(top-level)deviceData.heartRateData.detailed.hrSamplesdeepDeviceData.heartRateData.detailed.hrSamplesembedded.heartRateData.detailed.hrSamplespayload.hrSamples
If none has data, no HR graph renders.
12.4 2026-06 changes recap
| Date | Change | Where | What it does |
|---|---|---|---|
| 2026-06-04 | Terra Phase 1 — HMAC + secret binding + no-5xx hook | functions_terra.py:hook | Prevents silent webhook circuit-break; signature verification now functional. See services/terra.md |
| 2026-06-04 | Terra Phase 2 — per-uid lock + INSERT OR REPLACE | functions_terra.py:_acquire_terra_lock + terra_adapter.py 6 sites | Fixes data loss on Oura/Withings revisions + concurrent webhook GCS races |
| 2026-06-18 | Bug A — recovery_service timestamp coercion | functions_recovery.py:_write_if_missing | Prevents PlanTab from missing recovery summaries (string vs Timestamp) |
| 2026-06-20 | Apple-only sleep filter LIFTED | service_health_firestore_writer.dart:1452 | Whoop / Oura-via-HK / Garmin-via-HK sleep now reaches _upsertAppleSleepSession. Was: !src.contains('apple.health') → return. Now: src.startsWith('co.tryterra.') → return. Terra-direct sources still use the backend path. |
| 2026-06-20 | Per-source isolation in stages query | service_health_firestore_writer.dart:3317 | Multi-device nights (Apple Watch + Whoop both writing to HealthKit) no longer cross-attribute stages |
| 2026-06-20 | HR zones embedded-samples fallback | service_health_firestore_writer.dart:_updateHrZoneDurations (~line 6345) | Whoop iPhone bundles per-second HR into the workout payload (not separate HealthKit records) → zones path was empty → now falls back to embedded samples |
| 2026-06-21 | Terra adapter mirror includes hrSamples | terra_adapter.py:process_sleep line ~1066 + process_activity line ~590 | Closes the read-side HR graph gap — was 8 of 9 user's Oura nights with no graph. Mirror now writes summary.heartRateData.detailed.hrSamples so PlanTab can plot the time-series. |
12.5 Pending improvements / known gaps
| Tracker | Description | Severity | Note |
|---|---|---|---|
| Task #69 | Apple-only sleep filter lift extended to backfill + synthesize + window-write paths | Medium | Live writer lifted 2026-06-20; backfill paths still skip non-Apple sleep |
| Task #71 | Whoop direct API integration (Whoop is NOT Terra-supported; only path today is HealthKit) | Low | Future build; HealthKit path covers most data today |
| Task #36 | T1/T2 Terra writer consolidation + TZ-from-profile | Medium | Two Dart writers (A and B in §12.2) writing the same doc with different IDs + different shapes |
| (no tracker) | vt='tes' vs vt='terraSleep' collision — different doc IDs, no actual collision, but reader complexity ↑ | Low | Consolidate writers (task #36 above would resolve) |
| (no tracker) | Empty shell docs observed in user's Firestore (vt='terraSleep', source='?', summary keys=[]) | Medium — open | Likely _upsertAppleSleepSession header write succeeded but summary write at line 3464 failed/timed out. Tracked separately; needs repro. |
| Backfill of old missing hrSamples | The 2026-06-21 fix is forward-looking; existing sleep + activity docs missing hrSamples won't get them | Low | One-shot script could re-process Terra cached payloads through the now-fixed adapter; or trigger Terra to re-send via dashboard |
| Tier 1 #1 | RAG embedding gating (biggest cost lever per §6.6 — every write triggers a Vertex embedding) | High cost | functions_rag.py:674-686 |
| §9 | Multi-clinic mirror fan-out (~18-35x cost multiplier on multi-clinic users) | High cost | Existing cost dominant concern |
12.6 Quick "where do I look" cheat sheet
| Symptom | Where to look first |
|---|---|
| Sleep doc exists but no HR graph | Check summary.heartRateData.detailed.hrSamples is present (writer C, as of 2026-06-21) |
| Workout doc exists but no HR graph | Same — check writer C activity mirror |
| User has Apple Watch + Whoop, sleep stages wrong | Per-source isolation in stages query (writer A/B, 2026-06-20 fix) |
| Whoop workout shows 0 HR zone minutes | HR zones fallback (writer A, 2026-06-20 fix) |
| Oura sleep missing entirely (only summary on dashboard) | Was Terra direct, expected source co.tryterra.oura — check both user and clinic mirror paths |
Empty shell terraSleep doc with summary keys=[] | _upsertAppleSleepSession partial-write (writer B); root cause not yet diagnosed |
| Duplicate sleep doc for same night | Likely one from writer A (vt='tes') + one from writer C (vt='terraSleep'). Reader dedupes by start time but not by source-pair |
Updated 2026-05-20 with multi-clinic mirror finding (§9, ~18-35x cost multiplier on multi-clinic users — single most important architectural insight), basisflow-web writes inventory (§10, 17 direct → ~5 callables migration scope), Junction integration state (§11, backend substantially built / basishybrid is the gap). Verification pass: _find_user_clinics.js raw output preserved in §9; Junction file size confirmed at 1341 lines + 18 callables registered.
Updated 2026-06-21 with §12 end-to-end write pipeline — captures all four write paths (Dart live, Dart Apple-sleep, terra_adapter backend, clinic-stamper trigger), the 2026-06-21 Terra hrSamples mirror fix that closed the HR graph gap, and the known-open issues (empty shell docs, writer consolidation, hrSamples backfill).