Canonical source: docs/claude/atlas-operator-plan.md. This page is generated by docs/scripts/sync-handbook.mjs. Edit the source file in the repo; changes appear here on the next build.
Atlas as COO / Autonomous Operator — Master Plan
Strategy doc, drafted 2026-07-07. Synthesizes the operator discussion (loop engineering, autonomy model, model cascade, attribution, the growth/RevOps department, the cross-learning flywheel). Companion to
atlas-build-state.md. Reference companies: Polsia (nightly autonomous company-operator) and Boardy (voice-first outreach with double-opt-in). We build the compliant version.
0. The one-line thesis
Atlas becomes George's AI COO: a durable, 24/7, EV-prioritized control loop that autonomously runs growth, network/outreach, insights, and (later) fundraising-ops, while keeping platform/product/clinical work PR-reversible and gated. It optimizes for NRR + retention + profit above vanity metrics, uses a model cascade topped by Fable-5 as the strategist, and escalates to George only for the few irreversible calls.
0.5. Dual role — does the COO dilute the physician/growth assistant? NO, if isolated.
Two DEPLOYMENTS of one shared brain, isolated by tenant + per-tenant autonomy config:
- Clinic-tenant Atlas (the product clinics pay for): physician + growth assistant, scoped to that clinic's data + that clinician's permissions, PHI-gated, confirm-card gated (clinician always in control). Autonomy tier = conservative. UNCHANGED by any COO work.
- Ready-Practice-tenant Atlas (George's COO): autonomous operator on RP's OWN non-clinical business
(growth/ops/insights). Autonomy tier = high — but STILL never patient-messaging / charging / clinical.
LOAD-BEARING PRINCIPLE: autonomy is a per-tenant config, NOT a global change. Granting the COO
autonomy must not loosen one guardrail in any clinic tenant. Shared multi-tenancy (
clinic_idscoping) enforces data isolation. WHY IT'S A FLYWHEEL not dilution: the COO's capabilities ARE the product primitives (report engine → clinic reports; growth arm → clinics' growth assistant; insights → better clinical benchmarks/protocols). RP = customer-zero of its own product (highest-level dogfooding). Physician assistant = crown jewel + highest quality bar; everything else is measured against it. RISK MITIGATIONS: (1) clinical assistant keeps its OWN eval suite + gates — no COO change ships if it regresses clinical evals; (2) COO capabilities are ADDITIVE + tenant-scoped — clinical behavior (confirm cards, PHI gate, perms) stays byte-for-byte; (3) autonomous-operator framing is INTERNAL — clinic-facing promise stays "your assistant, you're in control" (never "Atlas autonomously runs your clinic"); (4) clinical safety NEVER inherits autonomous — high autonomy is non-clinical growth/ops only.
1. Objective / KPI charter (the utility function the loop optimizes)
Baked into the operator's system prompt:
"Revenue, profit, NRR, and retention trump all. Prefer durable, efficient, compounding growth over vanity metrics. Every initiative must trace to one of these. Report the truth even when it's bad." — a COO with a spine, not a cheerleader.
- North star: Net Revenue Retention (expansion + retention − churn).
- Tier 1: gross margin, CAC payback, logo + revenue retention, profit.
- Leading indicators it moves: activation rate, time-to-value, feature adoption (→ expansion), at-risk-churn signals (→ save plays), qualified-pipeline-per-CAC.
- Atlas owns a live KPI tree/dashboard, flags drift, proposes the highest-EV lever, and picks the right leading indicators (not vanity).
2. The operator loop (loop engineering — the hard parts, not just the shape)
Outer shape: Sense → Prioritize → Act → Verify → Learn → repeat. The engineering:
- Priority = dynamic EV, not a static list.
EV = impact × P(success) × urgency − cost − risk; pick argmax over candidate actions each cycle. Tier fallthrough: support > prod bugs > roadmap > feature requests > growth (growth wins whenever nothing higher-EV is queued). - Context engineering is the real bottleneck — fight context rot over long horizons: compact, evict-to-memory, retrieve, reassemble a lean context each turn.
- Sub-agents as context firewalls (not just parallelism) — each does messy work in its own context, returns a distilled result; orchestrator stays lean. Org-of-agents = context isolation.
- Durable execution + idempotency — checkpoint every step (resume after crash); idempotency keys on every side-effecting action (never double-email / double-PR).
- Online evals = the fitness function — score live decisions (did the fix resolve the ticket, did the growth experiment convert), feed back. Offline golden-sets gate changes.
- Non-blocking escalation — when it wants to exceed an autonomy tier, it pings George in Slack and keeps working other items while it waits.
Substrate already built: heartbeat (primitive loop), observation engine (sense), gap loop (learn), triage/build/QA agents (sub-agents), confirm cards (graduated autonomy), report engine (measure).
2.5 Loop taxonomy → Atlas mapping (Claude Code team's 4 loop types)
A loop = "an agent repeating cycles until a stop condition is met," classified by trigger, stop, and primitive. The four types map directly onto Atlas — and expose our ONE missing primitive:
| Loop type | Trigger → Stop | Claude Code primitive | Atlas analogue | Status |
|---|---|---|---|---|
| Turn-based | user prompt → Claude judges done | agentic tool loop | run_claude_agent tool loop (web/Slack) | ✅ have |
| Goal-based | prompt → evaluator confirms success or turn cap | /goal | evaluator-model stop gate on autonomous tasks | ❌ GAP — build |
| Time-based | interval → you cancel / work done | /loop, /schedule | heartbeat dispatcher + schedule_task + cron triggers | ✅ have |
| Proactive | event/schedule, no human live → per-task goal met | schedule + goal + workflows + auto | the COO operator loop itself (§0 thesis) | 🔜 compose |
| THE MISSING PRIMITIVE = goal-based stop gates. Today Atlas stops when the model judges it's done | ||||
(turn-based) or a human approves a card. For autonomous arms we need the /goal pattern: a **separate | ||||
| evaluator model** checks a deterministic success criterion before letting the loop stop, with an explicit | ||||
| turn cap so it can't spin. "Don't stop until every support ticket found this run is triaged, actioned, | ||||
| and responded — max N turns." Deterministic criteria (tests passed, score ≥ threshold, queue empty, ticket | ||||
| resolved) are what make this reliable; vague criteria let it quit early or loop forever. This is the safe | ||||
| throttle that lets us grant autonomy — the evaluator, not optimism, ends the loop. | ||||
The proactive COO loop is then a composition: schedule (when to wake) + goal (deterministic done + | ||||
| cap) + dynamic workflows (fan-out sub-agents to triage/fix/adversarially-review each item) + auto mode | ||||
| (runs without per-step approval, bounded by the autonomy tiers in §3). Pilot each new proactive loop on a | ||||
| small slice before a full run — workflows can spawn many agents. |
2.6 Quality + token discipline (the system around the loop is what makes output good)
Quality rules (bake into every autonomous arm):
- Verification skills — encode "what good looks like" as a reusable check the agent runs on ITSELF before declaring done, as quantitative as possible (it fixes → re-verifies from step 1, never hands back partial work). This is the self-check that substitutes for a human when we remove the confirm card. Clinical/payments arms keep the human card; low-stakes revertible arms graduate to skill-verified self-checks.
- Fresh-context adversarial review — a second agent (new context, not the author's reasoning) reviews before
merge/commit;
/code-reviewpattern. Already our workflow "verify" stage — make it standard on Tier-A auto-PRs. - Encode misses back into the system — when a result misses the bar, don't just fix the instance: add the skill/eval/rule so every future iteration clears it. The gap-loop (#30) is exactly this muscle.
- Clean codebase + reachable docs — Claude follows existing patterns; keep them clean and keep framework docs retrievable (KB / Sourcebot) so the loop pulls current best practice, not stale memory. Token discipline (a loop must have clear boundaries — ties to §4 model cascade):
- Right primitive + model for the job — don't spin multi-agent workflows or Opus on small tasks; cheap/fast models for high-volume low-stakes (Haiku triage), escalate only at judgment points (Fable/Opus).
- Clear success + stop criteria — specific "done" so it converges sooner (but not too soon).
- Scripts for deterministic work — running a script beats re-reasoning the steps every iteration (e.g. a report/form-fill script the loop invokes, not re-derived code). Cheaper and more reliable.
- Match interval to change-rate — don't poll faster than the watched thing changes; prefer event-triggered over time-triggered where possible (fewer wasted wakeups).
- Pilot before large runs + review usage — gauge cost on a slice first; track spend per skill/subagent/workflow
and kill runaways. (Our
budget.totalhard-ceiling + per-tenant kill switch in §8 enforce this.)
2.7 Loop ALTITUDE stack + the autonomy dial (independent-writer synthesis — orthogonal axis to §2.5)
§2.5 classifies loops by trigger (turn/goal/time/proactive). This axis classifies by altitude/scope — they compose (a proactive time-triggered loop can sit at any altitude). Bottom = mechanical, top = judgment:
| Altitude | Iterates on / ends on | Atlas mapping | Status |
|---|---|---|---|
| Execution (act→observe) | steps in ONE task / env feedback | run_claude_agent tool loop | ✅ have |
| Task (Ralph loop) | one artifact / spec-compliance + tests | fresh-context restart per iteration for isolated fixes | 🔜 adopt technique |
| Product (software factory) | a codebase + backlog / external signals (issues, logs, review) | §6 platform arm: ops-troubleshoot → auto-fix PR | 🔜 build, ratcheted |
| System (autoresearch) | prompts/harness/model/evals / eval+judge signals | self-improvement — elevate gap-loop (#30) to a first-class arm | 🔜 promote |
| Oversight | goals/budget/cull / "you" (George) | the human ring — set goals, allocate, kill | 👤 George holds this |
| LOAD-BEARING INSIGHT (validates §0.5 + §3): **autonomy is an independent DIAL on every loop, not one global | |||
| switch.** We can run a fully-autonomous execution loop inside a heavily-supervised product loop. So "Atlas | |||
| autonomy" is never a single setting — it's (altitude × arm × tenant), each dialed on its own trust curve. | |||
| Techniques/framing to adopt from this axis: |
- Ralph / task loop — for an isolated, spec-defined fix, RESTART the agent with a fresh context window each iteration (re-feed the full spec) rather than continuing a rotting session. One task per loop; ends on tests + spec compliance. Cheap-looking waste is the point — it defeats compaction drift. Use for Tier-A auto-fix PRs.
- Factory maturity ratchet — the platform arm (§6) graduates like Warp's Oz / Anthropic's Tag (now ~65% of their product code): start on low-risk repos/areas, raise the auto-merge rate as trust accrues (e.g. 20%→60%), humans stay on high-risk review. This IS our "earn autonomy as evals prove it" (§9), with a concrete dial.
- System loop / autoresearch = promote to a first-class arm. An OUTER loop whose product is the primary system: it tunes Atlas's prompts, tool harness, model routing, and the eval sets themselves, scored by evals+judges + a standing ask-a-human tool (accumulates George's tacit knowledge like onboarding a hire). Existence proofs: Karpathy's ~630-LOC overnight autoresearch; Meta Brain2Qwerty v2 (agents edited their own codebase to improve architecture) — but BOTH kept a human at the final checkpoint. So do we. Vocabulary discipline (keep us precise):
- Fan-out is NOT a loop. Dispatch→gather→validate (our parallel/pipeline workflows, "Agentic MapReduce") is a pipeline/topology deployed inside a loop — "a loop without feedback is just a for-statement." Don't call a workflow a loop; the loop is the thing with a feedback edge and a convergence signal.
- A loop without its signal doesn't converge — it runs till something external stops it. Every Atlas loop must name its exit signal (ties back to §2.5's goal-gate). An un-signalled loop is a runaway, not autonomy. THE REAL SCALING CONSTRAINT (design against it now): even Anthropic's Tag team reports being bottlenecked on review + the human's ability to conceptualize what the system is doing — the checkpoint humans keep becomes the cap. Implication for us: George's oversight ring is the scarce resource, so we must (a) make Atlas's actions legible (crisp diffs, why-traces, audit spine §7.3, dashboards) and (b) automate verification (self-check skills §2.6, fresh-context adversarial review, online evals) so oversight scales by exception, not by reading everything. Autonomy that isn't legible doesn't actually reduce George's load — it just hides the work.
3. Autonomy model (three tiers + two bright lines)
Reversibility is the safety substrate (George's reframe): everything a PR + constant backups → fixable.
- Tier A — Revertible (code/config/SEO/site/data/infra): FULL autonomy. PR + backup = the net.
- Tier B — Externally-irreversible but bounded (ad spend, outreach, hiring-sourcing): autonomous within budget / send-rate / brand-voice / compliance caps; exceeding a cap escalates (non-blocking). ("Revert the PR" is meaningless for a sent email or spent ad dollar — so bound, don't revert.)
- HARD-BLOCKED, never (wired as capabilities that don't exist):
- Messaging the clinics' patients/end-clients.
- Charging anyone (our clients or their clients).
- All within HIPAA / GDPR / SOC2 boundaries. Full audit trail (who/what/why/idempotency-key).
- Autonomy asymmetry: growth/network/insights ≈ non-PHI → more autonomous; platform/clinical → gated.
4. Model cascade (match model to marginal value of intelligence)
- Haiku: triage, classification, extraction, routing (high volume, low stakes).
- Sonnet: execution — draft replies, write code, run reports (workhorse).
- Opus: hard technical reasoning, adversarial verification, complex debugging.
- Fable 5 = the strategist/COO — rare, highest-leverage: roadmap prioritization, architecture "should we", growth thesis, reversibility judgment. Consulted at decision points, not per-action.
- Batch API (~50% cheaper) for background/non-urgent (overnight growth/intel/analysis).
- Compute allocation as policy: the loop meta-reasons "is this worth Fable/Opus or does Haiku suffice"; cost tracks value; idle → cheap batched growth, never Opus spinning on nothing.
5. Attribution & measurement (the growth reward signal)
Have: PostHog, GA, Facebook Pixel, Search Console, Google Ads. CRM = Ready Practice's own CRM (dogfooded in the demo clinic) → outcome ledger in our Firestore, Atlas reads directly. Deltas to build:
- Identity stitch (visitor→lead→deal, persistent first-party ID) — BUILD.
- UTM taxonomy + per-send IDs — tighten.
- Event capture — ✅ PostHog/GA.
- Server-side conversions (Meta CAPI + Google offline-conv) — BUILD (have pixel, not server feedback; this is what makes ad autonomy actually optimize).
- Multi-touch (position-based → data-driven) — BUILD.
- Outcome ledger — ✅ Ready Practice CRM (home-field advantage).
- Spend capture per channel — partial (Ads API ✅; add rest).
- Nightly reward rollup (CAC / ROAS / pipeline / NRR) → the growth-bandit reward; extends the
report engine as a
growthdataset — BUILD. Real work = 1, 4, 8 (+ tighten 2/5/7).
6. The arms / departments — phased order (George-approved)
- Sales-scribe / assistant (FIRST — fastest, safest, huge personal ROI): call scribe, meeting
notes, pipeline follow-up drafts, "manage sales without more headcount." Human-in-loop.
- Inbound sales AI voice (Boardy-style): prospect calls, AI explains Ready Practice, qualifies, books a demo. Grounded in full product knowledge. On the existing AI call center.
- GEO / SEO + website (autonomous, revertible, greenfield): content, schema, technical SEO, optimize to be cited by LLM search (Atlas is uniquely good — it IS an LLM). All PR-reversible.
- Outreach + ads + network (bounded tier): creator/affiliate/partner outreach with double-opt-in (Boardy pattern) + brand/health-claims filter + deliverability (warmup/SPF/DKIM) + rate limits; ads as a bandit (kill losers, scale winners on measured ROAS) under a budget cap. Graduate outreach: human-approve templates → autonomous once evals prove reply quality.
- Insights flywheel (the moat): aggregate NON-PHI operational data across clinics/gyms/med-spas (utilization, no-show, membership mix, pricing, adherence, retention drivers), privacy-safe (k-anon, min cohort, ToS opt-in) → (a) better benchmarks/protocols back into the platform, (b) content/case studies, (c) accelerator curriculum, (d) acquisition archetype screen for George. TRUST GUARDRAIL: acquisition analysis on archetypes / opt-in / public data — "acquire a specific platform customer" stays a human relationship decision, NOT an autonomous action.
- Hiring-ops (autonomous sourcing/screening/scheduling; offer/decision human).
- Platform / product / clinical (gated, ongoing): ops-troubleshooting tier-1 now; tier-2 auto-fix PRs after git reconciliation; the write-tool batch (#24) continues.
- Fundraising — LAST. Ops only (deck/data-room upkeep, investor CRM, scheduling, follow-ups). Voice/AI informs + qualifies + schedules, does NOT solicit/pitch/project (securities law).
6.B COO-as-COWORKER expansion (George, 2026-07-10)
Three additions that upgrade the operator from "works the queue" to "a real coworker building against a long-horizon thesis":
- Coworker channels — George works WITH the COO like a person: two-way SMS/WhatsApp, Slack (✅), voice calls + voice notes (make AND receive). All George/staff-facing on the operator tenant — the never-message-patients bright line is untouched (George ≠ patient). Twilio SMS/WhatsApp Business + ElevenLabs voice; same brain/session continuity across channels.
- Market-intelligence engine (real-time external learning) — beyond internal data + mapped competitors: watch X/social, funding rounds, healthcare-data & insurance policy changes, VENDOR API CHANGELOGS (Stripe/Terra/Recall/ElevenLabs/DoseSpot/Junction/Stedi — a breaking change auto-files a build-agent issue), growth case studies, organic traffic + revenue-from- organic KPIs (Search Console/GA → the #178 reward rollup). Sense → classify → route: changelog→build issue · policy→compliance brief · competitor→digest · case study→growth KB.
- Active information acquisition — when the data doesn't exist, GO GET IT: (a) drafted outbound insight-request emails (Tier B: George-approved sends initially, graduate on evals); (b) delegate research to humans (commission a researcher/VA under a budget cap). Spending under caps = Tier B bounded; the "never charge clients" line is about charging, not procurement.
- THE MASTER CHECKLIST ("$100M operating thesis") — the long-horizon backlog: a living, ~1000-step decomposition of what it takes to build a $100M software business (product, growth, retention, ops, compliance, capital readiness), maintained by Atlas (Fable-5 consults) and reviewed by George. The operator loop's idle-tier work source: every tick traces to a checklist node; progress is measured against it; the checklist itself is a living artifact that grows from market intel + learnings. #177 (scribe/voice), #178 (attribution), #159 (distribution) are all nodes of it — the checklist is what guarantees SURFACE COVERAGE + MOMENTUM rather than ad-hoc arms.
6.A Growth/RevOps arm — internal team structure (from the Frontal GTM piece; least authoritative — take the STRUCTURE, verify the NUMBERS with our own A/B via §10)
Don't build the outreach arm as one prompt — decompose it into a sub-agent-per-job org (context firewalls, §2), each backed by an editable skill (harness layer, §10 — plain text so sales/Isabel can tune copy + offer without eng):
| Role sub-agent | Owns | Skill/reference holds |
|---|---|---|
| Strategy | ICP, positioning, why-us | ICP definition, competitive angles |
| Signals | detect buy-readiness | signal taxonomy + how to track each (below) |
| Data | pull real contacts (phone/email) | enrichment sources, dedupe, CRM write |
| Copy | signal-hooked personalized outreach | templates, voice, claims-filter rules |
| Execution | send + follow-up (email/LinkedIn) | sequences, cadences, deliverability |
| System (orchestrator) | route request → the right specialist | master skill on top, reference files under (two-layer, lego-stack) |
| The lever = signal-based outreach: hook off the buy-signal, not a generic "congrats on your raise." (Their | ||
| claimed 3.4% → 15–25% reply lift is a single-vendor anecdote — treat as hypothesis, prove with our own A/B.) | ||
| Vertical-adapted signal taxonomy — OUR buyers are clinics / gyms / med-spas (NOT generic B2B SaaS): | ||
| new location or expansion · hiring practitioners/front-desk/GM · new owner / medical director / clinical lead · | ||
| switching booking or EHR software · adding service lines (med-spa, peptides, labs, wearables, IV, hormone) · | ||
| launching or raising membership pricing · review-volume / growth velocity spike · running paid ads (they value | ||
| acquisition) · funding / acquisition / franchise expansion. → score each account /100, tier (1 / 2 / excluded). | ||
| Action sequence (one motion): ICP from public data → scoring model → pull + tier accounts → select signals → | ||
| a play per signal → personalized sequence → matched LinkedIn/content (research-backed, infographic skill for visuals). | ||
| BRIGHT-LINE CLARITY (critical): this arm is prospect acquisition — selling Ready Practice to NEW clinics. | ||
| That is B2B outreach and is ALLOWED under Tier B (bounded: double-opt-in, health-claims filter, deliverability + | ||
| rate caps, human-approved templates → auto only once eval-green). It is CATEGORICALLY DIFFERENT from the hard-blocked | ||
| "message our clinics' patients/end-clients" (§3). Prospect ≠ patient — never let the arm's reach touch existing | ||
| clinic clients. Human keeps strategy / signal-choice / offer (their own caveat = our oversight ring, §2.7). | ||
| Dependency: signal capture is an attribution/§5 build (news/funding feeds, job-post scraping, tech-stack detection, review monitoring). |
7. Foundations (prerequisites — do carefully, together)
- Git reconciliation — make git = production truth (currently behind). DO READ-ONLY FIRST: diff deployed source vs git per function, show George each delta, no bulk overwrite. Gates tier-2 auto-fix.
- Automated, tested backups + restore — the substrate that makes "always revertible" real, not aspirational.
- Audit trail — every autonomous action logged (source, actor, idempotency key) to
transactions. - Eval harness — golden datasets + online evals; changes/rollouts gated on it.
8. Compliance & guardrails (always-on)
HIPAA / GDPR / SOC2 as boundary constraints. Hard blocks: patient messaging, charging. Privacy-safe cross-clinic aggregation. Health-marketing claims filter (no medical claims). Securities-law posture on fundraising. Budget caps + kill switch (per-clinic + global) on all spend/outreach.
9. Rollout principle
Start read-only + human-gated, earn autonomy as evals prove it. Each arm graduates independently. Never grant Tier-B autonomy to an arm whose evals aren't green. Reversibility + backups before autonomy.
10. Continual-learning engine (the compounding moat — the system loop, made concrete)
This is the mechanism for the §2.7 system loop and the answer to "how do Atlas + Ready Practice get better faster" — on BOTH products at once. Model after Replit Agent's year of practice. Foundational framing — we own two of the three layers. Agents improve at model · harness · context (ordered by who controls them). We run closed frontier models (Fable 5, Opus 4.8) → weight fine-tuning is OFF the table. So 100% of our compounding leverage is the two layers we fully control:
- Harness learning — mine production traces → systematically improve the code, tools, and instructions behind EVERY Atlas instance (prompts, tool schemas, skills, model routing, the confirm-card flow).
- Context learning — personalize per clinician / clinic / org so the product improves with every interaction (their phrasing, their protocols, their pricing/policies, what they accept vs. edit). Do both → compounding improvements shipped daily, no fine-tune required. Architecture = two measurement pillars + one optimization loop (Swiss-cheese: each layer leaks; together they catch — none is sufficient alone):
- Offline behavioral eval ("our ViBench") — grade the outcome/artifact, not local constraints. A golden
set drawn from anonymized/opt-in production traces, graded by natural-language rubrics. Pre-ship regression gate.
- Clinical track: "did the drafted protocol/note/answer match the clinician's intent + clinical correctness?"
- COO track: "did the support answer resolve the ticket?", "did the report match the spec?"
- Replit lesson to heed: frontier benchmark scores DON'T transfer to full-task outcomes, and models get worse extending their OWN output (errors compound) → prefer fresh-context (Ralph §2.7) + human checkpoints for compounding work; never let Atlas iterate unbounded on its own prior artifacts.
- Online A/B + production traces — real-user truth after ship. A/B most agent-affecting changes (prompts, tools, harness, model swaps), attribution kept clean across concurrent experiments. Signals: did the clinician keep the draft, did the ticket stay closed, did the growth experiment convert (§5 rollup), did sentiment/cost move. Caveat (Replit's): A/B aggregates don't self-explain (duration up = more work OR stuck?) — you need the cluster layer.
- Trace clustering ("our Telescope") — at scale nobody reads every trace. Summarize failure trajectories → embed → density-cluster → classify new sessions; evidence-grounded facets (Clio-style). Turns scattered failures into product questions and surfaces the ones hidden in plain sight. This is the gap-loop (#30) grown up.
- The self-improvement loop — "if agents can build software, they can improve the agent." Each pass: read logs + clusters + recent failures → pick a hypothesis → build a candidate (prompt edit / new skill / tool fix / harness change) → open a draft PR with the reasoning attached → measure vs. our-ViBench + A/B + baselines → recommend ship / iterate / drop. Records every attempt incl. failures, so future runs reuse wins + avoid dead ends. OUR UNFAIR ADVANTAGE — free per-action human labels. Most builders must synthesize evals; we HARVEST them. Every confirm-card accept / edit / reject is a graded human label, and the edit diff (proposed → shipped) is the highest-value training signal for both harness and context. Also: thumbs, dismissed observations, re-opened tickets. We start with a labeled stream Replit had to build Playwright agents to approximate. Human gates stay (shipping is NOT automatic) — where taste lives, esp. in healthcare:
- Hypothesis selection — George/leads decide which clusters earn the overnight budget (not every failure matters).
- Implementation architecture — smooth the path vs. change behavior vs. redesign the surface is a product call.
- Eval curation = shaping the hill. If the eval rewards the wrong thing, the loop faithfully optimizes toward it. For a clinical product this is a safety control: never reward speed/brevity over clinical correctness or consent.
- Launch approval — read the evidence, understand blast radius, own the rollout. (Ties to §9 + the oversight ring.) Substrate we already have: gap-loop (#30, primitive clustering), confirm-card signal stream, heartbeat (loop), workflows (candidate gen + parallel eval), report/eval engine (measure), audit spine (§7.3, traces). Gaps to build: our-ViBench golden sets (per track), the Telescope clustering job, A/B infra for agent changes, and the draft-PR self-improve loop. Sequence AFTER the §7 eval-harness foundation — the loop is only as safe as the hill it climbs.
Immediate next steps (when we start building)
- A. Foundation: git-reconciliation (read-only diff pass) + backup/restore + audit spine.
- B. Arm 1: sales-scribe/notes/follow-up + inbound sales AI voice (on the call center).
- C. Attribution deltas: identity stitch (#1) → reward rollup (#8,
growthdataset) → server-side conversions (#4). - D. The operator loop v1: EV priority + tier fallthrough to growth, on the heartbeat, with the KPI charter + Fable-5 decision consults. Needs the goal-based evaluator stop-gate (§2.5) as a prerequisite — autonomy without a deterministic exit signal is a runaway.
- E. Continual-learning engine (§10): after the §7 eval foundation — start by harvesting the confirm-card accept/edit signal (cheapest, highest-value), then our-ViBench golden sets per track → Telescope clustering (grow the gap-loop) → draft-PR self-improve loop. This is the compounding moat for BOTH the clinical assistant and the COO. Everything else (GEO/SEO, outreach/ads bandit, insights flywheel, hiring, fundraising) sequences after.