Debugging playbooks
Diagnose a production data/behavior issue safely (this is a live healthcare platform — no PHI in logs, no destructive ops without confirmation). 🤖 agent · 👤 staff · 🧑💻 George/engineering.
The core pattern: test AS staff against real deployed code
Most "X is broken for a clinic" issues are diagnosed by calling the deployed callable as a real staff user and reading what comes back.
Callable-test-auth (the reusable pattern): mint a custom token via the appspot service account signBlob → exchange for an idToken → call the deployed callable as staff (e.g. George's uid GoOU3d2QcyRZMD83IhbQeIclpwh2). See basis-functions/tools/assign_clinic_protocol.js and reference_callable_test_auth (memory) for the boilerplate. 🤖
Golden rules (CLAUDE.md): test with Admin role not System Admin (System Admin bypasses permission checks); never say "fixed" without runtime verification; use clinicsv2 not legacy clinics; Firestore field types are never guaranteed at runtime (guard every access).
Tools by symptom
| Symptom | Tool(s) | Notes |
|---|---|---|
| Slots / booking wrong | debug_sat_sweat_slots.py, debug_sat_sweat_coaches.py, audit_group_slot_visibility.py | slot listing, coach substitute coverage |
| Onboarding docs missing/mismatched | check_onboarding_docs.js, check_onboarding_match.js, backfill_onboarding_pdfs.py | signed-doc completeness (#219) |
| Duplicate / missing data | audit_duplicate_or_missing.py, audit_string_timestamps.py | date-only-string vs Timestamp is a classic |
| Lab values / registry | backfill_lab_verdicts.py, validate_lab_maps.py, validate_lab_ranges.py | ONE-way lab edit rule applies (CLAUDE.md) |
| Usage/billing counts off | debug_usage_count_miss.py, audit_xdoz_invoices.py | |
| Cost/invocation surge | Cloud Logging: crash signature (terminate called) vs volume vs loop | see the #540 investigation writeup; ratio-floor idea in #606 |
| Client-side (web/mobile) error | Sentry (once #636 lands) → auto-files to the board via intake | until then, report_client_error + Cloud Logging |
Reference: docs/claude/debugging-tools.md, docs/claude/firebase-backend.md, docs/claude/firestore-schema.md.
Cloud Run / infra checks
- Public-exposure drift:
configdrift_watchflags private→public and public→private (IAM-drop) transitions; intended-public services are allowlisted (_INTENDED_PUBLIC). - Function deploy dropped its invoker (403 after redeploy): the Firebase IAM-drop bug — re-add
allUsers run.invoker(needs George's gcloud auth:gcloud run services add-iam-policy-binding <svc> --region=us-central1 --member=allUsers --role=roles/run.invoker). 🧑💻 - Reserved Firestore id crash (
__nonexistent__/__.*__): validate ids before.document()(see #617 fix pattern infunctions_superbill.py).
Safety
Never git stash/git commit/firebase deploy/flutter clean without confirmation; never deploy functions from a wt-* worktree; PHI never logged; confirm before payments/Stripe/auth changes.