Path to Care

What it does

A community health worker (or the patient) sends a phone photo of a skin lesion plus a short text history. Path to Care runs the image and narrative through a multimodal agentic pipeline and returns:

• Top-3 conditions with confidence — never a single class label.
• Red / Yellow / Green urgency, with the rule-based safety net only ever escalating, never relaxing.
• Structured SOAP note for the clinic physician — chief complaint, HPI, exam, red flags, patient concerns.
• Patient framing that names the cost-of-care tradeoff in the patient's own numbers (transport ₹, daily wage ₹, distance km).

Every patient-facing string passes through a deterministic cardinal-rule rewriter that strips diagnostic phrasing ("you have X" → "signs suggest X") before it leaves the API. The output is decision support, not a diagnosis.

Architecture

• Multimodal Gemma 4 31B-it serves both the image classifier and the triage reasoner from one set of weights on a single AMD Instinct MI300X (192 GB VRAM, ROCm).
• Qwen 2.5-7B-Instruct handles text-only SOAP extraction via a DSPy NarrativeToSOAP signature.
• LoRA SFT on MI300X — supervised fine-tune of the triage head, 32 seconds wall-clock, loss 3.90 → 0.58 in one epoch. 180 MB adapter, served via vLLM --enable-lora.
• DSPy ReAct orchestrator coordinates five MCP servers: image classifier, SOAP extractor, village context, triage reasoner, and camera capture.
• vLLM (ROCm) in Docker for serving — OpenAI-compatible API on :8000. The Next.js frontend talks to it directly.
• Adversarial test set — 30 hand-authored cases (10 R / 10 Y / 10 G) with red-flag, contradiction, and off-distribution variants. Held out from training.

Results

Two complementary evaluations: a 30-case adversarial test set authored to probe the safety property (red flags, contradictions, off-distribution variants), and a 100-case held-out slice of the SCIN dermatology dataset to probe image-grounded classification.

Triage urgency — 30 adversarial cases

Reward R = 1.0 exact / 0.5 adjacent / 0.0 off-by-two.

Both runs hit the same ceiling — the single residual error is a Yellow → Green slip; no Red was missed. The headline here is the false-negative Red → Green rate at 0.0% — the safety property that matters in the field.

Image classification — SCIN top-16, 100-case holdout

Top-1 accuracy on a held-out slice of the Stanford SCIN dermatology dataset, restricted to the 16 most-frequent conditions.

A 32-second LoRA training run on the MI300X moved top-1 from 28% to 35% — a real learning signal beyond the saturated triage table above. Per-case results in results/scin_top16_topk_tuned.json and scin_top16_topk_baseline.json.

Try it

The cardinal rule

Path to Care never produces diagnostic statements. The output is always "signs suggest infection", never "you have cellulitis"; image output is always top-3 with confidence, never a single class label, never binary sick / healthy. Enforcement is defense-in-depth: a system prompt rule, a deterministic regex rewriter on every model output, and a unit test suite that fails the build on diagnostic phrasing.