I owned the agent backend end-to-end — the codebase, the tech choices, and the merge gate —
while leading the team. Every stack decision was made under one hard constraint:
HIPAA compliance could never be an afterthought.
The stack, inside a compliance boundary
HIPAA-compliant boundary
Service layer
Python · FastAPI (ASGI) · Dockerized container on AWS
Orchestration
LangGraph — multi-agent state machine
Models
Azure OpenAI — chosen because we held a signed BAA with Azure
per-asset MCPs over HTTPS · EHR via OneUp (third-party)
Observability
self-hosted LangSmith — tracing & the data source for eval annotation
The dashed boundary is the HIPAA line: traces stay in a self-hosted
LangSmith, models sit behind an Azure BAA — no PHI ever leaves a covered
path.
Detail — why LangGraph + LangSmith specifically
LangGraph orchestrated the agents; we chose it largely because its sister product
LangSmith gave us first-class tracing. Running LangSmith self-hosted meant no
trace data left our infrastructure — which is exactly what made the whole eval program
HIPAA-safe over real patient conversations.
Front-end / middleware / back-end — decoupled by design
Front-end
Renders the conversation & recommendation UI — agnostic of backend schema.
↔
Middleware
Absorbs schema translation, so neither side breaks when the other changes.
↔
Back-end
The agent backend — emits payloads on a stable contract.
Payloads followed the OpenAI API spec as the standard contract; the
middleware absorbed schema drift so the front-end and back-end could evolve independently.
Modular enough to repurpose in 15 days
15 days
to repurpose the whole system for a new use case
🏆
recognized with an award for it
swap-in
any tool or knowledge base, any time
Modularity as a first-class goal, not a happy accident.
Every piece of the backend — models, knowledge base, integrations, observability — was
swappable behind a stable interface. That paid off concretely: we lifted the entire system
into a completely different use case in 15 days.
Detail — the MCP bet
I built each recommendation-asset integration as its own MCP, served over HTTPS rather
than embedded in the PHP codebase. That meant any future assistant or service
could consume the same recommendation value PHP did — the integrations became reusable
products, not private plumbing.
Team & operating model
Leadership was a triad — I led data & engineering, a counterpart led product, and a
project manager ran tasks & ops.
Data & Engineering
Me — owned the agent backend, tech stack, and merge gate.
Product
A counterpart lead owned product direction.
Project / Ops
A PM tracked tasks and operations across the team.
4
data scientists managed directly
~7
engineers influenced indirectly
80 / 20
IC work vs. people leadership
A player/coach who never left the codebase.
80% of my time was hands-on: owning the agent backend and gatekeeping every merge into it.
The other 20% was coaching the team — the same balance this role calls for.
Reflections — lessons & what I'd revisit
What carried the project
High agency beats seniority.
I wasn't the most senior person on this — a principal data scientist worked under me for it.
I owned the product and grew into managing that relationship purely because I showed high
agency from day one, and my leader recognized it and gave me room to bloom. High agency, to
me, means never staying blocked: if I believed in an idea and no one else did, I
still built and shipped it so it could earn a fair fight against the competing hypothesis.
Fit the deadline to the work — then overpromise and overdeliver.
Teams pad deadlines with "buffer" in the name of under-promise / over-deliver, but the work
just expands to fill the buffer and it quietly becomes under-promise / under-deliver.
I learned I work best the other way: commit to an ambitious date, treat it as do-or-die, and
the work comes through. It cuts against the conventional advice — but it's a hard-earned
preference.
What I'd do differently
1 month → days
eval cycle, once I invested in automation (on a later product)
Invest in tooling sooner
I went too heads-down solving the problem while AI coding tools were rapidly improving.
I should have paused to raise the level of automation — build the tools
that solve the problem. When I finally did that on a later product, autonomous automation
ran large chunks of the eval loop and returned learned lessons to action, compressing the
cycle from a month to a few days.
Make the product condition-agnostic
Fitting tightly to 5 chronic conditions hurt later: adding even one more condition was a
high-lift change across prompts, the knowledge graph, and the evals. I'd identify that
scaling bottleneck early and design to stay scalable along it from the start.