Context & Problem
Support reps jump between several back-office tools (accounts, billing, ticketing and more) for every customer interaction. Calls run long, training is expensive, and the same lookups repeat all day. The goal was one conversation from which a rep can see data, act on it, and finish the interaction.
The hard part is the UI: it has to stream non-deterministic agent output into stable, interactive panels, safely, for several tenants served from one deployment. Agent text varies run to run, yet panels must stay predictable, risky actions must stay gated, and no request may ever reach another tenant's backend.
Design Constraints
- complianceSecurity: enterprise SSO, and no backend credentials stored anywhere. Every layer forwards the signed-in user's token downstream.
- complianceSafety: risky mutations must never be triggerable by typed text or model output.
- platformMulti-tenancy: one build and one server for every tenant, with no chance of cross-tenant routing.
- platformTesting: agent output is non-deterministic, so example-based tests alone are not enough.
Architecture Overview
A supervisor agent routes each request to specialist sub-agents (Python, Strands Agents, FastAPI, on AWS Bedrock); a fast path handles common requests without a full reasoning loop. A tool layer wraps upstream REST APIs, and sub-agents return a typed panel description that the UI renders. The Next.js/React frontend has its own chat layer built directly on the open AG-UI streaming protocol, wrapped in a React-free module, plus a registry-driven panel system (normalizers, per-panel specs, instance IDs, confirmation lifecycles). IDs travel in structured request context, never in model prose. The tenant is resolved per request at runtime, never from build config, and a missing tenant config fails closed. Sign-in uses Cognito with enterprise SSO (Azure AD / SAML) and token passthrough. Datadog APM carries a W3C traceparent from the browser through the UI server and agent to upstream APIs; browser errors go to a privacy-safe, rate-limited endpoint that strips tokens, IDs and query strings. The platform runs on ECS Fargate.
Interactive Architecture
Open full screen ↗Key Engineering Decisions
Outcomes & Lessons Learned
Every tenant
One build and one server serve every tenant, resolved per request.
Stored backend credentials
Each layer forwards the signed-in user's token downstream.
Retrospective
- Confirmation used to be LLM-mediated, and that let typed text stand in for a button press. Safety-critical approval belongs in a structured, code-checked signal.
- Owning the chat layer cost a phased migration, but made streaming bugs fixable at the root: stale "running" state in panels mounted mid-run was fixed with one page-level run tracker instead of per-case patches.
- Fail loudly on tenant config: a fallback default is a cross-tenant incident waiting to happen.
- Property-based tests (fast-check, Hypothesis) suit agent-driven UIs, where outputs vary. Bug fixes start with a failing "bug condition" test plus "preservation" tests for behaviour that must not change.