Context & Problem
The client needed to reduce the time-to-resolution for highly complex customer operations tickets that required reasoning across multiple disparate backend systems (billing, CRM, provisioning). A multi-agent LLM approach was chosen to handle the logic, but the human-in-the-loop operators required complete transparency into how the agents arrived at their conclusions.
Building a complex multi-agent system requires more than just prompting; it requires a bulletproof frontend that can parse, stream, and render non-deterministic agent event streams in real-time without overwhelming the operator. Furthermore, we needed a robust observability pipeline to trace multi-agent handoffs and track token costs across thousands of invocations.
Design Constraints
- complianceStrict data privacy requirements necessitated heavy PII redaction before logging any LLM inputs or outputs.
- teamCross-functional team of 12 engineers; frontend needed to integrate seamlessly with the Python-based agent orchestration layer.
Architecture Overview
We implemented a React/Next.js frontend that consumed Server-Sent Events (SSE) from the orchestration layer. The frontend dynamically rendered agent 'thoughts', tool invocations, and sub-agent handoffs using a state-machine-driven UI. For observability, we integrated OpenTelemetry and a custom logging pipeline that tracked token usage, tool latency, and agent logic forks.
Interactive Architecture
Open full screen ↗Key Engineering Decisions
Outcomes & Lessons Learned
Traceability
Achieved full request-level tracing for all agent invocations, enabling operators to audit any automated decision.
UI Latency
Optimized SSE parsing and rendering to ensure agent 'thoughts' streamed to the UI with minimal perceived latency.
Retrospective
- Formal state machines (XState) are invaluable when building UIs that interact with non-deterministic LLM agent streams.
- Observability must be built into the agent framework from day one; retrofitting traces onto a complex multi-agent system is nearly impossible.