Directing UX vision for an agentic AI assistant combining knowledge agents, insight reports, and automated workflows.
*Some content has been redacted due to PII.
01 — Context
02 — Discovery
I started by mapping the long tail of repetitive questions users were routing to support — anything an LLM could plausibly absorb. We pulled six months of transcripts and tagged them by intent, surfacing eight distinct capability areas where an agent could meaningfully reduce time-to-answer.
Alongside the data work, I ran 12 one-on-one sessions across power users, casual users, and internal operators. Two patterns surfaced repeatedly: people wanted the assistant to remember context across turns, and they wanted to know where an answer came from. Trust was the unspoken north star.
03 — Problem & Alignment
Existing chat experiences treated every request as a one-shot prompt — no memory, no source attribution, and no clear difference between "I'm guessing" and "I checked the source of truth." The result was high engagement, low trust, and a long tail of follow-ups that defeated the time-saving premise.
With the AI/ML team and product leadership, we aligned on a multi-agent model: a routing layer in front of eight specialized capabilities (knowledge lookup, reporting, commission insights, digital service tracker, plan recommender, universal census reader, renewals, and quoting). Each agent owned its own evaluation harness and confidence signal.
The shared goal: every response should make the agent's identity, sources, and confidence legible — without making the interface feel cluttered or technical.
04 — Strategy & Product Plan
Focused outcome: a single conversational surface where users feel the assistant is working on their behalf, not just generating text.
Target metrics: ≥80% satisfaction per capability, ≤2 follow-up turns to resolution, source-trust score ≥4/5 in moderated sessions.
Design plan:a consistent agent-response anatomy (scoped action, source chips, confidence band), reusable across all eight capabilities; a shared "what I can do" overview; and a transparent handoff pattern for moments the agent should defer to a human.
Feedback loops: weekly prompt-and-UI critique with the ML team, fortnightly moderated tests with 4–6 participants per round, instrumented post-launch dashboards tracking satisfaction and resolution per capability.
05 — Takeaways & Outcomes
Shipped the multi-agent assistant across all eight capabilities, hitting 90% user satisfaction overall and clearing the per-capability bar in every area. Time-to-answer on the long-tail intents dropped meaningfully, and the source-attribution pattern was adopted by adjacent surfaces.
Lesson: in agentic UX, the design problem isn't generating answers — it's making the system's reasoning visible enough that people are willing to trust it.