From managed service to customer self-serve: the intuitive AI chat that builds, tests, and improves voice agent
Responsibilities
Design lead for the Open Platform, and end-to-end designer on Agent Builder: the interaction model, onboarding, the build → plan → test flow, the visual language, in-product guidance, and the launch analytics: Pendo dashboards, instrumentation, trial emails, and multi-source data validation.
Client
PolyAI
Year
2026
Info
PolyAI’s platform is powerful, but it was built for people who were technical enough to understand it, and we’d just opened it up to people who weren’t. Agent Builder is a chat that lives inside the platform: you tell it what you want in plain language, and it builds the agent, tests it, and shows you exactly what it changed. I led design for the Open Platform, with Agent Builder as its biggest feature.
The Challenge
Before Agent Builder we had the Agent Wizard, an onboarding flow I’d designed that spun up a working agent in a few minutes. It worked, but it stopped at creation. The moment you landed in the real platform, the hand-holding vanished and you were staring at a tool built for people who already knew flows, tools, entities, and knowledge bases.
That’s where adoption broke. Even our enterprise customers, back when the platform was closed, rarely changed anything themselves; they were scared of breaking a live agent, so they asked us to do it instead. And the fear is fair: one wrong change is a missed booking or a dropped call with a real customer.
So the real brief wasn’t “add a chatbot.” It was: take a genuinely complex product and make the interaction with it simple, intuitive, and trustworthy enough that a non-technical person would change their own live agent, and believe the result.
Designing the Whole
Flow
The chat was the easy part. The hard part was the journey around it: how you arrive, how it figures out what you actually want, how it proves a change worked, and what it does when something goes wrong.
Two things sat under every decision: people had to trust it enough to let it change a live agent, and someone who’d never opened the platform had to be able to use it easily. Most of these design patterns exist somewhere already. My job was deciding what to borrow, what to rethink, and pulling it all into one flow built around those two ideas.
Trust
People won’t let an agent touch a live system if they can’t see what it’s doing, so I kept them in control at every step. When you ask for a big change, it doesn’t just make it; it shows you the plan first as well as its thinking steps. You approve or adjust it, and nothing reaches your agent until you say so.
Every change it makes is then listed and traceable back to the exact place it lives in the platform. Before you go live, you can ask it to test the work and decide what to do with what it finds. The same thinking runs through the harder moments too: rejecting a plan, an error mid-build, or walking away while it’s still working.
The plan card shows exactly what it’ll change before it touches anything. You approve or adjust; nothing happens until you do.
Every change is listed and clickable, opening the platform page where it lives with the section highlighted. Nothing it did is hidden.
Ask it to test a flow and it explains any result in plain language, say, a reschedule that books a new slot without cancelling the old one. It can suggest a fix, apply it, and re-run the tests until they pass.
Hiding the Complexity
The other half of the job was making the platform usable by someone who’d never seen it. All the depth stays for people who want it; a first-timer just shouldn’t have to meet it.
New users are walked through their first agent in plain steps. It asks for what it needs in tappable options instead of platform jargon, shows you the plan, and gets you talking to a working agent before you customize anything. Returning users start from a few clear intents instead of a blank input. You only ever deal with the part you actually need.
First-timers don’t land on an empty input. They get a screen that explains what Agent Builder does, with two choices: “Build my first agent” or “Skip.” Skip and you’re in the normal free-form view; build and it walks you through your first agent step by step.
When a request is vague, it asks in tappable options, so someone who doesn’t know the platform’s vocabulary still knows how to answer.
It opens as a side panel on any page and knows where you are, so help shows up where the work already is. (Side panel designed, rolling out after v1.)
One of the clearest audiences turned out to be PolyAI’s own sales team. Account executives build a personalized demo right after a call and send it to the prospect. Sales engineers take a thin demo and make it production-ready after discovery. Account managers use it to let existing customers change their own agents.
Because an agent is shareable over a link, an evaluator can demo it internally before procurement even starts, usually where these deals stall. Demos that took hours now take minutes, and a few customers told us the simplicity of building with it is why they chose us. It moved us off a managed-service model, where every change came through PolyAI, toward something customers can drive themselves.
After Launch
After launch I owned the analytics with our PM (built pendo dashboard to track key metrics as well as came up with a qualitative questionnaire inside the platform).
Two days in, the Pendo funnel said almost nobody was converting, while a raw query showed dozens of people mid-conversation with the builder right then. The funnel was lying: the events that fire when someone starts the builder weren’t firing on every entry path, so we were measuring our own tracking gaps, not real behavior. I traced the cause with engineering and data, then validated every number across three sources instead of one. Within two weeks they agreed, and leadership had numbers they could trust.
The session recordings told the rest. We’d read the top-of-funnel drop as a mobile problem and shipped a responsive fix, but the numbers didn’t recover the way that should. The real issue was onboarding: people landing on the wrong page, a guide repeating something they’d just skipped, and the word “agent” meaning two different things in two places. One person skipped the builder entirely because she thought she’d already made her agent. That’s what we’re redesigning now.
