Stay AI
2024
Doubling product adoption by increasing usability
Our flagship promotions tool sold well, but only about 10% of merchants used it. I redesigned it with design and front-end changes only, and more than doubled the number of merchants running promotions without adding a single feature.
Overview
A usability fix that changed how we measure success
ExperienceEngine was a promotions tool that Sales relied on to close deals, but merchants rarely used it. I put the first number on its adoption and pitched a small, design-only fix to the Director of Product. The project had one front-end engineer and no backend work. That limit made it possible to tie every gain directly to usability.
The Impact
Six months after launch, the number of merchants with promotions went from 121 to 262, and page traffic doubled. The results deck I put together made adoption a KPI for every launch at the company.
Problem
A flagship tool merchants couldn’t figure out
Business problem
ExperienceEngine helped convert sales, yet only 121 merchants, about 10%, had ever used it. That was low for a product that had been in the market for years and that we pushed so hard. No one had measured it before I pulled the number from our analytics.
User problem
Merchants had no sense of where to start or how the pieces fit together. FullStory showed them landing on the page, getting overwhelmed and leaving. Unclear names, unlabeled icons, a confusing input order and missing help text all added up.
The known-issues: confusing name and labels, disconnected icons, inputs disconnected from each other by distance.
Ideation
Even I was having trouble untangling exactly what the tool was meant to do
It took some time to figure out what each control did and in some cases it wasn't what I thought. I worked closely with a Product Manager who explained that what the backend did was different from what the front end spelled out. For example, AI wasn’t really an on or off switch the way it was shown, it was a completely different kind of test. 'Turning on AI' set up the test to give different gifts to users depending on if they were a churn risk or not, whereas setting it up 'manually' meant that you were running a simple A/B test. No wonder merchants were confused!
If certain selections were made in regards to which orders would get the offer, sometimes another panel appeared out of view in the bottom half of the page that would need to be filled out. There was no clear connection between the user's action and those controls and it often lead to frustrated clicks on the save button without users seeing the frontend error messages out of sight.
Different early concepts
Prototyping & User Testing
I built the company’s first functional AI prototype
I used Figma AI to turn the designs into a functional, data-persistent React app. This was the first time anyone at the company had used AI to make a functional prototype.
Testers entered real data and watched it save, so sessions tested actual logic and micro-interactions instead of a simulation's limits, shifting testing from explaining a concept to testing a reality.
This was the company's first functional AI prototype. Because it saved data, merchants tested something that behaved like the real product, which made interviews far more useful and showed engineers exactly what to build. Afterward, I was expected to prototype most major launches, PMs began making their own AI prototypes, and prototyping became how our product team shared and tested ideas.
FigmaMake AI prototype created by me
Merchant Interviews
I arranged interviews with merchants at various program levels and invited our product team to sit in.
During the interviews, merchants were invited to interact with the prototype and give their feedback. After they shared their initial impressions, I asked questions around specific parts of the experience.
The overall response was very positive. Merchants called it clear and organized.
Feedback from these sessions helped the product team to understand that having helper text exposed on the cards outright was more helpful and that this linear workflow did make a huge difference in user perception of the tool.
As a result of user feedback, I also changed some labeling and icons to enhance clarity, but the core strategy held. The step based flow with instructions made the tool less intimidating and more attractive.
“It finally felt like steps. I always knew what to set up next instead of guessing where to start.”
Interview participant
Merchant
Naming Conventions
I also asked merchants about what the name ExperienceEngine meant to them. Their answers showed that the term meant little and told users nothing about what it did. It made the tool sound complicated and they weren't sure what it did until someone at the company explained it to them. Snippets of these recordings convinced the CEO that semantic naming instead of unique, marketing terms would make the platform more intuitive. We renamed it based on my recommendation and the interviews. Then we applied the same rule to two other products with branding-first names. Our naming conventions moved from what sounds cool to what the product does.
Designs
Same capabilities, finally in an order that made sense
I rebuilt the setup as a step-by-step flow that follows how merchants actually plan a promotion: what to test, what to offer, who gets it and for how long.
The redesigned promotion builder, rebuilt as one linear flow with plain-language guidance on every step.
A step-by-step flow
Merchants used to have to zig zag up and down the page to complete the form. Now each section poses one clear and complete question at a time, in the order a merchant thinks about it. Progressive disclosure is used correctly and when user input changes the form, it happens in viewport in close proximity to the user's selection.
Helper text right on the cards
Instead of hiding explanations in tooltips or in an external help doc, I put a short description on each option. Interviews showed this did more than anything else to make the tool feel approachable.
Semantic Labeling
Labels and their inputs form complete sentences instead of fragments, so that the user's selection is crystal clear.
Naming by function, not branding
I shared quotes of merchants who didn't get the name “ExperienceEngine” with our CEO and that convinced her to rename it Promotions. We then applied the same rule to two other products with branding-first names. I also removed the AI controls and made this another test type to select called "Segment Test by Churn Risk", because to the user the fact that the tool uses AI to determine who is at risk of churn matters less than the test's actual structure.
Usability over new features
One request was to target a gift to only churn-risk customers, but building it would have meant rewriting the backend. I held the scope to UX only, so no new features shipped. This caveat had a silver lining, letting me show the direct impact of usability in an apples to apples scenario.
Outcomes
More merchants used it, and kept coming back
User outcomes
Merchants with promotions grew from 121 to 262, and page traffic doubled. First-try task success rose 13 points, from 28% to 41%, and time to convert dropped 12.5%. Merchants returning to check their analytics rose 130%, which shows they were running real tests and coming back for the results.
2x
Merchants with promotions (121 → 262)
2x
Page traffic to Promotions
+13 pts
First-try task success (28% → 41%)
130%
More merchants returning to analytics
Business outcomes
Once something shipped, the team moved on to the next thing. I wanted to know whether this redesign actually worked. So right before launch, I copied our FullStory dashboard for the tool and pointed each metric at the new flow. Then I saved the previous six months of data as a baseline: adoption, visits, time and success on core flows, and frustration signals. I checked in on trends along the way, but I held off on conclusions until I had a full six months to compare, because that would be the stronger test.
At the six-month mark, I put the results into an internal deck. Traffic to the page and the number of merchants creating promotions had doubled, merchants with a live promotion were up 44%, and every core metric had improved, all without a single new feature.
That deck was the first time anyone had measured a launch after it shipped, and it started a bigger conversation.
Our Director of Product made adoption tracking required for every new launch, and our CEO began setting adoption targets. One later project aimed for 90% adoption within 90 days so we could retire the legacy product.
With that momentum, I proposed tracking HEART metrics across all our core products, and I lead that reporting every quarter. Adoption went from something nobody measured to part of how we plan, how we work with engineering, and how we defined success.
