First visits to 21 SaaS websites: what AI personas found
We sent two AI personas to 21 well-known software products and asked each to work out what the product does, what it costs and where to sign up. On a desktop, one got through on every site. On a phone, finding pricing took five times as many actions, and 23 of the 25 problems that held up appeared only on the phone.
Last updated 17 September 2026
Key findings
- Desktop visits went smoothly. The Everyday user persona, on a desktop browser, completed all three steps on 21 of 21 sites.
- Phones were harder. The First-timer persona, on a phone-sized screen, completed all three on 18 of 21.
- Pricing took five times the effort on a phone. Finding the price took a median of 5 actions on a phone (taps, swipes, scrolls) against 1 on a desktop. Understanding the offer and finding sign-up took about the same on both.
- 16 of 21 sites (76%) had at least one problem that held up when checked against the screenshots. 23 of those 25 problems appeared only on the phone.
- The most common problem was the first screen on a phone: on 7 sites it didn't say plainly what the product does or who it's for.
- The personas were right about most things, not everything. Of the 36 problems they raised, 25 held up (10 fully, 15 overstated). 10 were caused by the test setup, and 1 wasn't supported by its screenshots.
Step by step
| Step | Desktop: completed | Phone: completed | Median actions, desktop | Median actions, phone |
|---|---|---|---|---|
| Work out what the product does and who it's for | 21 of 21 | 19 of 21 | 2 | 2 |
| Find what it costs | 21 of 21 | 21 of 21 | 1 | 5 |
| Find where to sign up | 21 of 21 | 20 of 21 | 1 | 1 |
On the phone, two sites counted as not completing the first step because the screenshots never showed who the product was for: what it did was clear. One sign-up step on a phone was never reviewed, and counts as not completed. The phone persona said it would have given up three times: once looking for pricing and twice looking for sign-up. Two of the three involved the test setup: a sign-up form on a separate domain that the test doesn't follow, and a swipe the emulated phone received as a mouse drag (see accuracy).
The problems that held up
Grouped by kind, with the number of sites where at least one held up. Examples are described rather than quoted, so the sites stay anonymous.
| Problem | Sites | Examples |
|---|---|---|
| The first screen doesn't say plainly what the product does or who it's for | 7 | A phone's first screen showing only a slogan and sign-up buttons, with the plain description of the product further down; a feature list of product names with no descriptions. |
| Plan terms, trials or limits are hard to understand | 5 | Prices shown as "$0/mo + compute" with no rate for compute; a free-plan limit in a technical unit with no explanation; a monthly/annual switch that shows the choice only by where the knob sits. |
| A sign-up button leads somewhere other than sign-up | 4 | A sign-up button that opened a blank white page on a phone; a free plan's start button that landed on a sign-in page, with account creation as a small link. |
| Plans are hidden or cut off on a phone | 4 | A plan carousel that opened on a paid plan with the free plan three swipes away; a plan table cut off at the screen edge with no cue to scroll sideways. |
| A chat or panel opens uninvited over the page | 2 | A sales chat that opened by itself over the phone menu, so a tap meant for the menu landed on the chat. |
| Other | 1 | A sign-up page headed as if the visitor's workspace already existed, before any account had been made. |
| The way to pricing or sign-up is hard to find | 1 | Pricing reachable only through an unlabeled menu icon on a phone. |
How accurate the personas were
AI personas are not people, so every problem they raised was checked before it counted. A separate AI reviewer, told to be strict, opened each problem's screenshots and the persona's action log and sorted it into one of four outcomes. A sample was then rechecked in a second pass, which moved one problem from fully supported to overstated. No person reviewed every screenshot.
- Supported: 10
- The screenshots show the problem as described.
- Overstated: 15
- A real problem, but milder than the persona made it sound: a menu that took one extra tap, not one that was hidden.
- Test setup: 10
- Caused by how the test ran, not by the site: keyboard shortcuts that a phone doesn't have, a hover menu driven by a simulated mouse, a sign-up form on a separate domain that the test doesn't follow, and time spent waiting for the AI read as time spent confused.
- Not supported: 1
- The screenshots didn't show it.
So about 69% of what the personas raised pointed at something real, and most of the rest came from how the test ran rather than from the model misreading a page, which points at fixes on our side. This is why a Meerkat report puts the screenshot next to every claim, and why synthetic testing complements research with people rather than replacing it.
Method
- Sites. 25 well-known self-serve software products, five in each of five categories, chosen before any visit and not changed after. 21 were completed. The other four were all in the AI and automation category and were stopped by an outage on our side (our model provider account ran out of credit), not by anything on those sites. Completed sites by category: Design and websites 5, Developer tools 5, Productivity and collaboration 5, Marketing and sales 5, AI and automation 1.
- Personas. Everyday user on a desktop browser (1280×800) and First-timer on a phone-sized screen (390×844, emulated in a desktop browser). One visit each, from each site's home page.
- Task. Work out what the product does and who it's for, find what it costs (including any free plan or trial), and find where to create an account. The scenario: a colleague mentioned it and you have a few minutes to decide whether it's worth trying.
- Boundaries. Personas never typed into a form or submitted one, so no accounts were created. They declined non-essential cookies. Every site's robots.txt allowed its home and pricing pages.
- How a visit runs. The persona sees only the screen and acts through the browser. A reviewing model checks each step against the screenshots. The persona says what it sees, what it will do and what it expects before each action. Visits ran on 17 September 2026 on Meerkat's hosted browsers.
- Actions are taps, clicks, scrolls, swipes, key presses and waits within a step, not counting page loads. Time isn't reported: most of a visit's clock is the AI thinking, not the site responding.
Limits
- Small and polished. 21 well-known sites, each visited once by two personas. Treat the numbers as indications, not rates for the web.
- Not people. AI personas stand in for visitors. They don't carry a real visitor's context, and their opinions are generated.
- Emulated phone. The phone is a small screen in a desktop browser, so some touch gestures behave differently. Several of the test-setup problems above came from this.
- Counted, not weighted. A mild problem and a blocking one count the same in the table.
Data
The anonymised data is free to use under CC BY 4.0: visits.csv (one row per persona per site: each step's outcome and actions, where it would have left, problems raised and held up) and issues.csv (one row per problem raised: kind, device and review outcome). Sites are labelled S01 to S21.
Cite as: Meerkat, "First visits to 21 SaaS websites: what AI personas found", 17 September 2026, runmeerkat.com/research/first-visit-2026. Questions about the method: hello@runmeerkat.com.
Run the same test on your product
This study used Meerkat's ordinary runs. Point it at your own site, pick the personas, and get the same report with every screenshot.