How AI Visibility Tools Actually Work (and Who Powers Them)

In July 2026, Webflow announced that its AEO tools now work “by running each of your tracked prompts through Claude, Gemini, and ChatGPT”. A dozen other platforms make similar claims. Profound, Peec, Otterly, Ahrefs Brand Radar and the Semrush AI Toolkit all promise to tell you how visible your brand is inside AI answers.

We wanted to know how it works, partly from curiosity and partly because we needed the same capability inside our own autonomous SEO agent. So we spent a week investigating: reading methodology pages, pricing pages and privacy policies, cloning the data vendors’ open-source repositories, and doing the per-request maths.

Here is the short version. AI visibility tools send your tracked prompts to AI engines on a schedule, store the full responses, and score each one for brand mentions, competitor mentions, sentiment and cited sources. The interesting questions are where the responses come from, what they cost, and who actually does the collecting. The answers changed how we built our own version.

How the big tools collect their data

The market splits into companies that publish their methodology and companies that say nothing at all.

The incumbents build in-house, and say so

Ahrefs published a full methodology post for Brand Radar. They maintain an index of over 271 million real prompts, sourced from their 110-billion-keyword database and Google’s People Also Ask corpus, and run them at scale: 13.3 million ChatGPT queries a month, 143 million AI Overviews checks, 41 million through Google’s AI Mode. They store the raw responses and mine them for citations and mentions.

Semrush documents a similar approach: a 289-million prompt and response dataset, refreshed weekly, sourced from “AI search clickstream data”. Semrush owns its clickstream supplier, Datos, and it acquired Otterly.ai, whose help docs now live inside the Semrush knowledge base. Consolidation in this market is well underway.

Profound, the best-funded startup in the category, shows every sign of building in-house too. Its Index methodology describes prompt ingestion from panels of “millions of users” (which means purchased clickstream data), its own response logging across engines, and direct CDN integrations that show which AI crawlers visit your site.

For these companies, collection scale is the moat, so they publish the numbers.

The startups say nothing, and that tells you something too

We checked the public record for Peec, AthenaHQ, Scrunch and KIME. None of them discloses how they collect AI responses.

The most interesting document we found was Peec’s privacy policy. As a Berlin-based GmbH subject to GDPR, Peec dutifully lists 22 data processors, from Firebase and PostHog down to its demo-scheduling tool. Not one of them is a scraping provider, a data-collection vendor, or an AI model provider. For a product whose entire value is collecting and analysing AI responses, that is a conspicuous silence about the most important data flow in the business.

We do not know who powers these platforms, and that is the point: in the startup tier, the fetch layer is treated as competitive plumbing rather than a selling point. Our best evidence about who supplies that plumbing comes from the suppliers themselves, and from open-source projects that disclose their dependencies. More on both below.

Three ways to get an answer out of an AI engine

Every tool in this market, and anyone building their own tracking, chooses between three collection methods.

1. Call the model APIs yourself

The cheapest and simplest option. The official OpenAI, Anthropic and Gemini APIs all offer web-search grounding, and a grounded response costs a cent or two. Batch endpoints halve that again: OpenAI, Anthropic and Google all discount asynchronous batch jobs by 50%, and a daily visibility sweep is exactly the kind of workload batching suits.

The catch is fidelity. The API is not the product your customers use. A consumer ChatGPT session carries a system prompt, browsing behaviour and citation formatting that the API does not reproduce. Scrapfly’s technical analysis spells out what the web interface returns that the API omits: citations, source URLs and shopping results. If citations are what you are tracking, that gap matters.

There is also a coverage ceiling. No API, at any price, reaches Google AI Overviews, Google AI Mode, Microsoft Copilot or the consumer ChatGPT experience. Google’s AI surfaces sit on top of actual searches, which arguably makes them the highest-traffic AI answers of all, and you cannot query them officially.

2. Buy official-API responses through an aggregator

Providers like DataForSEO wrap the official model APIs in a single interface. One endpoint returns structured responses from ChatGPT, Claude, Gemini or Perplexity, with optional web-search grounding, for $0.0006 plus the model’s own fee, roughly one to two cents per response. You get one bill, one response format and pay-as-you-go pricing, and DataForSEO also sells a pre-aggregated “LLM mentions” dataset showing which domains and pages the models cite most, which is visibility intelligence you can query without running a single prompt.

Same fidelity caveat as option one, because the underlying responses come from the same APIs. This route is terms-of-service clean, which mattered to us.

3. Scrape the consumer surfaces

A specialist category has emerged to capture what real users actually see, by automating the consumer interfaces themselves. As of August 2026 the main players are:

The per-scrape prices are the story. On cloro’s entry tier, a ChatGPT response with web search costs about $0.004; on higher tiers roughly half that. The maths only tips against pay-as-you-go at volume: below roughly 90 checks a day (about 2,800 a month), paying per response through an official-API aggregator is cheaper than a scraping subscription; above it, the subscription wins. In AIMultiple’s benchmark of 1,000 test runs per provider, Bright Data held a near-100% success rate across engines while Oxylabs was excluded from the ChatGPT test entirely for falling below the 90% success threshold. Scraping chatgpt.com reliably is genuinely hard, which is exactly why it is sold as a service.

Two honest caveats. First, these providers reach consumer surfaces by automating them, which strains the platforms’ terms of service. That risk sits on the vendor’s infrastructure rather than yours, but coverage can break when platforms push back, as Bright Data’s paused Grok scraper shows. Second, the answers are non-deterministic: the same prompt produces different responses run to run, so single data points mean little and trends over repeated samples mean everything.

What the open-source trackers teach

Three open-source projects let you see the whole machine with the covers off, and they settle the build-versus-buy question convincingly.

NameDrop, by Castos founder Craig Hewitt, is a self-hosted tracker whose schema is worth copying outright: Brands, Prompts, Topics, Surfaces, Responses, Mentions and Sources, with cited domains stored as first-class data. It queries ChatGPT via cloro, and Gemini and Perplexity via their official APIs, then uses a cheap model call to classify brands and domains in each response.

OneGlanse takes the maximal-fidelity route: it drives a real anti-fingerprint browser through the actual ChatGPT, Gemini, Claude and AI Overview interfaces. The operational cost is instructive. It needs residential proxies and your own authenticated accounts, and cannot offer a cloud version at all. That burden is precisely what the scraping vendors sell relief from. Its scoring rubric is excellent though: visibility, rank position, sentiment, and a recommendation-type scale running from top pick down to discouraged, which is far more useful than a yes/no mention flag.

Ansvisor covers eight surfaces with a hybrid approach: official APIs where they exist, cloro where they do not, plus a tracking pixel that measures visits arriving from answer engines. That last idea you can steal without installing anything: segment referral traffic from chatgpt.com, perplexity.ai and gemini.google.com in the analytics you already run, and you have an outcome metric to sit alongside visibility.

Notice cloro appearing twice. The only publicly verifiable customers of any scraping vendor in this space are open-source projects, because they are the only ones that have to show their dependencies.

The lesson from all three: a solo developer can build the collection layer in days. Fetching AI answers is a commodity. Deciding what to do about them is not.

How we chose a data provider

We wanted this capability inside Growth Method, where our customers’ agents could use it. That gave us three hard requirements: a mature MCP implementation so agents can call the tools directly, pay-per-use pricing so a team can dip a toe in without a subscription, and a maintained codebase we could trust.

The maintenance question turned out to be the most revealing. Because several vendors publish their MCP servers as open source, you can measure how well they are looked after. As of 10 August 2026:

Reading the source settled two questions no marketing page answered. First, Bright Data’s MCP exposes only three of the six AI surfaces its platform can actually scrape; Gemini, AI Mode and Copilot exist in the product but not in the integration. Second, and decisively for us, every DataForSEO tool declares itself read-only in machine-readable form. That one line of metadata means our agents can call the tools autonomously without stopping to ask a human for approval on every request, which is the difference between an agent that runs overnight and an agent that queues up approval requests.

DataForSEO won on the combination: official-API cleanliness, Claude coverage (which, oddly, almost no consumer-surface scraper offers), a single pay-as-you-go account covering both live queries and the mentions dataset, the healthiest codebase in the field, and pricing that rounds to pennies. Cloro remains the strongest option for the consumer surfaces nobody else reaches over MCP, and we may add it for exactly that.

Our choice does not have to be yours, though, because the agent and the data source are separate decisions. Any team in Growth Method can create an AEO/SEO agent and point it at whichever provider they prefer: connect a data source in seconds with OAuth or an API token, and update your agent’s instructions to use it. Connect DataForSEO like we did, run DataForSEO and cloro in parallel to get official-API answers and consumer surfaces side by side, or plug in an entire suite of providers as the market evolves. The evaluation above is our homework, not a lock-in.

We pay for DataForSEO like any other customer, and nothing in this article is sponsored or affiliate-linked. That goes for everything we publish: see our editorial standards.

What we shipped

The research forced a conclusion on us: monitoring AI visibility is not, by itself, very useful. A dashboard that tells you ChatGPT recommends your competitors is just bad news on a schedule. What you actually want is to improve your AI visibility, and improving it means changing things: covering the subtopics the AI answers include, earning citations on the sources they trust, giving them a direct answer worth quoting.

So we did not build a dashboard. We gave our autonomous SEO agent the ability to see AI answers and act on them. The agent already ran on a schedule, picking its next content opportunity from Google Search Console data. Now it checks the AI answers for its top queries too, and the block we appended to its instructions tells it what to do about what it finds. The abridged version:

Before selecting an opportunity, take the top queries by impressions from Search Console and check each one against ChatGPT, Gemini and Claude with web search enabled. Note whether we are mentioned, which competitors are, and which sources each answer cites. Check which domains and pages LLMs cite most for our topic, and prefer opportunities where the cited sources are ones we can realistically match or displace. When rewriting an article, cover the subtopics the AI answers include that we are missing, and put a direct, quotable answer near the top. Record which queries were checked and what the answers cited, so results can be re-checked after the change ships.

The agent reads the answers, picks the opportunity where a change can plausibly win a mention or a citation, makes the change, and records what it saw as a baseline so a later run can re-ask the same questions and check whether visibility actually moved. Monitoring is the input. The output is a rewritten article, a new comparison table, a citation-worthy answer near the top of a page.

The cost, for a daily run checking its top queries across three models with web search: under $5 a month. The agent’s own reasoning about the answers costs more than fetching them, which rather proves the commodity point.

What this means for your growth team

Three conclusions from the week.

The data layer is a commodity, priced like one. Venture-backed platforms and weekend open-source projects buy AI answers from the same small set of suppliers, at prices between a fraction of a cent and two cents per response. If a vendor’s pitch rests on proprietary access to AI answers, ask harder questions.

API answers and consumer answers are different products. Decide which you need before comparing tools. For content strategy, grounded API responses are usually enough, and they are cheap, stable and terms-of-service clean. For brand monitoring where citations and shopping placements matter, consumer-surface capture earns its premium.

Visibility data is only worth what you change because of it. This market is racing to build prettier dashboards on identical data. The compounding advantage comes from closing the loop: baseline the AI answers for the queries that matter, make a specific change, re-check, and keep what worked. That loop is a campaign discipline, and it is the part no data vendor can sell you. If you are building an AEO practice, the tracking is the easy third of the job.

About Growth Method

Growth Method is the agentic marketing platform for B2B teams. Pre-built AI agents plan, run and analyse campaigns from your live marketing data, with your martech stack connected in minutes. The AI visibility work in this article is that idea in practice: an agent that reads your Search Console data, checks the AI answers that matter for your queries, and turns the gaps into campaigns with a baseline you can re-measure. Get started here.

Frequently asked questions

How do AI visibility tools collect their data?

AI visibility tools send tracked prompts to AI engines on a schedule, store the full responses, then score each response for brand mentions, competitor mentions, sentiment and cited sources. Large incumbents like Ahrefs and Semrush run this collection in-house at enormous scale. Most startups buy the collection layer from specialist data providers such as cloro, Bright Data or DataForSEO, then build their analysis and dashboards on top.

What is the difference between API responses and what ChatGPT users actually see?

The consumer ChatGPT interface returns citations, source links and shopping results that the official OpenAI API does not reproduce. A response fetched through the API also lacks the consumer system prompt and default web browsing behaviour, so it approximates rather than mirrors what a real user sees. That gap is the reason consumer-surface scrapers exist as a product category.

How much does AI visibility tracking cost?

Far less than the tool subscriptions suggest. As of August 2026, a grounded response from ChatGPT, Claude or Gemini costs roughly one to two cents through DataForSEO, and consumer-surface scrapes cost fractions of a cent at volume through providers like cloro or Bright Data. Batch APIs from OpenAI, Anthropic and Google halve model costs again. Our own agent checks its top queries across three models daily for under $5 a month.

Can I build AI visibility tracking myself?

Yes. Open-source projects like NameDrop, OneGlanse and Ansvisor prove a solo developer can build the collection layer in days: a table of tracked prompts, a scheduled job that queries each AI engine, stored responses, and a scoring step. The hard part is not fetching the answers, it is deciding what to change in your marketing based on what they say.

Is scraping ChatGPT or Google AI Overviews allowed?

It sits in a grey area. There is no official API for Google AI Overviews, Microsoft Copilot or the consumer ChatGPT experience, so providers reach those surfaces by automating them, which strains the platforms’ terms of service. Established scraping vendors carry that risk on their own infrastructure, and it is one reason we chose an official-API provider for our own tracking.