The LLM Measurement Guide: Metrics, Methods & Tools

Despite all the noise in the SEO industry, LLM visibility/traffic is a reality and something we need to study and consider.

The question that matters is whether this data can actually improve your LLM positioning.

The answer is yes but there are a lot of caveats and details to consider to understand how AI systems affect your website.

Before we talk about measurement, we need to understand a little bit more of GEO (or how you call it).

How LLMs Work & What They (Actually) Do

LLMs have non-deterministic output meaning that you can’t reproduce it accurately or expect a fixed output.

Google results are affected by multiple factors but the retrieval process behind them is deterministic.

So this difference is actually true and why many advise against the entire concept of LLM Visibility.

LLMs generate words based on probability distributions learned from training data.

They don’t have a database they query BUT they can be supplemented with external searches (e.g. searching the web).

The key takeaway is that what they produce isn’t predictable and/or repeatable.

The crawler that fetches your page doesn’t run your scripts, so anything rendered on the client side never reaches it.

GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and Meta-ExternalAgent all read raw HTML.

Structured data is not extracted on a direct fetch either, wherever you put it.

In a 14-page canary test, JSON-LD never survived the head, the body or the footer.

Google’s own crawler renders, which is what makes AI Overviews and AI Mode the exception.

LLMs may rely on grounding, meaning that they retrieve additional information from external sources like the Web:

This doesn’t alter the base process of generation, though. So grounding isn’t a surefire fix for hallucinations!

On the contrary, a search engine like Google is based on good old information retrieval.

But if you are outside of the SEO bubble and think about the business, you would know that this doesn’t matter.

Say a person finds your brand via the output of an LLM… wouldn’t you say this interaction is business relevant?

If you answered no, it’s time to rethink your job priorities.

The Studies & Why They Can’t Settle It

Everyone claims LLM traffic converts better but the data is a mess and there are many factors to consider.

The Conversion Numbers

Amsive ran a proper statistical analysis on 54 sites and found no significant difference: 2.17% for LLM traffic against 1.16% organic in B2B, 6.58% against 6.78% in B2C, with p above 0.4 either way.

A University of Hamburg study of 973 ecommerce sites with $20B revenue found ChatGPT traffic converting WORSE than organic search. Only Paid Social performed worse.

Don’t alter your strategy (if you have one) because LLMs now get used.

Attribution Problems With LLMs

Just because LLMs converted doesn’t mean they are a superior channel.

Obsessing over the last touch is the quickest way to invest in the wrong marketing activities.

What about the touchpoints before it?

Mind you, the same applies when LLMs are NOT the last touchpoint.

Take it more as a philosophy and remember that you can’t evaluate a channel or source based on what converts.

Small samples are the other classic and that’s why I don’t trust most of these studies.

250-300 prompts is nothing to draw conclusions from, while even a huge sample misleads when the observations were picked badly.

Most sites do not have enough LLM traffic to compare in the first place.

SimilarWeb tracked what happens after a ChatGPT recommendation.

Brands that got recommended were 2.5x more likely to get a site visit within 7 days, with 55.9% of that traffic arriving as branded search instead of an AI referral.

The AI did the persuading, Google got the attribution.

Mentions, citations and branded search trends are where the signal actually lives.

Why Your AI Referral Numbers Are Wrong

Most of the traffic on your website is not coming from LLMs.

A 13-month dataset from Search Engine Land shows LLM referrals at 0.15% to 1.5% of total referral traffic, converting at roughly 18%, the highest of any channel, on about 25 times less traffic than SEO or Direct.

My personal tests show the same picture, LLM traffic is still a rounding error for most sites

I can’t exactly say the same in terms of conversion rate.

Organic traffic hasn’t collapsed, which is the part the LLM-absorbs-everything argument keeps skipping.

Plus, GA4’s default channel groups now support “AI Assistant” as a value.

As of mid-2026 its documentation recognizes 5 sources: ChatGPT, Gemini, DeepSeek, Copilot and Grok.

Claude and Perplexity still land in Referral.

The channel only fires when a referrer survives so it didn’t add anything worthy from a measurement POV, if not simplicity.

Google’s own AI Overviews and AI Mode are excluded by design: they carry the same google.com referrer as a normal search and count as Organic Search.

The 3 Buckets Of LLM Traffic

The channel classification doesn’t help that much and this is why the following still happens today:

Clickport measured this across 371,847 sessions and found 35.7% of AI traffic arrived with no referrer at all.

4 things strip the referrer: your site’s referrer policy, noreferrer on paid links, the mobile apps’ built-in browsers and copy-paste.

Only the first still tells you the visit came from chatgpt.com.

The App Blind Spot

Most of the invisible traffic comes from apps.

The ChatGPT apps on iOS and Android, plus the desktop app, open links through a system browser or an in-app webview.

The referrer rarely survives that handoff, so GA4 sees a session with no source and files it under Direct.

This is one reason your measured AI traffic looks 90% desktop: the sessions that survive are the browser ones.

Consent Makes It Worse

Rejected visits never reach GA4 at all with basic consent mode.

Advanced Consent mode allows for cookieless pings, you get the event but not the user or session id.

Run the math on 100 ChatGPT visits and roughly 1 in 6 ends up fully attributed.

What You Can Actually Measure

There isn’t much you can reliably measure due to how LLMs and the Web work.

This doesn’t mean you shouldn’t measure anything and let it go. GEO (or whatever) has an effect on your business.

Here is the framework I use:

First, measure if you appear in LLMs at all, through citations and mentions.

Second, estimate the contribution of AI to your conversions. Third, estimate the incremental change in revenue, which is the hardest part.

Attribution is questionable and the perfect model doesn’t exist, for me it’s like a system of beliefs, nothing more.

Treat this as a philosophy that drives decisions, not as a report.

Most importantly, your stakeholders don’t wanna listen to the nth excuse about web tracking.

This time they are right! Saying the results are inaccurate is different from not having data to show.

As an example, I don’t look at the exact numbers and still do fine:

This is the reality of Web Analytics:

• Consent Mode eats a good chunk of your traffic
• Most implementations are wrong
• Client-side tagging is unreliable (adblockers, browser restrictions)
• GSC data is heavily anonymized the bigger your website is
• Direct traffic is mostly misattribution (dark social, improper tracking)

Can’t demand perfection from LLM metrics when you accept 40% data loss in GA4.

Visibility VS Clicks

Forget about clicks only, you should also take visibility (or the awareness) seriously.

In B2B, the concept of being in the consideration set of people is quite important.

Do you think companies spend $50K/year to buy a SaaS without knowing your brand? Think twice.

Content per se doesn’t need to sell, it should facilitate that outcome, though.

Most of these concepts are old and foundational to marketing as a discipline.

Unfortunately, this wasn’t that obvious in SEO and we still face its consequences today.

1. Consented AI Traffic coming from LLMs

You can actually track some of the LLM traffic:

• ChatGPT (chatgpt.com referrals)
• Perplexity (perplexity.ai referrals)
• Claude (claude.ai referrals)
• Gemini (gemini.google.com referrals)
• Microsoft Copilot (copilot.microsoft.com, bing.com/chat)
• Grok (grok.com referrals)
• DeepSeek (chat.deepseek.com referrals)
• Meta AI (meta.ai referrals)

As previously mentioned, you won’t always get a referrer but a link of your website + UTM parameters with a specific LLM source:

This is already measurable and it works (partially).

Explorations (GA4)

If you still haven’t made the switch to BigQuery and have a proper modern data architecture, here’s how to do it.

Create a segment with a regex matching those referral sources, then compare sessions and conversions by source in an Exploration over the last 14 months.

Remember that this only makes sense if you want to analyze 14 months at most AND if you increased the data retention before.

And if you want, you can also add a custom channel group or even a segment for easier filtering.

BigQuery Code

If you have a proper modern data architecture and enable the BigQuery export from GA4, this is the code you can use:

WITH session_starts AS (
  SELECT
    CONCAT(
      user_pseudo_id, '-',
      CAST((SELECT value.int_value FROM UNNEST(event_params) WHERE key = 'ga_session_id') AS STRING)
    ) AS session_key,
    (SELECT value.string_value FROM UNNEST(event_params) WHERE key = 'page_referrer') AS referrer,
    privacy_info.analytics_storage AS analytics_storage
  FROM `project.analytics_123456789.events_*`
  WHERE _TABLE_SUFFIX BETWEEN '20260101' AND '20261231'
    AND event_name = 'session_start'
)
SELECT
  CASE
    WHEN referrer IS NULL OR referrer = '' THEN 'No referrer at all'
    WHEN REGEXP_CONTAINS(referrer, r'(?i)chatgpt\.com|chat\.openai\.com') THEN 'ChatGPT'
    WHEN REGEXP_CONTAINS(referrer, r'(?i)perplexity\.ai') THEN 'Perplexity'
    WHEN REGEXP_CONTAINS(referrer, r'(?i)claude\.ai') THEN 'Claude'
    WHEN REGEXP_CONTAINS(referrer, r'(?i)gemini\.google\.com') THEN 'Gemini'
    WHEN REGEXP_CONTAINS(referrer, r'(?i)copilot\.microsoft\.com') THEN 'Copilot'
    WHEN REGEXP_CONTAINS(referrer, r'(?i)grok\.com|x\.ai') THEN 'Grok'
    WHEN REGEXP_CONTAINS(referrer, r'(?i)deepseek\.com') THEN 'DeepSeek'
    WHEN REGEXP_CONTAINS(referrer, r'(?i)meta\.ai') THEN 'Meta AI'
    ELSE 'Other referrer'
  END AS llm_source,
  COUNT(DISTINCT session_key) AS sessions
FROM session_starts
WHERE analytics_storage IS NULL OR analytics_storage = 'Yes'
GROUP BY 1
ORDER BY sessions DESC

Adjust it accordingly depending on how you modeled the GA4 data and on consent.

As usual, you want to understand the behavior in relationship to the following variables:

• By page (which content gets LLM traffic?)
• By device (desktop vs mobile – expect 90%+ desktop)
• Over time (is it growing?)

Then, you can manually create your own channels, segments as usual.

2. Google AI Overviews/Mode Traffic (Not Really?) + Copilot

Google Search Console first introduced AI Performance in June 2026 as a beta feature, with an opt-out toggle for sites that don’t want to appear in AI responses.

Before, AI data was only mixed into GSC under the “Web” search type.

I mean, it still is but now we can get some very basic information in a separate section at least:

What Google says:

• “AI Mode clicks count toward totals in Search Console Performance report”
• “Clicks tend to be higher quality”
• “Users are more likely to spend more time on site”

We know the last 2 statements are completely false based on our personal experience.

However, you can still test if there was an effect on your website by running code like this:

-- Performance before and after AI Mode rolled out
SELECT
  CASE WHEN date < '2026-06-03' THEN 'before' ELSE 'after' END AS period,
  SUM(clicks) AS clicks,
  SUM(impressions) AS impressions,
  SAFE_DIVIDE(SUM(clicks), SUM(impressions)) AS ctr
FROM `project.searchdata.searchdata_site_impression`
WHERE date BETWEEN '2026-03-01' AND '2026-08-31'
GROUP BY period

Such an analysis is IMPOSSIBLE with GA4 and GSC alone unless you rely on the APIs or BigQuery exports.

Bing Webmaster Tools (BWT) on the other hand was the first mover in terms of measuring AI Performance, as I showed before when comparing BWT to GSC.

Microsoft gives you better metrics and more context compared to Google!

In fact, you can see grounding queries used by Microsoft LLMs and citations.

Log Files

LLMs need bots to crawl the web and hunt for information, otherwise they wouldn’t have reliable information.

You can inspect your log files to see if there is bot activity inherent to LLMs.

Spoiler: most likely YES.

Then, it’s easy to analyze which pages or paths are the most common.

Have common sense though, bots visiting a page doesn’t mean that page is more important or will be featured by LLMs.

This wouldn’t justify cases where llms.txt files are crawled but no tangible LLM improvement happens.

Prompts Detection

There is a way to detect some potential prompts and have an idea of what people actually type.

Based on logic and human syntax, it’s highly abnormal that a 30-word query is written inside Google, don’t you think?

P.S. Please don’t get the definition of long-tail wrong. A long-tail query is NOT defined by its number of words.

The volume defines what’s long tail and what is not. All the other details are correlations and fruit of SEO misinformation.

You could also use BigQuery to detect whether there was a change in the median length of your queries over time:

-- Median query length by month (GSC export in BigQuery)
SELECT
  FORMAT_DATE('%Y-%m', date) AS month,
  APPROX_QUANTILES(LENGTH(query), 100)[OFFSET(50)] AS median_query_length
FROM `project.searchdata.searchdata_site_impression`
WHERE date >= '2026-01-01'
GROUP BY month
ORDER BY month

I’ve been doing prompt tracking and despite the results changing, there are some commonalities.

You can also look for queries that mention your site or brand, that’s a pattern worth tracking on its own.

Long conversational strings and brand-plus-topic combinations are the shapes that show up when people ask an assistant instead of typing in a search box.

Brand Mentions & Citations

Manually or programmatically testing if your brand appears in LLM outputs.

A mention is when your brand gets mentioned in a response.

A citation is a source used in a response.

The issue with this approach is that the output is extremely volatile and unreliable.

Also keep in mind that a mention can happen without a citation being involved:

A typical case: auto.com gets the citation as the source, while Ferrari is just the mentioned brand.

Such is the case of 3rd-party websites aggregating results and mentioning alternatives.

The most useful comparison comes from Bing Webmaster Tools.

Its Citation Share field compares your citations against competitors, so you can gauge how prevalent your brand is.

The “New” Metrics

Let’s start by saying that SEO KPIs don’t exist.

A KPI (key performance indicator) points out what matters for your business.

For the vast majority of businesses, Marketing is one activity but not the only driver.

SEO happens to be a part of it, not the only driver.

An example of KPIs could be:

• Conversions
• MQLs
• Forms
• Downloads
• Revenue/Net Profit

Out of all the KPIs, one can be selected as a North Star, your most important indicator of success.

Once again, not every metric qualifies to be a KPI.

You may have read Sci-Fi takes like introducing new incredible metrics from patents involving chunking, retrieval, etc.

None of this is relevant for a business, it’s completely unrelated to how businesses operate.

It’s SEO fiction.

There are some nice metrics that we saw before for strict GEO purposes.

• Share of Voice / Search
• Mentions
• Citations

None of them is a KPI but they are interesting metrics.

The Measurement Loop

Measurement here is a loop, not a dashboard. This is the cadence I could recommend to a business.

• Weekly: scan the source report for domains you don’t recognize. Watch Direct and Unassigned for movement.
• Weekly: check GSC AI Performance impressions on the pages you care about. Rerun your citation checks for the same period.
• Monthly: run the BigQuery queries above, the prompt-length median plus a log analysis for new crawlers.
• Quarterly: recheck the assistant list. It changes faster than any other channel you measure.

Incrementality Is Tricky

You will hear about incrementality as the real way to measure AI impact.

The idea is to compare a page or group that received changes against one that didn’t.

In practice this is quite challenging. Organic search can’t be switched off the way paid can, unless you noindex the whole site, which defeats the point.

And you can’t run holdouts with LLMs.

You can’t (easily) instruct a model to ignore your brand for a week to measure the difference.

The Tools

During a gold rush, it’s normal for everyone to enter the hype market and sell shovels.

However, I have to say that traditional tools don’t capture LLMs as they are quite different from search engines.

As of now, there are many players:

• Waikay
• Profound
• Peec AI
• Searchable
• Semrush
• Ahrefs
• Adobe

If you ask me, the best advice is to try them out by yourself.

Testing is cheap. Outside Anthropic and OpenAI, LLM prices are peanuts, while OpenRouter lets you compare models side by side before you commit.

As usual, a good chunk of what they offer you can do it yourself with some decent developers.

Big companies have more interest in LLM traffic, especially if they invest in PR.

Small businesses should also have an interest in AI because visibility there is still cheap to win, unlike organic search.

The Discrepancies Between Tools

I tried comparing several tools but the output rarely matched.

That’s because they use different criteria and LLM outputs are not deterministic, as said before.

API VS UI

If you have played with LLMs, you would know that LLM APIs give you different output compared to you going to the website.

This isn’t just because the output is non-deterministic or “personal”, I suspect it’s because those are lab measurements.

This was also confirmed by Surfer’s study comparing results from the API and the UI.

Surfer put a number on it: only 8% of the sources overlap between API and UI results. Waikay also explained the problem in this article about result discrepancies in LLMs.

Simulating an actual browser session is feasible with DIY tools but it’s not as simple as you imagine.

OK… And So What Now?

Hard times created the need of omnichannel marketing.

It always existed but not many SEOs actually did it and it shows.

It’s more of a myth, something you hear at conferences but no one actually does.

If you have been doing omnichannel marketing, you are already safe. What’s written in this section will be a reassuring summary.

For all the other people, this is what they missed:

• good content that is unique and adds something to the industry
• distributing and repurposing content across multiple channels
• build a community
• having a newsletter for nurturing and converting

Obvious, right? Yet, how many businesses actually follow through and do it properly? You can count them on your fingers.

At Seotistics, I do all of it because it’s the only way to thrive as a “small” website compared to goliaths.