Sampled Observation Methodology for AI Visibility

MentionPop exists to make AI visibility evidence auditable. This document describes the sampled observation methodology behind weekly reports—what we run, what we parse, what we exclude, and what we refuse to claim.

Public product summary: methodology page. Operational readout guide: how to read a weekly GEO report.

Design goals

AI answers and search surfaces are non-deterministic and can vary by wording, device, time, and location. MentionPop records observed evidence; it does not promise universal ranking truth.

Observation pipeline

┌──────────────────┐
│ Versioned prompt │
│ library          │
└────────┬─────────┘
         ▼
┌──────────────────┐
│ Scheduled        │
│ provider samples │
└────────┬─────────┘
         ▼
┌──────────────────┐
│ Response capture │
│ + run metadata   │
└────────┬─────────┘
         ▼
┌──────────────────┐
│ Parse mentions   │
│ Parse citations  │
└────────┬─────────┘
         ▼
┌──────────────────┐
│ Apply exclusion  │
│ rules (failures) │
└────────┬─────────┘
         ▼
┌──────────────────┐
│ Weekly report +  │
│ deltas           │
└──────────────────┘

Diagram: /resources/evidence-pipeline-diagram.svg.

Provider scope

Perplexity Sonar supports internal cross-checks—it is not marketed as a default weekly GEO provider.

Prompt library rules

Prompts are fixed between version changes so deltas are interpretable:

• Branded prompts — test disambiguation and direct awareness
• Category prompts — test competitive narrative without brand injection
• Geo-modified prompts (when configured) — local visibility with explicit city strings

Prompt additions/removals are metadata events—compare reports only across compatible versions.

Parsing definitions

Mention

A mention occurs when configured brand strings (and approved aliases) appear in parsed answer prose after normalization rules.

False positives are mitigated via entity disambiguation copy and conservative alias lists.

Citation

A citation occurs when a returned source list or grounding metadata includes a URL whose registrable domain matches the customer domain configuration, after canonicalization (strip tracking params, normalize host).

Mentions and citations are never merged—see citations vs mentions.

Exclusion policy

This policy prevents infrastructure failures from looking like visibility crashes—a common failure mode in immature GEO tools.

Variability acknowledgment

Provider-sampled outputs vary run-to-run. Methodology response:

• Prefer trend lines over single-run hero numbers
• Document AI output variability in customer education
• Avoid causation language when content ships coincide with noisy weeks

Crawler access complement

Sampled observations do not replace technical fetch baselines. If bots cannot retrieve /methodology/ or guides, models may rely on third-party descriptions.

First-party crawler baseline (research): n=4 agents allowed on mentionpop.com with HTTP 200 on public crawler-control files as of 2026-07-30—access only, not mention proof.

Explicit non-goals

MentionPop is not:

• Social listening or share-of-voice across networks
• A promise to optimize or guarantee mentions
• Exact consumer UI replication for every user session
• A substitute for SEO rank tracking

Measurement vs optimization

Keep slides separate. Evidence-first measurement culture depends on this boundary.

Related guides

• What is AI visibility?
• What is GEO?
• Technical crawler access

Change log policy

When MentionPop adjusts methodology—parser rules, provider scope, exclusion handling—we will:

1. Publish an updatedAt date on this research note
2. Summarize what changed in plain language
3. Avoid retroactive re-scoring of historical weeks without version flags

Customers should not discover breaking definitional changes via silent dashboard shifts.

Security and customer data boundaries

Sampled observation runs use customer-configured brand strings and domains—never authenticated app content. MentionPop is not social listening; prompts simulate public buyer questions, not private support transcripts.

Engineering teams should keep /dashboard/, /account/, and auth routes disallowed for AI crawlers even when marketing allows /.

QA sampling (internal)

Internal QA periodically hand-checks automated parses against raw captures for:

• False mention matches on homonyms
• Missed citations with unusual URL formatting
• Competitor co-mention tagging

QA findings become parser tickets, not silent hotfixes—definition changes belong in changelogs.

Mapping methodology to product tiers

Sales and support should cite this table when prospects ask "does MentionPop track everything?"

Reading list for implementers

• Evidence-first GEO/AEO measurement
• Citations vs mentions
• AI crawler baseline
• Cloudflare rollout case study
• Public methodology page

Methodology documents age when providers change APIs. This note will be updated with dated changelogs—treat versioning as part of the product, not an embarrassment.