CompetLab MCP Tools Reference
How to read this reference
A few things hold across every tool:
- Authentication isn’t a parameter. Your API key travels on the request (a
CL-API-Keyheader, or?api_key=on the URL) — see Connect. A tool sees exactly what your key can see over the REST API. - Most tools are project-scoped. They need a
projectId, which you get fromlist_projects. IDs are 24-character hex strings. - Read vs. state-changing. 32 of the 35 tools only read. The three exceptions —
start_tech_stack_scan,start_trust_signals_scan, andstart_agent_adoption_scan— start a live scan and return ascanIdyou poll. - Rate limits. The free-tool routes are limited per API key: most at 1,000 requests
per minute, with
fetch_urlheld tighter at 60 per minute. - Errors. A missing or malformed key is rejected by the server with an HTTP 401
(
api_key_missing/api_key_invalid) before any tool runs. Once a call reaches a tool, failures come back as an MCP tool error (isError: true) with a machine-readable code — for examplescan_not_foundorapi_unreachable— rather than throwing. One worth knowing: aget_<dimension>_run_detailcall for a run that finished but produced no summary answersrun_not_summarized, which is different fromrun_not_found— the run exists, it just has nothing to report.
null means we did not measure it — never zero, never empty, never “no”. A measured 0,
false, or empty list is reported as itself and is a real finding.
This is the most important rule on this page. hasFreePlan: false means we read the pricing
page and there is no free plan; hasFreePlan: null means we could not read the page at all.
advantages: [] means we compared and you lead in none; advantages: null means no
comparison happened. A tool would rather return nothing than return something it never
checked.
Treat a null as “not measured” and say so — or say nothing. Never render it as 0, and
never let it reach a user as a finding about a competitor. Plot a null rate as a break in the
line, not a zero.
Shared parameters
| Parameter | Type | Notes |
|---|---|---|
projectId | string | A project’s 24-character hex ID, from list_projects. Required by every project-scoped tool. |
page | integer | 1-indexed page number. Default 1. |
limit | integer | Items per page. Default 20, max 100. Check pagination.hasMore to page. |
The tools at a glance
| Tool | Area | Read-only | What it does |
|---|---|---|---|
list_projects | Projects | ✔ | List accessible projects and their status. |
get_project | Projects | ✔ | One project’s per-dimension freshness and prompts. |
list_competitors | Competitors | ✔ | Competitors in a project (incl. your own domain). |
get_competitor | Competitors | ✔ | One competitor’s detail and monitored pages. |
get_ai_visibility_dashboard | AI Visibility | ✔ | Latest AI Visibility Score and per-engine breakdown. |
get_ai_visibility_history | AI Visibility | ✔ | Paginated AI Visibility check history. |
get_ai_visibility_check_detail | AI Visibility | ✔ | Full detail for one AI Visibility check. |
get_ai_visibility_trend | AI Visibility | ✔ | AI Visibility trend over time. |
get_positioning_dashboard | Positioning | ✔ | Latest homepage-messaging analysis. |
get_positioning_history | Positioning | ✔ | Paginated Positioning run history. |
get_positioning_run_detail | Positioning | ✔ | Full data for one Positioning run. |
get_pricing_dashboard | Pricing | ✔ | Latest structured pricing and gap analysis. |
get_pricing_history | Pricing | ✔ | Paginated Pricing run history. |
get_pricing_run_detail | Pricing | ✔ | Full data for one Pricing run. |
get_content_dashboard | Content | ✔ | Latest content categorization and gap analysis. |
get_content_history | Content | ✔ | Paginated Content run history. |
get_content_run_detail | Content | ✔ | Full data for one Content run. |
get_content_changelog | Content | ✔ | Detected content changes per competitor over time. |
get_tech_trust_dashboard | Tech & Trust | ✔ | Latest security, trust, and tech-stack profile. |
get_tech_trust_history | Tech & Trust | ✔ | Paginated Tech & Trust run history. |
get_tech_trust_run_detail | Tech & Trust | ✔ | Full data for one Tech & Trust run. |
list_alerts | Alerts | ✔ | Competitive alerts across dimensions. |
list_schedules | Schedules | ✔ | Monitoring schedules for the five dimensions. |
get_briefing | Strategic Briefing | ✔ | The synthesized Strategic Briefing. |
get_briefing_history | Strategic Briefing | ✔ | Past briefing editions, newest first. |
get_briefing_edition | Strategic Briefing | ✔ | One past edition in full, by runId. |
check_sitemap | Free Tools | ✔ | Live sitemap analysis for any domain. |
check_ai_crawlers | Free Tools | ✔ | Which AI assistants can fetch any domain’s pages. |
fetch_url | Free Tools | ✔ | Fetch and clean any public URL. |
start_tech_stack_scan | Free Tools | — | Start an async tech-stack scan. |
get_tech_stack_scan | Free Tools | ✔ | Poll a tech-stack scan. |
start_trust_signals_scan | Free Tools | — | Start an async trust-signals scan. |
get_trust_signals_scan | Free Tools | ✔ | Poll a trust-signals scan. |
start_agent_adoption_scan | Free Tools | — | Start an async Agent Adoption Check. |
get_agent_adoption_scan | Free Tools | ✔ | Poll an Agent Adoption Check. |
Projects
list_projects
Lists the projects your key can access, with status, competitor count, and last-monitored
time. This is the starting point — it’s how you discover the projectId values the other
tools need. It takes no parameters.
{ "name": "list_projects", "arguments": {} }Returns the accessible projects, each with status, competitor count, and last-monitored timestamp.
get_project
Returns one project’s details, including per-dimension monitoring freshness (Tech & Trust, Content, Positioning, Pricing, AI Visibility), the AI monitoring prompts, and overall status. Use it to see when each dimension was last updated.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
Competitors
list_competitors
Lists every competitor monitored for a project, including your own domain (marked
isOwn: true for self-comparison). Returns domain, name, and status for each.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
get_competitor
Returns one competitor’s detail, including the pages CompetLab monitors (homepage and pricing-page URLs).
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
competitorId | string | Yes | Competitor ID, from list_competitors. |
AI Visibility
The AI Visibility tools use a checkId rather than a runId: each check asks the
project’s 3 AI prompts against all three AI engines (9 queries).
get_ai_visibility_dashboard
Returns the latest AI Visibility Score (a 0–100 composite), Mention Rate, per-engine breakdowns for ChatGPT (OpenAI), Claude, and Gemini, and how your competitors compare.
Set includeAnswers to also get what the models actually said — read
The answers block before you do.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
includeAnswers | boolean | No | Default false. Also return the models’ raw answers. Large payload — see The answers block. |
provider | string | No | One of openai, claude, gemini. Return only that model’s answers. Requires includeAnswers=true. |
brand | string | No | Return only the entries for this domain, across every answer. Requires includeAnswers=true. |
promptIndex | integer | No | Return only the answers for this prompt. Zero-based. Requires includeAnswers=true. |
summary.customer.perPrompt, when present, already breaks your result down per prompt — a
label, which models named you, and a 0–100 position score. It’s on the plain response and
costs nothing, so reach for it before includeAnswers.
get_ai_visibility_history
Paginated history of AI Visibility checks.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
page | integer | No | Page number. Default 1. |
limit | integer | No | Items per page. Default 20, max 100. |
get_ai_visibility_check_detail
Full detail for one AI Visibility check — per-competitor comparison, mention rates, AI Visibility Scores, and per-engine results. Takes the same answer parameters as the dashboard; see The answers block.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
checkId | string | Yes | Check ID, from get_ai_visibility_history. |
includeAnswers | boolean | No | Default false. Also return the models’ raw answers. Large payload — see The answers block. |
provider | string | No | One of openai, claude, gemini. Return only that model’s answers. Requires includeAnswers=true. |
brand | string | No | Return only the entries for this domain, across every answer. Requires includeAnswers=true. |
promptIndex | integer | No | Return only the answers for this prompt. Zero-based. Requires includeAnswers=true. |
get_ai_visibility_trend
AI Visibility trend over time (up to 200 data points). Without a provider filter it returns
the aggregate Mention Rate plus the AI Visibility Score. With a provider filter it returns
that engine’s Mention Rate only — the AI Visibility Score is a single cross-engine
composite, so it comes back null when a provider is set.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
dateFrom | string | No | Start date, ISO-8601 (e.g. 2026-01-01). |
dateTo | string | No | End date, ISO-8601. |
provider | string | No | One of openai, claude, gemini. Omit for the aggregate view. |
{
"name": "get_ai_visibility_trend",
"arguments": {
"projectId": "65a1b2c3d4e5f6a7b8c9d0e1",
"dateFrom": "2026-01-01",
"provider": "claude"
}
}Filtering by claude returns Claude’s Mention Rate over time; the AI score fields are
null under a provider filter.
The answers block
get_ai_visibility_dashboard and get_ai_visibility_check_detail both return numbers by
default. Set includeAnswers=true and you also get the evidence underneath them: every
prompt sent, and every brand each model named in rank order with its stated reasoning,
target audience, pricing signal, messaging keywords and differentiation claims — plus
per-model reporting status.
Attribute it to the model, never to CompetLab. Every piece of prose in this block — descriptions, ranking rationales, audiences, claims — is unverified model output about the brands that model named, including third parties CompetLab does not monitor. It is a record of what the model said, not CompetLab’s assessment of those brands. Report it as what that model said; do not republish it as fact.
A rate is a share of the answers counted, never of the queries asked. Every rate divides
by the answers that came back. A model that named nobody still answered — that’s a measured
absence, and it is not the same as a query that produced no answer at all. Those live in
unansweredQueries, in their own array, precisely so answers.length can never be read as
queries sent. Never pair the two as a fraction, and never report an empty mentionedBy or a
0 score as “we couldn’t measure.”
Size it before you fetch it. Read summary.totalEntries first — an entry serializes to
roughly 200 tokens. In practice the block runs about 12k tokens for a 3-prompt check,
against roughly 2k with a brand= filter. Prefer a filter over fetching everything.
The three filters narrow differently, and the difference is the point:
| Filter | What it narrows |
|---|---|
provider | The answers array (and the matching unansweredQueries) to one model. |
promptIndex | The answers array (and the matching unansweredQueries) to one prompt. |
brand | Not the answers array. It reduces the brands list inside each answer, so every counted answer is still returned and the ones that didn’t name that domain arrive with an empty brands. |
That last row is what makes brand= the cheapest way to answer “where does this competitor
beat me, and where are they invisible” — you see both the wins and the silences in one
~2k-token call. It matches brands[].domain case-insensitively; brand names are the
model’s own wording and vary between answers, so they’re never matched.
No filter changes any number under summary — those are stored, computed over the whole
check, and never recomputed for a filtered view. Ranks stay stable under any filter. If
answersTruncated comes back true, the response-size cap fired and whole answers were
dropped from the end — narrow and retry.
{
"name": "get_ai_visibility_check_detail",
"arguments": {
"projectId": "65a1b2c3d4e5f6a7b8c9d0e1",
"checkId": "65a1b2c3d4e5f6a7b8c9d0e2",
"includeAnswers": true,
"brand": "competitor.com"
}
}Positioning
get_positioning_dashboard
Returns the latest homepage-messaging analysis for every competitor: page title, headline, tagline, value proposition, primary and secondary CTAs, key offerings, target audience, main differentiator, pricing mentions, and free-trial info.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
get_positioning_history
Paginated history of Positioning monitoring runs.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
page | integer | No | Page number. Default 1. |
limit | integer | No | Items per page. Default 20, max 100. |
get_positioning_run_detail
Full competitor-by-competitor data for one historical Positioning run.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
runId | string | Yes | Run ID, from get_positioning_history. |
Pricing
get_pricing_dashboard
Returns the latest structured pricing for every competitor — plans (each with a name, a
price such as $49/month, and a summary; up to five per competitor) — plus market pricing
statistics and a pricing gap analysis.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
get_pricing_history
Paginated history of Pricing monitoring runs.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
page | integer | No | Page number. Default 1. |
limit | integer | No | Items per page. Default 20, max 100. |
get_pricing_run_detail
Full competitor-by-competitor data for one historical Pricing run.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
runId | string | Yes | Run ID, from get_pricing_history. |
Content
get_content_dashboard
Returns the latest Content Intelligence for every competitor: sitemap URL counts, strategic URL identification, content categorization across 11 categories, sitemap structure, and a content gap analysis.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
get_content_history
Paginated history of Content monitoring runs.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
page | integer | No | Page number. Default 1. |
limit | integer | No | Items per page. Default 20, max 100. |
get_content_run_detail
Full competitor-by-competitor data for one historical Content run.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
runId | string | Yes | Run ID, from get_content_history. |
get_content_changelog
Detected content changes per competitor sitemap over time — URLs added and removed, with
per-category counts and up to three sample URLs per category by default. Filter by
competitor and/or category to scope it. Set allUrlsPerCategory: true for the full URL
list per category; high-activity competitors can produce large responses, so combine it
with filters and watch the truncated flag in the response.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
page | integer | No | Page number. Default 1. |
limit | integer | No | Items per page. Default 20, max 100. |
competitorId | string | No | Filter by competitor ID, from list_competitors. |
category | string | No | One of blog, docs, tools, landing, legal, caseStudies, comparison, integrations, changelog, webinars, other. |
allUrlsPerCategory | boolean | No | Default false (up to 3 sample URLs per category). Set true for full URL lists. |
Tech & Trust
get_tech_trust_dashboard
Returns the latest Tech & Trust profile for every competitor: security headers (an A–F
grade, plus HSTS, CSP, X-Frame-Options, and X-Content-Type-Options), trust signals (26 signals
across five categories: compliance, reviews, socialProof, certifications, and
disclosures), the detected technology stack, AI access — which named assistants can obtain the
site’s pages and which model operators may train on it — and DNS infrastructure.
On aiAccess, absent is not empty and not open. Check measurement.status before reading
anything else. On could_not_measure the assistantAccess and modelTrainingAccess keys are
omitted entirely — an empty array would claim we evaluated every assistant and none can reach
the site, which is a different fact. Writing assistantAccess ?? [] reintroduces exactly the bug
this shape exists to prevent.
measured_no_policy_found is the opposite case and a real finding: the site publishes no
robots.txt, which under the standard allows every crawler, so the verdicts render and are all
open.
disclosures is the newest category — today a single signal, a linked privacy policy. Every
check carries all five categories, so the denominators above hold for every response you’ll see.
One category name collides, and it is the trap on this response. socialProof is a field on
both this dimension and the standalone trust-signals scan, spelled identically, and both have
exactly five members — so neither the name nor the count tells you they differ.
Here the five are customer logos, a customer-count claim, case studies, a money-back guarantee and a free trial. In the scan they are customer logos, hero-only logos, customer count, case studies and testimonials.
So a socialProof of 3 from this tool and 4 from a scan is not a change, not an improvement
and not a discrepancy — the two were never measuring the same set. If you hold both numbers,
report them separately or not at all.
The two share three members (logos, customer count, case studies) and differ in the other two
in each direction, which is why the totals track each other closely enough to look like drift.
They are produced by different detectors against different rule sets: this dimension is the
monitored 26-signal taxonomy, start_trust_signals_scan is a separate 34-signal set. Both now
count five categories, so the number of categories no longer tells them apart either — and
socialProof is the one name the two lists share.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
get_tech_trust_history
Paginated history of Tech & Trust runs, with completion timestamps.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
page | integer | No | Page number. Default 1. |
limit | integer | No | Items per page. Default 20, max 100. |
get_tech_trust_run_detail
Full competitor-by-competitor data for one historical Tech & Trust run.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
runId | string | Yes | Run ID, from get_tech_trust_history. |
Alerts
list_alerts
Returns paginated competitive alerts — detected changes across all monitored dimensions, with change diffs and action hints. Filter by dimension, severity, and/or competitor.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
page | integer | No | Page number. Default 1. |
limit | integer | No | Items per page. Default 20, max 100. |
dimension | string | No | One of tech-trust, content, positioning, pricing, ai-visibility. |
severity | string | No | One of critical, high, medium, info. |
competitorId | string | No | Filter by competitor ID, from list_competitors. |
{
"name": "list_alerts",
"arguments": {
"projectId": "65a1b2c3d4e5f6a7b8c9d0e1",
"dimension": "pricing",
"severity": "critical"
}
}Schedules
list_schedules
Returns the monitoring schedules for all five monitored dimensions: enabled/disabled status, interval in days, and the next- and last-run timestamps for each.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
Strategic Briefing
get_briefing
Returns the project’s Strategic Briefing — the synthesized, prioritized read across all
13 dimensions: what changed, what it means, and what to do about it. This is the analyzed,
as-of read, not raw monitoring; for live per-dimension data use the get_<dimension>_dashboard
tools. It’s generated automatically per project and refreshed roughly monthly.
By default it returns the executive hub — a cheap digest (headline, top moves, and a
per-dimension verdict that names the deeper section to open next) that answers most
questions in a single call. Request extra sections only when a question needs them.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
sections | string[] | No | Which sections to return. Default ["hub"]. See the values below. |
includeCharts | boolean | No | Default false. Set true to include full chart series (larger payload). |
Valid sections values (pass any combination):
hub actions competitors all
deep-ai-visibility deep-positioning deep-pricing deep-content deep-tech-trust
deep-agent-readiness deep-ai-ecosystem deep-customer-voice deep-funding-capital
deep-hiring-gtm deep-landscape deep-product-launches deep-reliability-statusThere’s one deep- section per dimension — the five monitored plus the eight researched
dimensions researched for the briefing (13 in all). (deep-agent-readiness is a frozen API
identifier; the dimension is Agent Adoption.) The hub’s verdicts tell you which
deep- section to open, so you rarely need to guess. all returns the entire briefing and
is large — use it for export or a full read only.
{ "name": "get_briefing", "arguments": { "projectId": "65a1b2c3d4e5f6a7b8c9d0e1" } }{
"name": "get_briefing",
"arguments": {
"projectId": "65a1b2c3d4e5f6a7b8c9d0e1",
"sections": ["actions", "deep-pricing"]
}
}get_briefing returns the latest run in whatever state it is in. Check meta.status:
on done the briefing is in item; on running it’s being generated now (meta.progress
gives the step); on failed the last attempt ended without producing an edition; on null
the project has never had a briefing at all.
On running or failed, item is null — but an earlier edition is usually still
readable. Only meta.status === null means the project genuinely has nothing. Never tell
a user no briefing is available on the strength of a null item without calling
get_briefing_history first.
get_briefing_history
Lists this project’s past briefing editions, newest first — one cheap metadata row each
(runId, publication date, edition number, status, and that edition’s one-line headline
verdict). It never returns briefing content.
Use it to find which edition to open — “what did we say in April”, “how has the read
changed” — then pass the runId to get_briefing_edition. For the project’s current state
use get_briefing instead.
Runs that failed or are still generating are included too, with a null date and headline, so
a gap between two editions is explained rather than left a mystery. This is also the correct
fallback when get_briefing reports running or failed: the newest readable edition is
the most recent row here with status done.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
page | integer | No | Page number. Default 1. |
limit | integer | No | Items per page. Default 20, max 100. |
Check pagination.hasMore to fetch additional pages.
get_briefing_edition
Returns one past edition in full, by runId — the same shape as get_briefing, with the
same sections and includeCharts options and the same hub default. Use it to read or
quote a specific past edition, including the last readable one when get_briefing reports
running or failed.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectId | string | Yes | Project ID, from list_projects. |
runId | string | Yes | Briefing run ID, from get_briefing_history. |
sections | string[] | No | Which sections to return. Default ["hub"]. Same values as get_briefing. |
includeCharts | boolean | No | Default false. Set true to include full chart series. |
A runId naming a run that failed or is still generating returns successfully with
meta.status set and item null: that run genuinely produced no edition, which is an
answer, not an error. A malformed id returns 400 invalid_run_id; an id that belongs to
another project returns 404 run_not_found.
{
"name": "get_briefing_edition",
"arguments": {
"projectId": "65a1b2c3d4e5f6a7b8c9d0e1",
"runId": "65a1b2c3d4e5f6a7b8c9d0e3",
"sections": ["hub", "actions"]
}
}Run vs edition. A run is one attempt of the generator, addressed by runId, with
status running, done, or failed. An edition is the content a run produced — only a
done run has one. editionNumber counts published editions, so a failed or running run
has a runId but no editionNumber and no content.
Free Tools
The free tools run live against any public domain — they don’t need a project. Three of
them are async: a start_* call kicks off a scan and returns a scanId, and the matching
get_* call polls for the result.
check_sitemap
Live sitemap analysis for any domain — discovers URLs, categorizes them by section, and
reports depth, freshness, and per-category counts. Discovery reads both the conventional
/sitemap.xml and every sitemap the site’s robots.txt declares, merged and deduplicated, so a
site publishing several gets all of them.
| Parameter | Type | Required | Description |
|---|---|---|---|
domain | string | Yes | Domain to scan, e.g. example.com. |
sitemapUrl | string | No | Full URL to a specific sitemap (with http:// or https://). Skips discovery. |
It does not find content gaps. Everything it returns is a measurement of the sitemaps it read — there is no second site in the picture, so nothing here is a gap analysis. (Gap analysis across competitors is the Content Intelligence dimension.)
status: "partial" has two causes and they are not interchangeable. It means the scan did
not cover the whole corpus — never that the site is broken.
- The site’s own defect —
unreadSitemapCount > 0: a sitemap its robots.txt declares that could not be read (a relative path instead of a full URL, a redirect off its own domain, a server that refused us). This is the reportable finding. QuoteunreadSitemapCount, neverunreadSitemaps.length— the list is capped at 20 and omits any value carrying a scheme the scan won’t repeat back, so a scan can report unread sitemaps with an empty list. That’s still the site’s defect, not a gap in the count. - Our own limits —
truncatedalone, with no count: we stopped at a cap. Filing that under “could not be read” blames the site for where we stopped.
A declared sitemap that answers 404 is neither: that is a measured absence, not a look we failed to take. And never report a count from a partial scan as the site’s total.
One thing this tool cannot tell you, however it reads: what any other crawler can reach. We fetch once, from one IP, under one redirect policy — we refuse cross-domain redirects, for one. A crawler operating under different rules may well reach what we couldn’t, so an unread sitemap is a fact about our fetch, not a verdict on the site’s reachability in general. Report what we read.
The programmatic category covers templated pages generated from a database or a
pattern. Two properties of it govern how you may report it:
- It is assigned to a group, never to a single URL — 25 or more sibling URLs under one
parent path, at least 80% of whose slugs look machine-generated. A page cannot look
generated on its own, so a small handful of code-named URLs stays
other. - It says how pages are generated, never why. A URL shape can’t tell a deliberate programmatic-SEO play from a reference database, so this is a count, not a verdict — don’t report it as a content strategy. A tax-code lookup service with 17,000 generated pages is running a product.
Evidence ships with the count: insights.sampleUrlsByCategory.programmatic carries example
URLs deliberately spread across different groups rather than five consecutive siblings, so a
reader can check the label — and so three generated sections don’t read as one. Those samples
are the only URLs an MCP caller ever sees: this tool takes domain and sitemapUrl and
nothing else, so there is no flag that returns the full URL list. Don’t go looking for one.
status is one of ok, partial, not-found, access-denied, or invalid. Two are easy
to misread: partial means a limit stopped the scan early, and it can arrive with zero URLs
(sitemap indexes nested deeper than the walker goes), while invalid means a document was
retrieved and couldn’t be parsed as XML — never that nothing arrived, which is not-found.
check_ai_crawlers
Live check of which AI assistants can fetch a site’s pages, read from its robots.txt.
assistantAccess is the answer — one verdict per assistant (ChatGPT, Claude, Perplexity,
Microsoft Copilot, Google AI Overviews, Gemini Apps), each with the crawlers that decided it named
beside it. Count that array for totals: no count is stored, and there is deliberately no overall
score.
| Parameter | Type | Required | Description |
|---|---|---|---|
domain | string | Yes | Domain to scan, e.g. example.com. |
industry | string | No | Industry context for benchmarking. One of news-media, arts-entertainment, law-government, finance-healthcare, saas-tech, ecommerce, other. |
Two things it does not tell you, both of which get misreported:
- It says an assistant is permitted to fetch the site, never that it cites it.
modelTrainingAccessis a separate, neutral fact. Blocking training crawlers costs no visibility and is a legitimate content decision — never report it as a gap or advise undoing it. The one exception is mechanical: where a token undermodelTrainingAccess[].decidedByCrawlersalso appears underassistantAccess[].decidedByCrawlers—Google-Extendedis the documented case — that block does cost visibility. Match onuserAgentTokenbefore applying the general rule.
crawlers[].ruleAudience tells you whether a rule named the crawler or a User-agent: *
catch-all swept it up. The second is usually accidental, and it’s the more actionable finding.
Check robotsTxt.read first. When the file cannot be read the tool returns the read outcome
and no verdict — no assistant access, no crawler list, no advice. A failed read is not an open
site.
{ "name": "check_ai_crawlers", "arguments": { "domain": "example.com" } }fetch_url
Fetches any public URL with automatic JS-rendering and common bot-protection handling, and
returns the body, headers, and clean-up stats. cleanHtml strips HTML noise while keeping
the text — a real token saving when an agent is about to read the page.
| Parameter | Type | Required | Description |
|---|---|---|---|
url | string | Yes | Target URL. Must be http(s) and resolve to a public host. |
bodyNeeded | boolean | No | Include body and contentType in the response. |
headersNeeded | boolean | No | Include headers and headersAvailable. Some sites reveal nothing — then headersAvailable is false and headers is {}, present and empty rather than missing. Branch on headersAvailable, never on whether headers exists. |
cleanHtml | boolean | No | For HTML responses, strip scripts/styles/comments and keep text. Requires bodyNeeded. |
maxTimeoutMs | integer | No | Timeout budget in ms. Range 1000–120000. |
bodyMaxBytes | integer | No | Response body cap in bytes. Range 1024–104857600 (1 KiB–100 MiB). |
At least one of bodyNeeded or headersNeeded must be true. This tool has the tighter
rate limit of the free tools — 60 requests per minute per key.
{
"name": "fetch_url",
"arguments": { "url": "https://example.com", "bodyNeeded": true, "cleanHtml": true }
}start_tech_stack_scan · get_tech_stack_scan
starts a scan
start_tech_stack_scan begins an async tech-stack detection on a domain — 117 detection
rules spanning the tech stack (hosting, frameworks, CMS, payments), the growth stack
(analytics, marketing, CRM, advertising), and the engagement stack (support, forms, video,
monitoring). It returns a scanId immediately; scans typically finish in seconds, up to
~90 seconds for heavy-render sites.
get_tech_stack_scan returns the current status while running and the detected
technologies (with confidence scores) once complete — poll every 2–5 seconds.
| Tool | Parameter | Type | Required | Description |
|---|---|---|---|---|
start_tech_stack_scan | domain | string | Yes | Domain to scan, e.g. example.com. |
get_tech_stack_scan | scanId | string | Yes | Scan ID, from start_tech_stack_scan. |
{ "name": "start_tech_stack_scan", "arguments": { "domain": "example.com" } }{ "name": "get_tech_stack_scan", "arguments": { "scanId": "65a1b2c3d4e5f6a7b8c9d0e1" } }start_trust_signals_scan · get_trust_signals_scan
starts a scan
start_trust_signals_scan begins an async trust-signals analysis on a domain — 34 signals
across five categories: enterprise readiness, third-party validation, social proof, brand
authority, and risk reversal. It returns a scanId; get_trust_signals_scan returns the per-signal verdicts
and an overall tier verdict once complete. Poll every 2–5 seconds.
| Tool | Parameter | Type | Required | Description |
|---|---|---|---|---|
start_trust_signals_scan | domain | string | Yes | Domain to scan, e.g. example.com. |
get_trust_signals_scan | scanId | string | Yes | Scan ID, from start_trust_signals_scan. |
start_agent_adoption_scan · get_agent_adoption_scan
starts a scan
start_agent_adoption_scan begins an async Agent Adoption Check on a domain — 25 checks
across discoverability, access control, content readability, and agent endpoints, following
the open Agent-Adoption Specification. It returns a scanId; get_agent_adoption_scan
returns the current status while running and the complete results once finished. Poll every
2–5 seconds.
| Tool | Parameter | Type | Required | Description |
|---|---|---|---|---|
start_agent_adoption_scan | domain | string | Yes | Domain to scan, e.g. example.com. |
get_agent_adoption_scan | scanId | string | Yes | Scan ID, from start_agent_adoption_scan. |
Next steps
- Connect to the server → — configs for every client.
- Back to the overview → — how it works, auth, and the FAQ.