Sitemap Visualizer

REST endpoints for Sitemap Visualizer. Base URL https://api.competlab.com; every request needs a CL-API-Key header (see Authentication). Responses wrap in { item } or { items }; errors in { error }.

Analyze sitemap structure of a domain

POST/v1/tools/sitemap-visualizer

Discover and analyze the sitemap structure of any public domain. Locates sitemap.xml (or accepts an explicit URL), categorizes URLs by content type, validates structure, and surfaces freshness and depth-distribution insights with sample URLs per category. Optionally include the full URL list (off by default to keep payload bounded). Stateless live check — no project required, no data is persisted. Suitable for content-strategy audits, SEO discovery, and content-volume benchmarking.

Returns · { item }

Request body — application/json

Field	Type	Description
`domain`	`string`	Target domain whose sitemap to analyze. Accepts a bare hostname or a full URL — normalized (lowercased, scheme and path stripped) before analysis.
`sitemapUrl`	`string`	Explicit sitemap URL to fetch directly, short-circuiting /sitemap.xml then robots.txt discovery. Must be a full http:// or https:// URL with a public hostname.
`includeUrls`	`boolean`	When true, include the full raw `urls[]` list in the response. Defaults to false, which returns a lean insights-only payload without the URL list.

Request


curl -X POST "https://api.competlab.com/v1/tools/sitemap-visualizer" \
  -H "CL-API-Key: YOUR_COMPETLAB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"domain":"stripe.com","sitemapUrl":"https://stripe.com/sitemap.xml","includeUrls":false}'

Response 200 OK

Example


{
  "item": {
    "domain": "competlab.com",
    "fetchedAt": "2026-05-16T14:30:00.000Z",
    "status": "ok",
    "source": "sitemap-xml",
    "sitemaps": [
      {
        "url": "https://competlab.com/sitemap.xml",
        "type": "urlset",
        "urlCount": 247,
        "actualUrlCount": 247
      }
    ],
    "truncated": false,
    "accessDenied": false,
    "metrics": {
      "totalUrls": 247,
      "totalActualUrls": 247,
      "totalSitemaps": 3,
      "categoriesFound": 6,
      "maxDepth": 4,
      "avgDepth": 2.6
    },
    "urls": [
      {
        "url": "https://competlab.com/blog/competitive-intelligence-guide",
        "category": "blog",
        "depth": 2,
        "lastmod": "2026-04-12"
      }
    ],
    "categories": {
      "blog": {
        "count": 47,
        "percentage": 18.4
      },
      "docs": {
        "count": 47,
        "percentage": 18.4
      },
      "tools": {
        "count": 47,
        "percentage": 18.4
      },
      "landing": {
        "count": 47,
        "percentage": 18.4
      },
      "legal": {
        "count": 47,
        "percentage": 18.4
      },
      "case-studies": {
        "count": 47,
        "percentage": 18.4
      },
      "comparison": {
        "count": 47,
        "percentage": 18.4
      },
      "integrations": {
        "count": 47,
        "percentage": 18.4
      },
      "changelog": {
        "count": 47,
        "percentage": 18.4
      },
      "webinars": {
        "count": 47,
        "percentage": 18.4
      },
      "careers": {
        "count": 47,
        "percentage": 18.4
      },
      "other": {
        "count": 47,
        "percentage": 18.4
      }
    },
    "validation": {
      "sitemapFound": {
        "passed": true,
        "message": "Found valid sitemap at /sitemap.xml"
      },
      "validXml": {
        "passed": true,
        "message": "Found valid sitemap at /sitemap.xml"
      },
      "nonEmpty": {
        "passed": true,
        "message": "Found valid sitemap at /sitemap.xml"
      },
      "validIndex": {
        "passed": true,
        "message": "Found valid sitemap at /sitemap.xml"
      }
    },
    "insights": {
      "freshness": {
        "fresh": 32,
        "recent": 18,
        "aging": 41,
        "stale": 122,
        "noDate": 34
      },
      "depthDistribution": {
        "0": 1,
        "1": 24,
        "2": 87,
        "3": 92,
        "4+": 43
      },
      "stalePageCount": 163,
      "sampleUrlsByCategory": {
        "blog": [
          "https://competlab.com/blog/competitive-intelligence-guide",
          "https://competlab.com/blog/ai-visibility-tracking"
        ],
        "docs": [
          "https://competlab.com/docs/getting-started"
        ],
        "case-studies": [
          "https://competlab.com/case-studies/acme"
        ]
      },
      "summary": "247 URLs across 6 content categories. Blog dominates at 18% with fresh content (32 URLs in the last 3 months). 163 pages are stale (>6 months), concentrated in the docs and legal categories — possible refresh opportunity."
    }
  }
}

Fields

item — object

Field	Type	Description
`domain`	`string`	Normalized domain, as actually analyzed (after normalization)
`fetchedAt`	`string`	ISO-8601 timestamp of scan completion
`status`	`string`	Coarse status — ‘ok’/‘partial’ for success, ‘not-found’/‘access-denied’/‘invalid’ as first-class envelope states. Values: `ok`, `partial`, `not-found`, `access-denied`, `invalid`.
`source`	`string`	How the sitemap was discovered. Values: `sitemap-xml`, `robots-txt`, `provided`, `none`.
`sitemaps`	`array<object>`	All sitemap resources fetched or attempted. See `item.sitemaps[]`.
`truncated`	`boolean`	True if the URL list was capped at 10,000 or child-sitemap fan-out hit the 20-child cap
`accessDenied`	`boolean`	True if the root sitemap returned 401/403
`metrics`	`object`	Quantitative metrics — URL counts, sitemap count, category count, depth stats. See `item.metrics`.
`urls`	`array<object>`	Raw categorized URLs. Present only when request had includeUrls: true. Capped at 10,000; check `truncated` flag. See `item.urls[]`.
`categories`	`object`	Per-category breakdown. Every category key is populated (count may be 0). See `item.categories`.
`validation`	`object`	Quality checks for sitemap discovery and parsing. See `item.validation`.
`insights`	`object`	AI-consumable insight layer — freshness, depth distribution, stale-page count, category samples, summary. See `item.insights`.

item.sitemaps[] — each entry

Field	Type	Description
`url`	`string`	Absolute URL of the fetched sitemap resource
`type`	`string`	Sitemap resource shape. Values: `index`, `urlset`.
`urlCount`	`number`	Count of URLs collected from this sitemap (post 10k-cap)
`actualUrlCount`	`number`	Count of URLs this sitemap actually declared (pre 10k-cap)

item.metrics — object

Field	Type	Description
`totalUrls`	`number`	Count of URLs analyzed (post 10k-cap)
`totalActualUrls`	`number`	True upstream URL count before the 10k cap
`totalSitemaps`	`number`	Number of sitemap resources fetched
`categoriesFound`	`number`	Number of distinct content categories with at least one URL
`maxDepth`	`number`	Maximum path depth observed
`avgDepth`	`number \| null`	Average pathname depth across all URLs. Null when urls[] is empty. Nullable.

item.urls[] — each entry

Field	Type	Description
`url`	`string`	Absolute URL of the page
`category`	`string`	Content-category classification (forked taxonomy, kebab-case). Values: `blog`, `docs`, `tools`, `landing`, `legal`, `case-studies`, `comparison`, `integrations`, `changelog`, `webinars`, `careers`, `other`.
`depth`	`number`	Path depth — segment count of URL.pathname after filtering empty parts
`lastmod`	`string \| null`	Value of <lastmod> from the source XML if present, else null. Nullable.

item.categories — object

Field	Type	Description
`blog`	`object`	Blog posts and editorial content. See `item.categories.blog`.
`docs`	`object`	Product documentation. See `item.categories.docs`.
`tools`	`object`	Free tools and utilities. See `item.categories.tools`.
`landing`	`object`	Landing / marketing pages. See `item.categories.landing`.
`legal`	`object`	Legal pages — privacy, terms, DPA, etc. See `item.categories.legal`.
`case-studies`	`object`	Customer case studies. See `item.categories.case-studies`.
`comparison`	`object`	Comparison pages (X vs Y). See `item.categories.comparison`.
`integrations`	`object`	Integration pages. See `item.categories.integrations`.
`changelog`	`object`	Changelog / release notes. See `item.categories.changelog`.
`webinars`	`object`	Webinars and events. See `item.categories.webinars`.
`careers`	`object`	Careers / hiring pages. See `item.categories.careers`.
`other`	`object`	Uncategorized pages. See `item.categories.other`.

item.validation — object

Field	Type	Description
`sitemapFound`	`object`	Whether a sitemap was discovered (via /sitemap.xml, robots.txt, or provided URL). See `item.validation.sitemapFound`.
`validXml`	`object`	Whether the discovered XML parsed successfully. See `item.validation.validXml`.
`nonEmpty`	`object`	Whether the sitemap contained at least one URL. See `item.validation.nonEmpty`.
`validIndex`	`object`	Whether sitemap index referenced valid child sitemaps (if applicable). See `item.validation.validIndex`.

item.insights — object

Field	Type	Description
`freshness`	`object`	Bucketization of URLs by content freshness. See `item.insights.freshness`.
`depthDistribution`	`object`	URL count grouped by path depth (0 = home, ‘4+’ = 4+ segments). See `item.insights.depthDistribution`.
`stalePageCount`	`number \| null`	Count of URLs older than ~6 months (180 days). Null when no <lastmod> data is present at all. Nullable.
`sampleUrlsByCategory`	`object`	Up to N sample URLs per populated category, largest categories first. Keys are content categories; absent keys mean the category had zero URLs.
`summary`	`string`	Deterministic, rule-based interpretation string (1-3 sentences)

item.categories.blog — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.docs — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.tools — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.landing — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.legal — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.case-studies — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.comparison — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.integrations — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.changelog — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.webinars — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.careers — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.categories.other — object

Field	Type	Description
`count`	`number`	Number of URLs in this category
`percentage`	`number`	Percentage of total URLs (0-100, one decimal place)

item.validation.sitemapFound — object

Field	Type	Description
`passed`	`boolean`	Whether the validation check passed
`message`	`string`	Human-readable reason

item.validation.validXml — object

Field	Type	Description
`passed`	`boolean`	Whether the validation check passed
`message`	`string`	Human-readable reason

item.validation.nonEmpty — object

Field	Type	Description
`passed`	`boolean`	Whether the validation check passed
`message`	`string`	Human-readable reason

item.validation.validIndex — object

Field	Type	Description
`passed`	`boolean`	Whether the validation check passed
`message`	`string`	Human-readable reason

item.insights.freshness — object

Field	Type	Description
`fresh`	`number`	URLs last modified within the last 3 months (0-90 days)
`recent`	`number`	URLs last modified 3-6 months ago (90-180 days)
`aging`	`number`	URLs last modified 6-12 months ago (180-360 days)
`stale`	`number`	URLs last modified more than a year ago (>360 days)
`noDate`	`number`	URLs missing the <lastmod> tag

item.insights.depthDistribution — object

Field	Type	Description
`0`	`number`	URLs at path depth 0 (home pages)
`1`	`number`	URLs at path depth 1
`2`	`number`	URLs at path depth 2
`3`	`number`	URLs at path depth 3
`4+`	`number`	URLs at path depth 4 or deeper

Errors

Status	Code	Meaning
400	`invalid_parameters`	Bad request — the payload failed validation.
401	`api_key_missing` · `api_key_invalid` · `api_key_revoked` · `api_key_expired` · `insufficient_scope`	The `CL-API-Key` is missing, malformed, revoked, expired, or lacks the required scope.
429	`rate_limit_exceeded`	Rate limit exceeded — retry with backoff.
502	`bad_gateway`	Upstream fetch failed (e.g. the target page could not be retrieved).

Every error uses the shared { error: { code, message, status } } envelope; code is one of the values listed above.


{
  "error": {
    "code": "invalid_parameters",
    "message": "Domain is required",
    "status": 400
  }
}

Transient failures (429, 5xx) are safe to retry with exponential backoff.