---
title: "Track competitor content through their sitemap.xml"
description: "Open the competitor's sitemap.xml to list every page they have published. If the file does not load, read robots.txt for the Sitemap: line, then open each sitemap it lists. Copy the URLs into a sheet and use them to spot content they have launched that is not ranking yet."
url: "https://seo-mcp.io/t/track-competitor-content-through-their-sitemap-xml"
---

# Track competitor content through their sitemap.xml

Open the competitor's sitemap.xml to list every page they have published. If the file does not load, read robots.txt for the Sitemap: line, then open each sitemap it lists. Copy the URLs into a sheet and use them to spot content they have launched that is not ranking yet.

## Prerequisites

- A competitor domain to analyze
- A spreadsheet to store the collected URLs

## Steps

1. **Open the competitor's sitemap** *(general best practice)* — Request competitor.com/sitemap.xml in the browser. If it returns XML, note whether the root element is a urlset (page URLs) or a sitemapindex (a list of further sitemaps).
   Check: The URL returns HTTP 200 with an XML body whose root element is <urlset> or <sitemapindex>.
2. **Fall back to robots.txt** *(general best practice)* — If sitemap.xml does not load, open competitor.com/robots.txt and find every line that starts with Sitemap:. Those lines give the real sitemap locations.
   Check: robots.txt contains at least one line beginning with 'Sitemap:' followed by an absolute HTTPS URL.
3. **Open each listed sitemap** — Open every Sitemap: URL from robots.txt. When a file is a sitemapindex, open each child sitemap it lists, since a large site splits its URLs across several files.
   Check: Each opened sitemap returns a <urlset> containing <loc> entries rather than another level of <sitemapindex>.
4. **Extract the URLs into a sheet** — Collect every <loc> value into one column of a spreadsheet. Fetching the XML and extracting the loc tags with a command avoids manual copying on large sitemaps.
   Check: The sheet column count equals the number of <loc> entries in the sitemap when the same URL is counted once.
5. **Flag pages that are not ranking yet** *(general best practice)* — General best practice: check the extracted URLs against the competitor's visible rankings, and keep the URLs that do not appear, because sitemaps list published pages that have no search visibility yet. Prioritise the topics those pages cover that fit your own product.
   Check: The sheet contains a filtered set of URLs that do not appear in a site: search or in the rankings you checked for the competitor.
6. **Re-run the check on a schedule** *(general best practice)* — General best practice: repeat the extraction monthly and diff the new list against the previous one, using sitemap lastmod values where they exist to focus on recently published URLs.
   Check: The new URL list contains entries absent from the previous month's list, and each is dated by its lastmod value where present.

## Source

@denohawari on 2026-10-18: https://x.com/denohawari/status/2111808283220820356

Canonical: https://seo-mcp.io/t/track-competitor-content-through-their-sitemap-xml