sitemap 1.0.0

Sitemap Check v1.0.0

Validates that a site exposes a fetchable sitemap discovery surface using the Sitemaps protocol.

There is no IETF RFC for XML Sitemaps. The normative protocol reference is the Sitemaps XML protocol published at sitemaps.org. That protocol references URI and IRI syntax standards, but the sitemap protocol itself is not an RFC.

The check is isolated. It does not consume the result of the robots.txt check. It independently discovers sitemap candidates from standard sitemap paths and, when available, Sitemap: directives in /robots.txt.

Input Contract

[email protected]

Requires the scan origin. The check fetches standard sitemap candidates and may fetch ${origin}/robots.txt only to discover additional Sitemap: URLs.

Output Contract

[email protected]

Emits a stepped report check result with discovery, fetch, parse, field-quality, and URL-scope evidence.

Pass Criteria

• At least one sitemap candidate is discoverable.
• At least one candidate responds with a successful HTTP status.
• The response parses as a supported sitemap format: XML <urlset>,

XML <sitemapindex>, plain text, RSS, or Atom.

• The document contains at least one absolute HTTP(S) URL.
• For URL sitemaps, page URLs belong to the scanned origin host.
• Optional fields, when present, use valid values: <lastmod> is a parseable W3C/ISO-style date, <changefreq> is one of the protocol values, and <priority> is between 0.0 and 1.0.
• URL sitemap and sitemap index entry counts do not exceed the protocol's 50,000-entry limit.
• Sitemap content stays within the 50 MB uncompressed size limit.
• Supported extension records contain required fields and absolute HTTP(S) URL

values.

Warning Criteria

• A valid URL sitemap is found, but one or more page URLs point to a different

host than the scanned origin.

• A sitemap is valid, but optional fields such as <lastmod>, <changefreq>,

or <priority> contain invalid values.

• A sitemap is valid, but image, video, news, or hreflang extension records are

incomplete or contain relative/non-HTTP(S) URL values.

Failure Criteria

• No sitemap candidates can be discovered.
• No candidate returns a successful HTTP response.
• Fetched content is not a supported sitemap format.
• URL values are missing or are not absolute HTTP(S) URLs.
• The sitemap exceeds the protocol's 50,000 URL or 50,000 child sitemap limit.
• The sitemap exceeds the 50 MB uncompressed size limit.

Core Protocol Documents

The Sitemaps protocol defines two XML document shapes.

URL Sitemap Fields

Sitemap Index Fields

Protocol Limits

Large sites should split URLs across multiple sitemap files and publish a sitemap index.

URL Scope And Host Rules

Sitemap URLs are scoped. The protocol requires sitemap entries to belong to the same site or appropriate host scope as the sitemap location. In practice:

• A sitemap at https://example.com/sitemap.xml should list URLs under

https://example.com/.

• A sitemap should not mix unrelated hosts.
• Separate hosts or subdomains should publish their own sitemaps, or a sitemap

index should be placed at a scope that is valid for those child sitemaps.

• URL values should be canonical, absolute, HTTP(S) URLs.

This check warns when URL sitemap entries point outside the scanned origin host.

XML Requirements

• Sitemap files must be valid XML.
• XML entities must be escaped, including &, <, >, quotes, and apostrophes

where required.

• URLs must use valid URI or IRI syntax.
• The document should declare sitemap XML namespaces when using the XML

protocol or extensions.

Discovery Model

The check probes these conventional locations:

/sitemap.xml
/sitemap.txt
/sitemap_index.xml
/sitemap-index.xml

These paths are conventions, not the only valid sitemap locations.

The check also reads Sitemap: directives from /robots.txt when robots.txt is available. Robots discovery is additive; a missing or invalid robots.txt file does not make this check fail if a standard sitemap URL is valid.

Sitemap: directives:

• Use a full sitemap URL.
• Are not scoped to a specific User-agent group.
• May appear multiple times.
• Can point to sitemap indexes or individual sitemap files.

Submission And Notification

Alternate Sitemap Formats

The protocol allows alternatives to XML sitemap files.

This v1.0.0 check validates plain text, RSS, and Atom sitemap formats when they are found at a conventional path or advertised through a Sitemap: directive.

Extension Namespaces

The Sitemaps protocol supports XML namespace extensions. Extension support is crawler-specific; unsupported extension fields should not invalidate a core sitemap document.

This version validates basic extension correctness for image, video, news, and hreflang records. It checks required fields and absolute HTTP(S) URL values, but does not attempt crawler-specific eligibility decisions such as whether a news publisher is approved for Google News.

Search Engine Interpretation Notes

Related But Not Sitemap Protocol

Scoring Steps

Current v1.0.0 Coverage

This version checks:

• Conventional XML sitemap paths.
• Conventional plain text sitemap path.
• robots.txt Sitemap: directives.
• Fetch success.
• XML <urlset> and <sitemapindex> structure.
• Plain text, RSS, and Atom sitemap formats.
• Absolute HTTP(S) URL values.
• Same-origin host scope for URL sitemap entries.
• Same-origin host scope for child sitemap index entries.
• Optional <lastmod>, <changefreq>, and <priority> value quality.
• 50,000 URL and 50,000 child sitemap entry-count limits.
• 50 MB uncompressed sitemap size limits, including decompressed response bodies

as exposed by fetch.

• Basic image, video, news, and hreflang extension correctness.

External Signals Not Emitted By This Check

This package validates the public sitemap file. It does not emit search engine console submission state, ping submission state, indexing status, or crawler-specific eligibility decisions because those require external account data or live search-engine submission telemetry rather than the sitemap document itself.

References

• www.sitemaps.org/protocol.html
• www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd
• developers.google.com/search/docs/crawling-indexing/sitemaps/overview
• developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap
• developers.google.com/search/docs/crawling-indexing/sitemaps/image-sitemaps
• developers.google.com/search/docs/crawling-indexing/sitemaps/video-sitemaps
• developers.google.com/search/docs/crawling-indexing/sitemaps/news-sitemap
• developers.google.com/search/docs/specialty/international/localized-versions
• www.indexnow.org/documentation
• example.com/sitemap.xml
• example.com/

sitemap v1.0.0 Changelog

Initial versioned package for sitemap.

• Declares discovery, fetch, parse, and scope scoring steps.
• Documents XML sitemap and sitemap index semantics.
• Documents robots.txt Sitemap: discovery as an isolated additive discovery mechanism.
• Adds optional field-quality validation for <lastmod>, <changefreq>, and <priority>.
• Adds entry-count limit checks for URL sitemaps and sitemap indexes.
• Adds same-origin scope evidence for child sitemap URLs in sitemap indexes.
• Adds /sitemap.txt discovery and validates plain text, RSS, and Atom sitemap formats.
• Adds uncompressed 50 MB sitemap size validation.
• Adds basic image, video, news, and hreflang extension-quality validation.