1. Abstract
Verify that IndexNow ownership key placement is detectable when the site advertises it.
IndexNow lets sites notify participating search engines about changed URLs, but ownership verification requires a UTF-8 key file whose filename matches the key.
2. Classification
- Check ID
- indexnow-key
- Check version
- 1.0.0
- Package path
- lib/checks/indexnow-key/versions/1.0.0
- Category
- AI Discoverability
- Subcategory
- Discoverability
- Check group
- Crawl Discovery
- Check group ID
- crawl-discovery
- Maturity
- Established
- Scope
- site
- Check weight
- 0.05
3. Input And Output Contracts
- Input
- [email protected]
- Output
- [email protected]
- Resources inspected
- IndexNow key file, robots.txt IndexNow-Key, rel=indexnow-key, meta name=indexnow-key-location
4. Scoring Semantics
| Step ID | Title | Weight | Description |
|---|---|---|---|
advertised-location | Find advertised IndexNow key location | 0.35 | Find a same-origin IndexNow key location advertised through scanner-supported discovery hints. |
fetch-key | Fetch and validate IndexNow key file | 0.65 | Fetch the key file and verify that filename and body contain the same protocol-valid key. |
5. Package Documentation
IndexNow Key Check v1.0.0
Validates a discoverable IndexNow ownership key file when the site advertises its location.
IndexNow is a URL-change notification protocol. It lets a site notify participating search engines that URLs were created, updated, or deleted. The key file proves that the submitter controls the host or host path being submitted.
Input Contract
Requires the scan origin, homepage HTML, homepage headers, and parsed homepage document. The check may independently fetch /robots.txt to look for scanner-supported key-location hints.
Output Contract
Emits a stepped report check result with key-location discovery and key-file validation evidence.
Official Protocol Model
IndexNow has two moving parts:
| Part | Purpose |
|---|---|
| Key file | Proves ownership of a host or host path. |
| URL submission API | Notifies participating engines about URL changes using the same key. |
The key file is a UTF-8 text file. Its filename is {key}.txt, and its body contains the same key value.
Key Format
| Rule | Requirement |
|---|---|
| Length | 8 to 128 characters. |
| Characters | ASCII letters, digits, and hyphen. |
| Filename | {key}.txt. |
| File body | The key value. This check tolerates a UTF-8 BOM and surrounding whitespace/newline when reading text files. |
| Encoding | UTF-8 text with a text-compatible content type when one is supplied. |
Examples:
52e163cb29f13843a5b58d2854fe46c9
CanAgentUse-IndexNow-2026Key Location
The simplest official placement is:
https://example.com/{key}.txtA key file can also be placed elsewhere on the host, but the submitted IndexNow request must include keyLocation when the key file is not at the root.
Scope matters:
| Key file location | Valid submission scope |
|---|---|
https://example.com/{key}.txt | Any URL under https://example.com/. |
https://example.com/blog/{key}.txt | URLs under https://example.com/blog/. |
| Different host | Does not verify ownership for the scanned host. |
This check validates same-origin advertised key files. It does not submit URLs to IndexNow endpoints.
URL Submission API
Official submission supports both single URL and batch URL submission.
Single URL style:
https://api.indexnow.org/indexnow?url=https://example.com/page&key=KEYBatch style uses a JSON POST body with host, key, optional keyLocation, and urlList.
Protocol limits and behavior:
| Feature | Requirement |
|---|---|
| Batch size | Up to 10,000 URLs per POST. |
| URL ownership | Submitted URLs must belong to the host verified by the key. |
keyLocation | Required when the key file is not at the default root location. |
| Changed URLs only | The protocol is intended for created, updated, or deleted URLs, not full-site recrawling. |
Submission Response Codes
| Status | Meaning |
|---|---|
200 | URL submitted successfully. |
202 | URL received; key validation or processing may still be pending. |
400 | Invalid request format. |
403 | Key is invalid or key file could not be verified. |
422 | Submitted URL does not belong to the host. |
429 | Too many requests. |
This check does not call submission endpoints, so these codes are documented for implementation guidance but are not emitted by v1.0.0.
Discovery Surfaces
The official protocol does not define a universal public discovery document for IndexNow key locations. A scanner cannot guess a private key value. Therefore this check only verifies IndexNow key files when the site advertises a key location through supported hints.
Supported v1.0.0 discovery hints:
| Surface | Status | Example |
|---|---|---|
robots.txt IndexNow-Key | Nonstandard scanner convention | IndexNow-Key: https://example.com/key.txt |
HTML <link rel="indexnow-key"> | Nonstandard scanner convention | <link rel="indexnow-key" href="/key.txt"> |
HTTP Link header | Nonstandard scanner convention | Link: </key.txt>; rel="indexnow-key" |
| HTML meta location | Nonstandard scanner convention | <meta name="indexnow-key-location" content="/key.txt"> |
These hints are not part of the official IndexNow protocol. They make the key file discoverable to CanAgentUse and similar scanners without submitting a URL to an IndexNow endpoint.
Pass Criteria
- A supported discovery hint advertises a same-origin key file URL.
- The key file responds with a successful HTTP status.
- The key file is served as text-compatible content, such as
text/plain. - The key file path ends in
{key}.txt. - The decoded key-file body matches
{key}after removing a UTF-8 BOM and surrounding whitespace/newline. - The key is 8 to 128 characters and contains only letters, digits, and hyphen.
Informational Criteria
- No supported key-location hint is advertised.
IndexNow is optional. Absence of a discoverable key is informational unless the site explicitly advertises a key location that cannot be validated.
Failure Criteria
- The site advertises an IndexNow key location, but the key file cannot be
fetched.
- The advertised key file is cross-origin.
- The key file body does not match the filename key.
- The key value contains unsupported characters or has an invalid length.
- The key file is advertised but served as a non-text response such as JSON or HTML.
Failures are low severity because IndexNow is a low-impact discovery signal in this assessment suite.
Scoring Steps
| Step | Weight | Purpose |
|---|---|---|
advertised-location | 0.35 | Find a supported same-origin key-location hint. |
fetch-key | 0.65 | Fetch and validate the key file. |
Current v1.0.0 Coverage
This version checks:
robots.txtIndexNow-Keyhints.- HTML
link rel="indexnow-key"hints. - HTTP
Linkheaderrel="indexnow-key"hints. - HTML
meta name="indexnow-key-location"hints. - Same-origin key-file URLs.
- Text-compatible key-file responses.
- Key filename and body match.
- Official key character and length constraints.
External Submission Signals Not Emitted By This Check
This package validates the public ownership key discovery and key file. It does not emit live IndexNow submission behavior because the scan input does not include submission logs, outgoing API requests, or search-engine endpoint responses.
Submission telemetry belongs in a separate check or integration that can observe actual IndexNow API calls. That integration can validate:
- Whether the site submits changed URLs to IndexNow.
- Whether
keyLocationis used correctly in live API submissions. - Whether submission endpoints return
200,202,400,403,422, or
429.
- Whether batch submissions stay within the 10,000 URL limit.
- Whether submitted URLs are fresh changes rather than full-site recrawl spam.
References
Source: lib/checks/indexnow-key/versions/1.0.0/docs.md
6. Version Changelog
indexnow-key v1.0.0 Changelog
Initial versioned package for indexnow-key.
- Documents official IndexNow key format, key file placement, submission API, response codes, and URL batch limits.
- Documents scanner-supported nonstandard key-location hints separately from the official protocol.
- Declares package scoring steps for key-location discovery and key-file validation.
- Validates that advertised key files use a text-compatible content type when one is supplied.
- Documents the implemented BOM/whitespace tolerance used when reading key-file bodies.
Source: lib/checks/indexnow-key/versions/1.0.0/changelog.md