tdmrep 1.0.0

TDMRep Check v1.0.0

Status

• Version: 1.0.0
• Check identifier: tdmrep
• Input contract: [email protected]
• Output contract: [email protected]
• Scope: site

Abstract

This check validates Text and Data Mining Reservation Protocol declarations and linked TDM policy documents. Version 1.0.0 recognizes the well-known /.well-known/tdmrep.json file plus resource-level tdm-reservation and tdm-policy declarations in HTTP headers and HTML metadata.

When TDM rights are reserved and a tdm-policy URL is declared, this version fetches the policy. JSON and JSON-LD policies receive full ODRL/TDMRep semantic validation. HTML policies are treated as human-readable policy evidence because TDMRep defines text/html for human-readable policies, not for machine-readable ODRL validation.

Motivation

TDMRep lets rightsholders declare whether text and data mining rights are reserved for applicable content. It is a rights and usage policy signal, not a generic crawl access rule and not a content-quality signal.

The declaration is only useful when machines can discover and validate both the reservation value and the policy terms for reserved content.

Normative Model

This version recognizes these declaration surfaces:

• /.well-known/tdmrep.json.
• tdm-reservation HTTP response header.
• tdm-policy HTTP response header.
• <meta name="tdm-reservation" content="...">.
• <meta name="tdm-policy" content="...">.

tdm-reservation values:

• 1: TDM rights are reserved.
• 0: TDM rights are not reserved.

tdm-policy values must resolve to HTTP(S) URLs. Origin-relative paths are accepted and resolved against the scanned origin.

TDM policy documents can be served in these TDMRep-recognized forms:

• application/json or application/ld+json for machine-readable policy data.
• text/html for human-readable policy pages.

Only JSON and JSON-LD policy bodies are semantically validated as ODRL/TDMRep policy data in this version. RDFa, Microdata, and embedded JSON-LD inside HTML may be reported as non-standard structured evidence, but they do not satisfy the machine-readable ODRL validation gate.

Applicability

The check applies to every scanned site. It looks for TDMRep declarations in the well-known file, response headers, and HTML metadata.

If no declaration is present and /.well-known/tdmrep.json is not found, the result is warning.

Pass Criteria

• At least one TDMRep declaration exists.
• All declared reservation values are 0 or 1.
• Declared policy URLs are valid HTTP(S) URLs or origin-relative paths.
• If tdm-reservation is 1 and tdm-policy is declared, every fetched JSON

or JSON-LD policy passes ODRL/TDMRep semantic validation.

Warning Criteria

• No TDMRep declaration is found at /.well-known/tdmrep.json, in response

headers, or in HTML metadata.

• tdm-reservation is 1 but no tdm-policy URL is declared.
• Multiple declaration surfaces disagree on reservation values.
• A declared tdm-policy URL resolves to a meaningful human-readable HTML

policy page instead of machine-readable JSON/JSON-LD policy data.

• A semantically valid ODRL policy omits non-fatal publisher/license details

such as assigner, duty, or constraint.

• tdm-policy is present with tdm-reservation: 0, because the policy is

ignored for non-reserved content.

Failure Criteria

• /.well-known/tdmrep.json is reachable but not parseable JSON.
• /.well-known/tdmrep.json is reachable but is not served with a JSON-compatible

Content-Type, such as application/json, application/ld+json, or another +json type.

• The well-known file is not a JSON array of rule objects.
• A rule is missing location or tdm-reservation.
• tdm-reservation is not 0 or 1.
• tdm-policy is malformed.
• A declared policy URL is unreachable, empty, or served as an unsupported

content type.

• A JSON/JSON-LD policy is not parseable JSON.
• A JSON/JSON-LD policy lacks a semantically valid ODRL/TDMRep policy.
• An HTML policy page is reachable but too thin to be useful as human-readable

policy evidence.

Evidence Model

The result emits:

• Well-known fetch status, content type, length, and parsed declarations.
• Header and HTML metadata declarations.
• Declaration validation errors and warnings.
• Linked policy fetch status, content type, format, and semantic validation

output.

• JSON/JSON-LD policy candidates with ODRL type, id, profile, and rule presence

summaries.

• Non-standard HTML structured evidence signals, when present, such as embedded

JSON-LD, RDFa attributes, or Microdata attributes.

Validation And Scoring Steps

Standard Behavior

For reserved TDM content, publish a TDMRep declaration with tdm-reservation: 1 and a tdm-policy URL. The policy should be a structured ODRL/TDMRep JSON or JSON-LD policy with:

• An ODRL policy type such as Offer, Agreement, Set, or Policy.
• uid or @id.
• The TDMRep profile or vocabulary.
• At least one permission, prohibition, obligation, or duty.
• A rule with action tdm:mine.
• Valid duties and constraints when present.

Non-Standard And Real-World Behavior

This version accepts reachable, meaningful HTML policy pages as human-readable TDM policy evidence and reports them with a warning because the scanner cannot machine-validate their ODRL semantics. RDFa, Microdata, or embedded JSON-LD in HTML are reported only as non-standard evidence; they are not TDMRep-specified machine-readable policy formats.

Non-Goals And Limitations

• This check does not decide legal enforceability.
• This check does not verify crawler compliance.
• This check does not validate EPUB or PDF embedded TDM metadata.
• This check does not replace robots.txt crawl access checks.
• This check does not consume sibling check output from Content Signal, ai.txt,

RSL, Web Bot Auth, or AI bot rules.

References

• www.w3.org/2022/tdmrep
• www.w3.org/TR/odrl-model
• www.w3.org/TR/odrl-vocab
• www.iana.org/assignments/well-known-uris/well-known-uris.xhtml

tdmrep v1.0.0 Changelog

Initial versioned package for tdmrep.

Migrated v1.0.0 runtime ownership into the versioned package. The check now validates TDMRep well-known declarations, resource-level headers and HTML metadata, linked tdm-policy reachability, JSON/JSON-LD ODRL/TDMRep policy semantics, and human-readable HTML policy evidence.