Validate Last-Modified header [fail]! No valid Last-Modified HTTP header was found.INFOValidate Last-Modified headerINFOReading Last-Modified response headerFAILCheck Last-Modified header is present and parseable actual="missing" expected="valid" alternateDates=[]FAILLast-Modified header is missing or invalidValidate structured freshness dates [fail]! No Schema.org freshness dates were found.INFOValidate structured freshness datesINFOCollecting Schema.org freshness dates from JSON-LD, Microdata, and RDFaFAILCheck structured freshness date count actual=0 expected="> 0" invalidDateCount=0 formats={} alternateDates=[]FAILNo structured freshness dates foundValidate metadata freshness dates [warning]! No Open Graph, Dublin Core, or generic meta freshness date was found.INFOValidate metadata freshness datesINFOCollecting Open Graph, Dublin Core, and generic meta freshness datesWARNCheck metadata freshness date count actual=1 expected="> 0" metaDateCount=1 invalidDateCount=1 alternateDates=[]WARNMetadata freshness dates include invalid values invalidDateCount=1Corroborate with sitemap lastmod [warning]INFOCorroborate with sitemap lastmodINFODiscovering sitemap candidates and matching the scanned page URL candidateCount=2 attempts=2WARNCheck matching sitemap lastmod actual="no match" expected="valid match when available" robotsStatusCode=200WARNNo matching sitemap lastmod found after sitemap attempts attempts=2Check date validity and consistency [fail]! meta date is not a parseable date (20250206).INFOCheck date validity and consistencyINFOChecking collected freshness dates for invalid values, future dates, and inconsistent orderingFAILCheck critical freshness issue count actual=1 expected=0 warningIssueCount=0 issues=[{"severity":"fail","message":"meta date is not a parseable date (20250206)."}]FAILInvalid freshness date issue found issues=["meta date is not a parseable date (20250206)."]

Validate transport [fail]! ai.txt appears to be an HTML page or error document rather than a plain-text policy.INFOValidate transportINFOChecking ai.txt media type, size, line count, and whether the response looks like plain text contentType="text/html; charset=utf-8" length=33563 lineCount=17FAILCompare media type to preferred text/plain transport actual="text/html" expected="text/plain preferred; readable text required"FAILai.txt appears to be an HTML page or error document rather than a plain-text policy.Parse ai.txt policy [fail]! Transport validation failed before policy parsing could be trusted.INFOParse ai.txt policyINFODetecting ai.txt convention from bracket sections, frontmatter, headings, and ai.txt context linksINFODetected policy structure convention="unknown-text" sections=[] markdownHeadings=[]WARNCheck AI Visibility required sections actual="missing identity, permissions, restrictions" expected="identity, permissions, and restrictions" missingRecommended=["attribution","contact","content-types"]WARNTransport validation failed before policy parsing could be trusted.Validate policy content [fail]! ai.txt appears to expose secrets, tokens, or credentials.INFOValidate policy contentINFOChecking required section content, permission/restriction language, attribution, contact details, and safety risks convention="unknown-text"INFODetected policy signals hasPermissionLanguage=false hasRestrictionLanguage=false hasTrainingLanguage=false hasAttributionLanguage=true hasContact=falseFAILCheck required and recommended section coverage actual={"missingRequired":["identity","permissions","restrictions"],"missingRecommended":["attribution","contact","content-types"]} expected={"missingRequired":[],"recommendedPresentWhenPossible":true}FAILai.txt appears to expose secrets, tokens, or credentials. possibleSecrets=["/ai.txt: Public metadata appears to expose a secret detected by Betterleaks. match=\"generic-api-key\" snippet=\"42dd...4d23\""]

Classify AI crawler rules [fail]! No explicit User-agent rules were found for major AI crawler tokens.INFOClassify AI crawler rulesINFOParsing User-agent groups and Allow/Disallow records for known AI crawler tokens evaluatedPath="/"INFOEvaluating exact User-agent matches before wildcard fallback exactAiPolicyCount=0 totalCrawlerTokens=18FAILNo explicit AI crawler User-agent groups were found examplesExpected=["GPTBot","OAI-SearchBot","ClaudeBot","Google-Extended","CCBot"]FAILCompare explicit AI crawler coverage actual=0 expected="> 0 explicit non-search AI crawler policies" missingTokens=["GPTBot","OAI-SearchBot","ChatGPT-User","ClaudeBot","Claude-SearchBot","Claude-User","Google-Extended","Applebot-Extended","Amazonbot","Amzn-SearchBot","Amzn-User","PerplexityBot"]INFOResolved effective root-path policy for crawler tokens blocked=0 allowed=21 unspecified=0