Mediapartners-Google · AdsBot Wildcard Rules · /ads.txt Reachability Audit
Paste your site's robots.txt to verify whether Mediapartners-Google, AdsBot-Google, and programmatic /ads.txt crawlers are permitted or accidentally blocked.
Mediapartners-Google and AdsBot-Google Differ from GooglebotGoogle's ad-serving crawlers do not follow the exact same wildcard inheritance rules as standard web search crawlers, which causes frequent production bugs after staging migrations:
Mediapartners-Google (AdSense Contextual Crawler): Falls back to User-agent: * when not explicitly named. Leaving Disallow: / under * blocks AdSense contextual matching unless you add an explicit User-agent: Mediapartners-Google + Allow: / group.AdsBot-Google & AdsBot-Google-Mobile: Per Google's crawler specification, AdsBot ignores wildcard User-agent: * rules and must be named explicitly if you intend to restrict or permit paths./ads.txt & /app-ads.txt: Blocking root seller files in robots.txt stops Google and SSP crawlers from verifying your pub- authorization.Diagnostic checks reflect public crawler specifications as technical publisher guidance, not a Google approval guarantee.
Blocking Mediapartners-Google prevents Google AdSense from reading page content for contextual ad targeting, often causing blank ad slots, generic low-CPM fallback ads, or site review delays.
No. Google documents that AdsBot-Google and AdsBot-Google-Mobile ignore wildcard (*) groups; only an explicit User-agent: AdsBot-Google block affects them.
Yes. If your robots.txt disallows /ads.txt or the root path for Google crawlers, AdSense cannot verify authorized seller lines and may display an Earnings at risk warning.
No. Unblocking crawlers is a technical prerequisite so reviewers and bots can fetch your pages, while site approval depends on Google's content and policy review.