Problem · diagnosis and fix
Ask ChatGPT for a business like yours. Someone else comes up.
AI assistants recommend businesses whose pages they can fetch, read and quote. A site that blocks their crawler, or renders its text with JavaScript, is invisible to them.
14%
refuse AI crawlers such as GPTBot or ClaudeBot at the firewall
RankMeHi study of 662 Orange County business websites. Crawl 25 to 26 Sep 2026; PageSpeed Insights, mobile, 26 Sep 2026.
4
usual causes, each with its own fix below
Each cause is checked in the fix below
Free
audit that tells you which cause is yours, in writing
Every engagement starts with it
Sound familiar?
How it shows up.
- Your competitors are named in AI answers and you are not
- Your firewall logs show blocked requests from GPTBot or ClaudeBot
- The page text only appears after the scripts run
Root causes
The 4 reasons it happens.
- 01
The crawler is blocked
14% of local sites refuse requests from AI crawlers at the firewall, often without the owner knowing.
- 02
The content is not in the HTML
Builders and apps that render the page in the browser leave an empty document for a crawler that does not run scripts.
- 03
Nothing specific to quote
Assistants repeat concrete details: prices, areas, hours, what is included. A page of adjectives gives them nothing.
- 04
No structured data
Schema for the business, its services and its FAQs tells a machine what the page is about without guessing.
The fix
What to do, in order.
- 01
Allow the crawlers
Step 1Firewall and robots rules that let GPTBot, ClaudeBot, PerplexityBot and Google-Extended fetch the pages.
You getA cause you can rule in or out.
- 02
Put the text in the HTML
Step 2Every word a visitor sees should be in the raw page response. Our builds are checked by fetching them as GPTBot.
You getA cause you can rule in or out.
- 03
Add the specifics
Step 3Prices, service areas, hours, licence number, what a job includes. The details an assistant can repeat.
You getA cause you can rule in or out.
- 04
Mark it up
Step 4LocalBusiness, Service and FAQPage schema that matches the visible text.
You getA site that produces enquiries you can count.
Questions
Questions about this problem.
Want the cause in writing, free?
First three approved leads free · One partner per trade per city · You own your domain and content from day one
Start with clarity
Get your free website & SEO audit
- 1Share your name, email and website.
- 2We review your current website and SEO foundation.
- 3We reply with the clearest next steps for your business.
Certified partner
Tell us about your website
It only takes a minute. We use these details to review the right website and reply to the right person.




