August 4, 2026

Is Your Robots.txt Blocking ChatGPT From Ever Citing You?

Here’s an uncomfortable scenario: a prospective student asks ChatGPT to recommend a course in your niche. Your site has the best answer. And ChatGPT can’t mention you — because a file you’ve never opened told it to stay out.

That file is robots.txt, and in our audit of 102 live Kajabi sites we found sites blocking AI crawlers without their owners ever having made that decision.

The 30-second check

Open a browser tab and go to yourdomain.com/robots.txt. You’re looking for lines like these:

User-agent: GPTBot
Disallow: /

Translation of the bot names you might see:

• GPTBot — OpenAI’s crawler. Blocking it removes you from what ChatGPT can read.
• ClaudeBot — Anthropic’s crawler, feeding Claude.
• PerplexityBot — Perplexity, the AI search engine.

Disallow: / under any of those user-agents means that assistant is told not to read your site. And a broader trap: User-agent: * followed by Disallow: / blocks everyone — including Google.

Why this happens by accident

Robots.txt files accumulate. A template ships with defaults, a developer copies a config from a different project, a “block bad bots” snippet from a forum gets pasted in during a redesign — and two years later the file quietly excludes the exact crawlers deciding whether AI assistants can recommend you. Nobody chose it. Nobody noticed.

There’s a legitimate debate about whether to let AI systems read your content. But it should be a decision you make, not a default you inherit — and for a course creator whose business depends on being discovered, the case for being readable is strong: AI assistants are becoming a real referral channel, and they can only refer businesses they can see.

How to fix access

If your robots.txt is managed through your DNS or hosting layer (common with custom domains on Cloudflare or similar), remove the disallow rules for the AI user-agents you want to allow — or simply make sure no rule targets them, since no rule means access is allowed by default. If you can’t locate where the file is generated, that’s worth one support ticket, because everything downstream depends on it.

While you’re in there, confirm the file references your sitemap:

Sitemap: https://yourdomain.com/sitemap.xml

Check it without reading a single directive

The free CourseFound Chrome extension runs this exact check — plus 25 more — against any page on your Kajabi site. If robots.txt is blocking ChatGPT from ever citing you, it says so in plain English, tells you which crawlers are affected, and shows you what to change.

It’s the first thing we’d check on any course site in 2026. It takes ten seconds, and the answer is occasionally business-critical.