Blog
5 min read
How to let GPTBot, ClaudeBot and PerplexityBot read your site
A practical guide to the AI crawler user-agents, how to allow or block each one in robots.txt, and how to verify they can actually reach your content.
Read more →4 min read
What is llms.txt, and why your site needs one
llms.txt is the AI-era robots.txt: a plain-text map that tells models what your site is and where the good content lives. Here's what it is and how to add one.
Read more →