How to Create and Post an llms.txt File to Control AI Access to Your Website

With AI-powered tools like ChatGPT, Claude, and Gemini scanning the internet for information, more site owners are asking: How can I control what content large language models (LLMs) can see and use? The answer lies in a new, emerging standard: llms.txt.

In this guide, we’ll show you exactly what llms.txt is, how to write it, and how to post it properly on your website to manage your site’s exposure to generative AI.

---

What Is A llms.txt File?

llms.txt is a plain text file placed at the root of your domain (like https://yourdomain.com/llms.txt) that tells large language models what content they are allowed—or not allowed—to access for training or content generation.

It works similarly to robots.txt, which controls search engine bots, but this one is for LLMs.

---

Step-by-Step: How to Create an llms.txt File

1. Open a Text Editor

Use a simple text editor like Notepad (Windows), TextEdit (Mac), or VS Code.

2. Define Your Rules

Here’s a sample llms.txt that allows LLMs to access your public content but blocks sensitive areas:

# llms.txt for example.com

User-Agent: *
Allow: /blog/
Allow: /about/
Allow: /services/
Allow: /portfolio/

Disallow: /admin/
Disallow: /checkout/
Disallow: /cart/
Disallow: /my-account/
Disallow: /privacy/
Disallow: /terms/

# Specific model access
User-Agent: ChatGPT
Allow: /

User-Agent: Gemini
Disallow: /

# Contact info
Contact: https://example.com/contact/

User-Agent: Refers to the LLM or crawler name. Use * to apply rules to all LLMs.

---

Step 3: Upload the llms.txt File

Upload the file to the root directory of your website, just like you would with robots.txt.

For WordPress:

• Use your hosting file manager or FTP.
• Place llms.txt in the public_html directory.

For other platforms:

• If using Laravel, Next.js, or a custom CMS, ensure your server serves static files from /public or root.
• Test by visiting https://yourdomain.com/llms.txt in your browser.

---

Step 4: Test & Monitor

While llms.txt is voluntary (as of now), many LLM providers have stated they will respect it going forward. Here’s what you can do:

• Visit the file URL directly to confirm it’s accessible.
• Use analytics tools to watch for traffic from known LLM agents (e.g., ChatGPT-User, AnthropicBot).

---

Bonus: Why Use llms.txt?

• Protect private or sensitive areas from AI usage
• Maintain content control while still benefiting from brand exposure
• Support ethical content usage as AI evolves

---

Should You Block All AI Access?

If you’re concerned about content being used by AI models, you can block everything:

User-Agent: *
Disallow: /

But keep in mind: blocking everything means less visibility in AI-powered tools and assistants.

---

Summary

The llms.txt file gives you a voice in the AI age. Whether you’re a business, publisher, or developer, it’s a simple but powerful way to say: Here’s what I allow—and here’s what I don’t.

Taking 5 minutes to create one could help protect your content and steer how AI tools represent your brand.