llms_txt

Provides an /llms.txt endpoint for your Drupal site to help large language models and AI agents better understand your website content.

What is llms.txt?

The llms.txt standard is an emerging convention designed to help large language models better understand website content during inference. It works by providing clean, well-organized information in a standardized format. For comprehensive details about this standard, visit llmstxt.org.

In essence, the llms.txt file serves as your homepage for LLMs, AI agents, and other bots, directing them to the content required to understand and properly use your website.

Features

• Flexible content management: Configure content through both code-based config entities and environment-specific database entities
• Permission-based access: Users with "Administer /llms.txt configuration" permission can manage the endpoint content
• Custom tokens: Embed dedicated menus in machine-readable Markdown format within your llms.txt
• Optimized caching: Built-in render caching integration with proper invalidation logic
• Environment flexibility: Balance between generic llms.txt parts (stored in code) and environment-specific content (stored in database)

Installation and setup

1. Install this module following standard Drupal module installation procedures
2. Configure your web server (see Web server configuration below)
3. Navigate to Administration » Content » llms.txt to configure and author your site's llms.txt content

Web server configuration

Important: Before using the module, ensure your web server is configured to serve the /llms.txt file directly. For some development stack-specific instructions, check the DEVELOPMENT.md file.

Web servers often restrict access to .txt files in the web root for security reasons. Since /llms.txt is intended to be public, you must explicitly allow access to it.

Nginx configuration

Add this to your Nginx configuration:

# Enable support for the llms_txt module
location = /llms.txt {
  access_log off;
  try_files $uri @drupal;
}

Apache configuration

For Apache users, refer to your server's documentation to enable direct access to /llms.txt.

Comparison with llmstxt module

This module differs from the llmstxt module (version 1.0.0-alpha1) in several key ways:

Feature llms_txt llmstxt Content storage ✅ Environment-specific content stored in database others in config ❌ Config-only approach (Config Ignore might be necessary) Render caching ✅ Maximized with proper invalidation ⚠️ Basic caching Test coverage ✅ Higher than 90% ❌ None Programming API ❌ No alteration API ✅ Provides alteration hooks