AI Migration

Introduction

The AI Migration module is under active development. Please let us know in the issue queue if you are interested in participating.

This module helps with migrating content into Drupal using AI to parse web pages based on your Drupal content type schema, generating migrations, that migrate the content creating Drupal nodes.

Goals

• To make it easier to parse fairly unstructured content from web pages or documents and transform it into structured content in Drupal. "Fill the fields, not bloat the body."
• To make it easy to migrate slices of content into a Drupal instance to support phased migration of legacy sites into Drupal sites.
• Spend less time tuning migrations and more time migrating.

Requirements

This module requires:

• drupal/ai:^1.2
• drupal/document_loader_html_processor:^2.0 (brings in document_loader and html_processor)
• drupal/jsonapi
• drupal/migrate:^6.0
• drupal/migrate_plus:^6.1
• drupal/serialization
• drupal/schemata:^1.0
• drupal/schemata_json_schema

Recommended Models

We recommend using these models with AI Migration:

• OpenAI GPT-4.1-mini
• OpenAI GPT-5-nano
• Gemini 2.5 Flash

We are currently addressing compatibility issues with Anthropic Claude.

Installation & Configuration

1. Install the module as you would any other module.
2. Configure an AI API provider at /admin/config/ai/providers.
3. Set the default AI API provider for Chat and Chat with Structured Response at /admin/config/ai/settings.
4. Create a Drupal content type with fields that you want to populate with migrated content.
5. Create a migration yml file in your custom module. See the simple_content_migration.yml file in the ai_migration_example submodule for an example.
6. Execute the migration to migrate content.

HTML Processing

The html_processor: key in a migration YAML controls how raw HTML is cleaned before it is sent to the AI provider. It is processed by the document_loader_html_processor module.

ai:
  html_processor:
    # Extract a specific region of the page by CSS selector.
    container: [ 'article', '#main-content' ]
    # Strip matched patterns before sanitizing (optional).
    strip_regex:
      - |-
        /]*>.*?<\/script>/is
    # Symfony HtmlSanitizer options (nested under 'sanitizer').
    sanitizer:
      allowStaticElements: true
      allowRelativeLinks: true
      dropAttribute:
        - [ 'style', [ '*' ] ]

Supported top-level keys: container, strip_regex, remove_ads, base_url, sanitizer, output_full_document, minify. See the document_loader_html_processor README for the full options reference.

minify: true collapses whitespace to reduce AI token count. Only add it when the processed HTML goes directly to the AI provider (minified HTML breaks Markdown converters and other downstream processors that depend on whitespace structure).

Breaking change: sanitizer options now nested under sanitizer:

Prior to the document_loader_html_processor integration, Symfony HtmlSanitizer method names (allowStaticElements, allowSafeElements, allowElement, allowRelativeLinks, dropAttribute, etc.) were placed directly at the top level of html_processor:. They must now be nested under a sanitizer: key.

# Before (no longer works—sanitizer options silently ignored)
html_processor:
  container: [ '.main' ]
  allowStaticElements: true
  dropAttribute:
    - [ 'style', [ '*' ] ]

# After
html_processor:
  container: [ '.main' ]
  sanitizer:
    allowStaticElements: true
    dropAttribute:
      - [ 'style', [ '*' ] ]

container and strip_regex remain at the top level and are unchanged.

AI Migration Example

Provides a sample migration that demonstrates AI Migration functionality. The submodule creates a content type called 'Simple Content Migration' that has fields that can be populated by the simple_content_migration.yml migration. The simple_content_migration.yml migration is a good starting point for creating your own migrations. The migration targets CA example website https://accessibility.civicactions.com/ and converts blog posts into Drupal nodes.

Caching

The module creates a cache bin where combinations of AI responses keyed by provider-model-prompt are stored. This way, identical queries to the same model will not incur additional costs from the provider. Keep in mind that clearing all caches also clears out the AI Migration cache.

Developers and Contributors

See CONTRIBUTING.md for details on how to set up a local development environment and contribute.

More Documentation

See our additional documentation for more details.