migrate_staging_table

A Drupal module providing support for using database tables as migration sources, destinations and for data lookups.

Overview

This module provides a generic method to migrate data into, out of, and lookup from a database staging table. It's useful for complex migrations that need an intermediate staging area for data transformation and lookup.

Requirements

• Drupal 9, Drupal 10 or Drupal 11
• Migrate module (core)

Installation

composer require drupal/migrate_staging_table
drush en migrate_staging_table

Usage

Provided Plugins

Source Plugin: staging_table
Use a database table as a source for migrations.

source:
  plugin: staging_table
  table: my_staging_table
  fields:
    - id
    - field1
    - field2
  conditions:
    - column: status
      value: 1
      operator: '='

Destination Plugin: staging_table
Write migration data to a database table.

destination:
  plugin: staging_table
  table: my_staging_table
  fields:
    field1:
      type: varchar
      length: 255
    field2:
      type: text
    field3:
      type: int
  indexes:
    field1_index:
      - field1
  description: "My staging table for migration"

Process Plugin: staging_table_lookup
Lookup values from a staging table during migration.

process:
  field_destination:
    plugin: staging_table_lookup
    source: source_id
    table: my_staging_table
    field: field_to_retrieve

Configu

ration
Each plugin requires specific configuration parameters:

table: Name of the database table to use
fields: Field definitions for the table (for destination plugin) or field names to select (for source plugin)
conditions: (Source only) Optional conditions to filter source data

Notes

The module automatically manages the creation of tables for the destination plugin.
Each table automatically includes id and created fields
When a table already exists, the module will only add new fields, not modify existing ones.