Skip to main content
drupalreleases

Use direct release data with your AI assistant.

Learn more

Migrate Source Directory

Security covered Under active development 230 sites

This module allows you to import files or content from a directory into Drupal. You can configure it to import files as Drupal file entities or to process the content of files (like Markdown) to create other Drupal entities, such as nodes.

Use a directory of files as the source for a migration.

Common use cases include:

  • Import a set of files from a directory as Drupal file entities
  • Find all Markdown files in a directory and create a content entity for each one

Source plugin configuration options:

source:
  plugin: directory
  track_changes: true
  # One or more directories to scan for files. Will recurse into subdirectories.
  directory:
    - /path/to/files/for/import
  # (optional) A regex mask to apply to file paths, relative to the directory(s)
  # above. Only files that match this pattern will be processed. Use to do
  # things like only process files with a given extension.
  file_mask: '/(.*\.(mp3|m4a|wav)$)/i'
  # (optional) How deep to recurse into subdirectories. -1 (default) for no
  # limit.
  recurse_level: -1
  # (optional) prefix to add to the sourceID when processing files. The default
  # for sourceID is the path to the file relative to the directory set above.
  id_prefix: 'optional_sourceid_prefix'
  # (optional) List of one or more strings that a file must contain in order
  # to be considered for processing. This probably only really works with text
  # like files since it uses strpos().
  file_must_contain_string:
    - '<div class="chapter">'

This plugin provides the following source fields:

  • sourceID: Unique ID of the file (filepath)
  • source_file_basename: Base name of the file
  • source_file_extension: File extension
  • source_file_filename: Filename and extension
  • source_file_mtime: Time the file was last updated
  • source_file_path: Path without filename
  • source_file_pathname: Path to the file
  • source_file_realpath: Absolute path to the file
  • source_file_size: File size
  • source_file_type: File type

Create Drupal File entities from a directory of files

Use the plugin like so to import files in to Drupal:

id: directory_mp3
label: 'Import audio files'
source:
  plugin: directory
  constants:
    uri_file: 'public://'
  track_changes: true
  directory:
    - /path/to/files/for/import
    - /path/to/additional_files
  # (optional) A regex mask to apply to file paths, relative to the directory(s)
  # above. Only files that match this pattern will be processed. Use to do
  # things like only process files with a given extension.
  file_mask: '/(.*\.(mp3|m4a|wav)$)/i'
  # (optional) How deep to recurse into subdirectories. -1 (default) for no
  # limit.
  recurse_level: -1

process:
  source_full_path: source_file_path
  uri_file:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/uri_file
        - source_file_filename
    -
      plugin: urlencode
  filename: source_file_filename
  uri:
    plugin: file_copy
    source:
      - '@source_full_path'
      - '@uri_file'

destination:
  plugin: 'entity:file'

migration_dependencies:
  required: {  }
  optional: {  }

To reference these files in other migrations, use the source property sourceID. This is the path to the file relative to source.directory.

Create Drupal content entities from a directory of files

You can also use this to iterate over a set of files, and import the content of those files as Drupal content entities. For example; a directory of Markdown files could be imported as nodes with a migration like this:

id: markdown_import
id: markdown_import
label: 'Import markdown files'

source:
  plugin: directory
  track_changes: true
  directory:
    - /path/to/files/for/import
  # All files with a .md extension, excluding README.md files.
  file_mask: '/^(.*\.md)(?<!README\.md)$/i'
  recurse_level: -1

process:
  type:
    plugin: default_value
    default_value: "basic_page"
  title: title
  body/value: body
  body/format:
    plugin: default_value
    default_value: "markdown"
  changed: last_updated

destination:
  plugin: 'entity:node'

migration_dependencies:
  required: {  }
  optional: {  }

And either a corresponding implementation of hook_migrate_prepare_row(), or a custom source plugin class that extends \Drupal\migrate_source_directory\Plugin\migrate\source\Directory and implements the required logic in its own prepareRow() method.

Example:

function hook_migrate_prepare_row(Row $row, MigrateSourceInterface $source, MigrationInterface $migration) {
  if ($migration->id() == 'markdown_import') {
    $content = file_get_contents($row->getSourceProperty('source_file_realpath'));

    // The first line of the file, if it starts with `#` is the title.
    $title = strtok($content, "\n");
    if (substr($title, 0, 1) == '#') {
      $row->setSourceProperty('title', trim(substr($title, 1)));

      // Remove the first line (we're done with it), and any empty newlines
      // above/below the resulting content.
      $content = trim(substr($content, strpos($content, "\n") + 1));
    }

    // And the rest is the body.
    $row->setSourceProperty('body', $content);
  }
}
Drupal 9–11

Ask your assistant about Migrate Source Directory

Check compatibility with your Drupal and PHP version, maintenance and security coverage, from current release data. How it works

Depends on

Dependencies of the latest stable release

  • migrate Drupal core

Required by

Tracked projects that depend on this one

No tracked projects depend on this one yet.

Activity

Tracked releases
2
Tracked since
Jun 2023
Latest release
2 years ago
Releases (12 mo)
0
Maintenance
Slowing

Releases

Version Type Core PHP Release date
2.2.0 Stable 9–11 Jul 8, 2024
2.1.0 Stable 8–10 Jun 15, 2023