batch_plugin

Batch API, Cron jobs and queues, and Drush queues. Three different methods to process large amounts of data in Drupal, and quite frankly quite a lot of hassle and coding to achieve something that should be very simple.

Batch Plugin is the "one ring to rule them all" solution to processing large amounts of data in Drupal.

Batch plugin addresses the need for a quick and simple way of achieving all of the above and more with a few lines of code, and more importantly allows developers to use exactly the same plugin to run Batch API jobs, Cron jobs and queues, and drush queues.

Batch plugins can also be configurable allowing a UI driven approach. For example, selecting nodes from a certain data range, or content types, etc. Whatever you need really. Or you can just pass the data your plugin needs as an argument to help select the proper items to process.

If your batch plugins do need configuration, just implement the configuration methods you'd find on any other plugin. From the front end, they can be UI driven from either the batch_plugin_entity sub module, or by using a batch_plugin_config Form API element to embed your plugin's config form somewhere else in your code.

Features

This module provides a batch plugin, with minimal and simple implementations to: -

1. Setup a list of operations
2. Process each operation
3. Do something when all the operations have finished
4. Append more operations during or at the end of the processing
5. Process your plugins with Batch API, Cron jobs and queues, or Drush queues, or all three if you like
6. Provide configuration forms and form elements to make integrating your plugins into any other area of Drupal simple

The code that runs the batch plugin is also plugin driven - processors are plugins, so if a custom processor is required just extend or add a new processor plugin to suit your own needs.

A sub module (batch_plugin_entity) allows developers to add config entities to perform the batch tasks via the UI - again very simple as opposed to creating a new form / integration point each time.

Creating batch plugins

To create a new batch plugin, simply implement 2 methods: -

• setupOperations() - add a list of data payloads to the operations array on the base class. Each array item gets passed as the first argument to the processOperation function. The data needs to be serializable.
• processOperation($payload, ....) - perform the processing for each individual operation.

Check out the batch_plugin_example sub module to see how easy it is to get started.

There are other optional methods should you need a configuration form, etc.

Executing batch plugins

Batch plugins can be called from a function on the plugin, from a UI with the batch_plugin_entity sub-module, from CRON, or from a drush command.

Calling the plugin directly example

  /** @var \Drupal\batch_plugin\BatchPluginInterface $plugin */
  $plugin = \Drupal::service('plugin.manager.batch_plugin')->createInstance($plugin_id);
  $plugin->process();

Or, even easier,

  YourPluginClass::processStatic();

With both approaches you can tell the batch plugin which processor (Batch API, Cron queue, etc.) to use and pass any helpful data as optional parameters, or by setting them using the batch plugin's methods. If you don't pass anything the processing will happen with your plugin's default processor (described in an annotation on the batch plugin class) or Batch API if no processor annotation is present. Again - check out the examples sub module - trying to explain this in English makes it sound way over complicated and it really isn't.

Running the plugin from the UI

• Enable the batch_plugin_entity submodule
• Visit the batch plugin page, under the Structure menu
• Add the plugin, giving the config entity a title and choose a processor
• Save, and then process the plugin

Alternatively use a Form API element in your own code: -

  $form['batch_plugin_config'] = [
    '#type' => 'batch_plugin_config',
    '#plugin_id' => 'your_batch_plugin_id',
    '#plugin_configuration' => [],
    '#show_processor_element' => boolean,
  ];

Processing the plugin from drush example

drush batch_plugin:process your_batch_plugin_id

Batch plugin entity sub module

The entity module creates config entities based on plugins. The plugins can provide configuration forms, for example, for adding operations in a more UI driven way.

This saves having to build additional forms and routes, etc., just to run a batch.

Batch plugin entities can be exported and imported to config as per any other config entity.

Additional developer notes

At its core, Drupal's Batch API and Queue processing serializes the operation data. This means your operation data must be Serializable. Try to keep the data payloads as simple as possible.

Furthermore, Batch API calls static methods. We work around this by creating the plugin from the static method and calling the process or finish method. This means if you store data in properties in the batch plugin, it will be lost with each process function. Either recreate the services, if you need them, with dependency injection in your plugin, or store data in the operation payload. Just remember the operation payload needs to be serializable.

Simple and Complex examples

Examples are provided in the batch_plugin_example sub-module.

Supporting this Module

Please reach out to offer any development support or co-maintenance.

Versions

Version 1 uses annotations for the plugin definition and will no longer be maintained, apart from bug fixes.

Version 2 uses attribute plugin definitions, with fallback to annotations still supported. This is the actively maintained branch, but will require >10.2 for Drupal core when attributes were introduced.