ymca_sync

YMCA Sync provides a framework for defining and running synchronization tasks or batch processes via Drush commands, ensuring that tasks do not run concurrently, preventing race conditions and data conflicts.

Features

The YMCA Sync module offers a structured way to manage and execute potentially long-running or resource-intensive tasks, primarily designed for data synchronization or other batch operations that should not overlap.

• Syncer Definition:
  Allows developers to define custom "syncers" as Drupal services. Each syncer can consist of multiple steps, where each step typically calls a method on another service.
• Drush Integration:
  Provides Drush commands to interact with syncers:
  • drush yn-sync <syncer_service_name>: Executes a specific, activated syncer.
  • drush yn-sync:list: Lists all defined syncers discovered in the system and shows whether they are active or disabled.
  • drush yn-sync:enable <syncer_service_name>: Activates a specific syncer, making it runnable.
  • drush yn-sync:disable <syncer_service_name>: Deactivates a specific syncer.
• Race Condition Prevention:
  Automatically utilizes Drupal's lock system. When a syncer is executed via drush yn-sync, it attempts to acquire a lock specific to that syncer. If the lock is already held (meaning another instance of the same syncer is running), the new execution attempt will be aborted, preventing concurrent runs and potential data corruption.
• Configuration UI:
  Provides an administrative interface (/admin/config/system/ymca-sync) where site administrators can view available syncers and activate or deactivate them using checkboxes.
• Extensibility:Modules can define their own syncers by creating tagged services and can provide descriptions for their syncers on the settings page using hook_ymca_sync_description_alter().

Use Cases:

• Synchronizing data from external APIs or sources on a schedule (e.g., via cron calling the Drush command).
• Running complex data migrations or updates as distinct, manageable tasks.
• Performing periodic cleanup or processing tasks that need exclusive access to resources.
• Any scenario where distinct, potentially long-running command-line tasks need to be defined and executed without overlapping.

Post-Installation

After installing the YMCA Sync module:

1. Developers: Define custom syncers by creating services tagged with syncer in their module's services.yml file, as shown in the module's README.md. Clear Drupal's cache after adding new syncers.
2. Administrators:

• Navigate to the configuration page at Administration > Configuration > System > YMCA Sync Settings (/admin/config/system/ymca-sync).
• On this page, you will see a list of all syncers discovered in the system.
• Use the checkboxes to activate the syncers you want to be able to run. Only active syncers can be executed.
• Alternatively, use the Drush commands drush yn-sync:enable <syncer_name> and drush yn-sync:disable <syncer_name> to manage syncer activation status.

3. Execution:

• Run an active syncer using the Drush command: drush yn-sync <syncer_name>.
• List available syncers and their status: drush yn-sync:list.

The module itself doesn't provide any syncers out-of-the-box; it provides the framework for other modules to define and run them safely.

Additional Requirements

• Drupal Core: Depends on core services like Lock, Logger, Config Factory, and the Service Container.
• Drush: Required for executing syncers and using the command-line management commands (list, enable, disable, sync).

Recommended modules/libraries

There are no specific recommended modules, but this module is intended to be used by other custom modules that need to implement synchronization or batch tasks.

Similar projects

• Drupal Core Queue API:The Queue API is designed for handling a large number of small tasks asynchronously in the background, typically processed by cron or a dedicated queue worker. YMCA Sync is different as it focuses on defining distinct, named, potentially long-running synchronous tasks executed via Drush, with built-in locking to prevent concurrent execution of the same named task. It's not designed for distributing work across multiple workers like the Queue API.
• Drupal Core Batch API: The Batch API is primarily for breaking down long-running processes into smaller chunks manageable via HTTP requests, typically initiated from the user interface, providing user feedback. YMCA Sync is focused on command-line execution via Drush and preventing concurrency of the entire named task, rather than breaking a single task into UI-manageable steps.

YMCA Sync provides a specific solution for managing named, lockable Drush commands, which can be simpler for certain use cases than setting up Queue workers or Batch API processes, especially when the primary need is just to run a defined script via cron/CLI without overlap.

Supporting this Module

Module supported by YMCA Website Services distribution community

Community Documentation

https://ds-docs.y.org