External Entities Cache Warmer

The External Entities Cache Warmer module solves a common performance problem for sites that use the External Entities (https://www.drupal.org/project/external_entities) module: data fetched from remote APIs is only cached the first time a visitor (or a cron run) loads each entity. On large datasets this means cold-start pages are slow, and clearing the cache wipes everything at once with no easy way to rebuild it proactively.

This module adds a dedicated cache-warming pipeline that pre-populates Drupal's persistent entity cache for any External Entities type, so your remote data is always served from the local cache rather than hitting the remote API on every request.

Features

• Per-type opt-in: enable or disable the cache warmer individually for each External Entities type from the type's edit form (Caching tab). No global switch that affects unrelated types.
• Configurable batch size: override the number of entities fetched per remote page for each type, or leave it blank to inherit the value already configured in the storage client's Pager settings.
• Queue-based warming for large datasets: the xntt:cw Drush command populates a Drupal queue, then run a process similar to drush queue:run, external_entities_cache_warmer processes it in disposable subprocesses. Each subprocess starts with a clean PHP memory state, making it safe to warm hundreds of thousands of entities without running out of memory. If the main process is interrupted, the queue remains populated and the next xntt:cw call resumes from where it stopped.
• Cache-status overview: the External Entities type list page gains two extra columns: Cache warmer (Yes/No) and Cached items (N / total at N%), giving a quick fleet view without opening each type.
• One-click cache clear: a "Clear entity cache" button on the edit form invalidates the type's cache tags and resets the progress counters, ready for the next warm run.

Post-Installation

1. Go to Administration > Structure > External entity types (/admin/structure/external-entity-type). You will see two new columns on the list: Cache warmer and Cached items.
2. Click Edit on any External Entities type that has persistent caching enabled (the Maximum cache age must be set to a value greater than zero in its Caching tab).
3. On the Caching tab, check Enable cache warmer and optionally set a Batch size override, then save.
4. Run the Drush warm command from your site root: drush xntt:cw
5. Check the status at any time: drush xntt:cs

Drush commands

external-entities:cache-warm
Warms the persistent cache for one or all enabled external entity types.

drush xntt:cw location  # warm a single type
drush xntt:cw           # warm all enabled types
drush xntt:cw --all     # warm every type, enabled or not

external-entities:cache-enqueue
Counts the remote items, resets the progress counters, and pushes offset-based chunks into the Drupal queue without processing them. Use this when you want to separate enqueueing from consumption (e.g. schedule queue:run via cron separately).

drush xntt:eq location  # enqueue a single type
drush xntt:eq           # enqueue all enabled types
drush xntt:eq --all     # enqueue every type, enabled or not

external-entities:cache-clear
Clears the persistent cache for one or all external entity types by invalidating cache tags, and resets the cached-item progress counter.

drush xntt:cc location  # clear a single type
drush xntt:cc           # clear all enabled types
drush xntt:cc --all     # clear every type, enabled or not

external-entities:cache-status
Lists all external entity types with their cache warmer status.

drush xntt:cs                   # table output
drush xntt:cs --format=json     # JSON for scripting

Additional Requirements

• External Entities (https://www.drupal.org/project/external_entities): this module is a companion to External Entities and cannot be used without it.
• The External Entities type being warmed must have persistent caching enabled (a non-zero Maximum cache age set in the Caching tab of the entity type). Without persistent caching, there is nothing to warm.
• Ultimate Cron or a system cron job to schedule drush xntt:cw (or drush queue:run external_entities_cache_warmer) on a recurring basis so the cache is refreshed automatically after it expires.

Similar projects

Cache Warmer (https://www.drupal.org/project/cache_warmer) warms Drupal page/render caches by crawling URLs. It works at the HTTP layer (warming page responses) rather than at the entity layer, so it does not target the External Entities persistent entity cache specifically.