LiipGPT

Keeps a LiipGPT search index in sync with your Drupal site: it pushes recrawls when content changes, removes documents nothing links to any more, and tells the crawler what each file is called.

A crawler can only learn what it can see by crawling. Drupal knows three things it cannot: that a page changed a second ago, that a document was deleted, and that a PDF still sitting at its URL is no longer linked from anywhere. This module is the channel for that knowledge.

Features

Three jobs, and a site can take any subset of them:

• Crawl push. Saving a page the site hands over queues a recrawl of its URL.
• Eviction. A bounded cron sweep removes documents nothing links to any more, behind six independent controls.
• File metadata. An authenticated endpoint tells the crawler which media a file URL belongs to, so results carry a readable title instead of a filename.

How it decides what is still used

A document is used when it is reachable from a published node. Reachability is one recursive CTE that descends from every published node over entity_usage edges, through containers such as paragraphs, to whatever they reference. It never climbs, and it never reads paragraphs_item_field_data.parent_id, which Paragraphs leaves pointing at a host the paragraph was removed from.

A document is also named by more than one entity. A PDF is both a file and the media that wraps it, and entity_usage records only one of those, so the module resolves the whole group and asks each question of the member that can answer it. Without that, ordinary editing arms a delete of the document it just saved.

Safety

Deleting from a search index is irreversible on the far side, so the module is built to fail toward keeping. Every ambiguous answer resolves to "still used": a refused query, a missing entity, an unresolvable URL. Six controls must all pass before anything is deleted, each sufficient to refuse on its own, and drush liipgpt:eviction-preview prints exactly what a sweep would send without sending any of it.

The API key is the environment boundary. A key whose value comes from the environment or a file cannot travel inside a database dump, so a staging site restored from production sends nothing. Eviction refuses outright when the key comes from config or state, because then there is no boundary at all.

Post-Installation

Configure the tenant endpoint, the API key and the site's public base URL at Administration › Configuration › Web services › LiipGPT, and list the bundles whose saves should trigger a recrawl.

There is no "enable eviction" checkbox. Installing the module gives you the crawl push; running drush liipgpt:backfill is the deliberate act that admits the destructive half, because it writes the baseline every comparison is made against. Run drush liipgpt:eviction-preview before leaving it to cron.

Sites coming from page_refresh_webhook can run drush liipgpt:adopt, which migrates that module's configuration, including reshaping the endpoint and folding its per-bundle settings into this module's.

Additional Requirements

• Entity Usage 2.2 or newer, with its data tracked and regenerated. Everything this module decides about reachability it reads from that table.
• Key.
• PHP 8.3 or newer.

Scope limits, stated up front

The metadata endpoint resolves public-scheme file URLs. Private files and CDN URLs cannot be reversed from a URL back to an entity; sites that need them register a subscriber on entity_usage's Events::URL_TO_ENTITY. This limit is confined to the endpoint. Eviction goes entity to URL, which is the forward direction and works for every scheme, including rokka.

URLs no Drupal entity produces can never be evicted, because no ledger row can exist for them: legacy uploads with no file entity, assets shipped in a theme, anything on another host. The module records every metadata lookup it could not answer, which is the only inventory of that gap available without a listing endpoint on the index side.

Extending

• hook_liipgpt_index_access() decides whether an entity belongs in the index. The crawl push, the endpoint and eviction all consult it, so one implementation governs all three.
• hook_liipgpt_reachability_roots_alter() removes nodes from the set of reachability roots, for sites whose content model says some published pages are not really public.