ai_automator EC Eli to Cellar url transformer

Turns a EUR-Lex link into the matching EU Publications Office (Cellar) URL. Cellar serves the same document over plain HTTP, so a crawler can read it.

Why

EUR-Lex is behind a bot challenge. The challenge needs JavaScript, so a server side HTTP client cannot pass it. Cellar is the repository behind EUR-Lex. It needs no challenge, no token and no account.

This module adds one AI Automator type. The automator reads a EUR-Lex link from a base field, and writes the Cellar URL into a link field. A crawler automator then reads that URL.

How the URL is built

The module pads the OJ number to five digits.

The module reads these source formats:

• ELI paths in the C/year/number shape, on eur-lex.europa.eu or data.europa.eu
• Links with a uri parameter, such as ?uri=OJ:C_202602425 or ?uri=CELEX:32016R0679
• URLs that already point to publications.europa.eu, which stay the same

An older ELI such as /eli/reg/2016/679/oj maps to a CELEX number that the URL does not hold. For these the module asks the Cellar SPARQL endpoint, and it stores the answer for 30 days.

Requirements

• Drupal 10.4+ / 11.x / 12.x
• AI ^1.4, with the ai_automators submodule enabled
• One link field for the source URL, and one link field for the result

How to use

1. Add a link field for the result.
2. Open the settings of this field.
3. Enable the AI Automator and select EC URL Transformer: EUR-Lex to Cellar URL.
4. Set the base field to the link field that holds the EUR-Lex URL.
5. Give this automator a lower weight than the crawler that reads the result.

Reading the result

The client that reads a Cellar URL must send two headers:

Accept: application/xhtml+xml
Accept-Language: eng

Accept-Language takes ISO 639-2 codes, such as eng, fra or nld. Cellar uses this header to select the language of the document.