ai_automator EC Eli to Cellar url transformer
This module automatically converts EUR-Lex links into direct URLs for the EU Publications Office (Cellar). This is useful for crawlers that cannot bypass EUR-Lex's JavaScript-based bot challenge, as Cellar provides direct access to the documents. The module can transform various EUR-Lex link formats into their corresponding Cellar URLs.
Turns a EUR-Lex link into the matching EU Publications Office (Cellar) URL. Cellar serves the same document over plain HTTP, so a crawler can read it.
Why
EUR-Lex is behind a bot challenge. The challenge needs JavaScript, so a server side HTTP client cannot pass it. Cellar is the repository behind EUR-Lex. It needs no challenge, no token and no account.
This module adds one AI Automator type. The automator reads a EUR-Lex link from a base field, and writes the Cellar URL into a link field. A crawler automator then reads that URL.
How the URL is built
The module pads the OJ number to five digits.
Source Result.../eli/C/2026/2425/oj
.../resource/oj/C_202602425
.../eli/C/2026/511/oj
.../resource/oj/C_202600511
The module reads these source formats:
- ELI paths in the
C/year/numbershape, oneur-lex.europa.euordata.europa.eu - Links with a
uriparameter, such as?uri=OJ:C_202602425or?uri=CELEX:32016R0679 - URLs that already point to
publications.europa.eu, which stay the same
An older ELI such as /eli/reg/2016/679/oj maps to a CELEX number that the URL does not hold. For these the module asks the Cellar SPARQL endpoint, and it stores the answer for 30 days.
Requirements
- Drupal 10.4+ / 11.x / 12.x
- AI ^1.4, with the
ai_automatorssubmodule enabled - One link field for the source URL, and one link field for the result
How to use
- Add a link field for the result.
- Open the settings of this field.
- Enable the AI Automator and select EC URL Transformer: EUR-Lex to Cellar URL.
- Set the base field to the link field that holds the EUR-Lex URL.
- Give this automator a lower weight than the crawler that reads the result.
Reading the result
The client that reads a Cellar URL must send two headers:
Accept: application/xhtml+xml Accept-Language: eng
Accept-Language takes ISO 639-2 codes, such as eng, fra or nld. Cellar uses this header to select the language of the document.