html_to_markdown (for ai_automator)
This module converts HTML content from one field into Markdown in another field. It removes unnecessary HTML tags and converts elements like tables, headings, and lists into their Markdown equivalents, significantly reducing token count for AI processing.
Reads the HTML in one field and writes it into another field as Markdown. Markdown holds the same text in far fewer tokens, so it costs less to send to a model.
Why HTML to markdown for Automators?
A model that reads HTML pays for every tag, every attribute and every class name. None of that carries meaning for the model. On a EUR-Lex notice the same document is 239,452 characters as HTML and 73,430 characters as Markdown, a drop of 69%.
The source can be any field that holds HTML: a body field, a WYSIWYG field, the output of a crawler, or imported content. This module knows nothing about where the HTML came from.
What the conversion does
script,styleandnoscriptare removed- Tables become Markdown pipe tables
- Headings, lists, links, bold and italic keep their meaning
- A tag with no Markdown equivalent loses the tag and keeps its text
- The input is read as UTF-8, so accented characters survive
Requirements
- Drupal 10.4+ / 11.x / 12.x
- AI ^1.4, with the
ai_automatorssubmodule enabled - One field that holds HTML, and one field for the Markdown
How to use
- Add a field for the Markdown. A plain long text field keeps the Markdown intact. A formatted long text field also works.
- Open the settings of this field.
- Enable the AI Automator and select HTML to Markdown.
- Set the base field to the field that holds the HTML.
- Give this automator a higher weight than the automator that fills the HTML field, so it runs after it.
An automator cannot write into its own base field, so the HTML field and the Markdown field must be two different fields.
Selector
The module has one setting. Leave it empty to convert the whole field, or give one CSS selector to convert only a part of it.
Selector Result (empty) The whole field#document1
The content of the element with id document1
div.content
The content of every div with class content
main article
The content of every article inside main
The module returns the content inside each match, not the match itself. When the selector matches more than one element, the parts are joined in document order. When the selector matches nothing, or when the selector is not valid, the field stays empty.
Allowed source fields
text_long, text_with_summary, text, string_long and string.
Related
- AI Automator: EC URL Transformer turns a EUR-Lex link into a Cellar URL that a crawler can read.
- Simple Crawler fills a field with the HTML of a page.