ai_text_cleaner

AI Text Cleaner

AI Text Cleaner is a small Drupal module that provides a text filter and a Drush command to clean formatting quirks from ChatGPT or other LLM outputs.
It removes hidden or problematic characters, converts non-breaking spaces, normalizes dashes and quotes, converts ellipses, and strips Markdown-style headings and stray asterisks.

Features

• Text filter plugin that can be enabled on input formats to clean content on save.
• Options supported:
  • Remove hidden/control characters
  • Convert non-breaking spaces to regular spaces
  • Normalize dashes (hyphen/en dash/em dash) to a standard hyphen
  • Normalize quotes (curly “smart” quotes → straight quotes)
  • Convert ellipses (…) to three periods (...)
  • Remove trailing whitespace
  • Remove stray asterisks and stray list artifacts
  • Strip Markdown-style headings (leading # characters)
• Drush command ai:text-clean (alias ai-text-clean) to run the cleaner across nodes in bulk.
• Per-option statistics exported for reporting or CSV-like table output.
• Analysis mode (dry run) that reports what would change without saving.
• Ability to limit by content type (bundle), language, and maximum node count.

Dependencies

• Drupal core (module developed for Drupal 11)
• PHP 8.1 or newer
• Optional: Drush (to use the ai:text-clean command)
• Filter system: integrates with Drupal's core filter subsystem (ensure input formats are configured).

Installation

1. Place the module in web/modules/custom/ai_text_cleaner.
2. Enable the module (example using Drush):

drush en ai_text_cleaner -y

To automatically apply the filter, go to Configuration → Text formats and editors, edit the target input format, and enable the "AI Text Cleaner" filter at the desired weight.

Plain-text field cleaning (since v1.1)

A new configuration option allows the module to clean simple text fields (fields without an assigned Text Format) when content is saved. To enable and configure this:

1. Go to Administration → Configuration → Content → AI Text Cleaner settings (or visit /admin/config/content/ai-text-cleaner).
2. Enable "Clean plain text fields" and choose which cleaning options to apply to plain text fields.

When enabled, the same cleaning rules that the filter uses for formatted text (remove hidden characters, normalize dashes/quotes/guillemets, convert ellipses, etc.) will be applied to fields that don't have a text format set.

Usage with Drush

Analyze (dry run) all nodes of type article:

drush ai:text-clean --types=article --analysis=TRUE

Apply cleaning and save changes for up to 50 nodes across all languages:

drush ai:text-clean --limit=50 --analysis=FALSE

You can supply multiple content types or languages as comma-separated lists, for example --types=article,page or --languages=en,de.