Part of the AI ecosystem · 358 projects
This module provides a framework for evaluating and improving how AI models handle ethical dilemmas with empathetic reasoning. It integrates with Drupal's AI module to benchmark LLM responses, acting as a testing and quality-assurance layer for AI in Drupal by scoring responses on empathy, accuracy, and explanation quality. It also offers tools for analysis, visual benchmarking across different AI providers and models, and optional integrations for guardrails, governance, observability, and content moderation.
AI Empathy Evaluation measures and improves how AI models handle ethical dilemmas that call for empathetic reasoning. It runs research-backed scenarios through any provider configured in the AI module, scores the responses, and helps you compare models, track changes over time and audit high-stakes decisions.
Features
- Four core metrics: Decision Accuracy (0-100%), Empathy Alignment (1-5), Explanation Quality (1-5) and Consistency Index (0-100%), each with a configurable threshold.
- 20 default scenarios across Military, Medical, Emotion-based and Cultural categories, each with a difficulty level (1-5) and a reference decision.
- AI-assisted scenario builder for drafting new scenarios, plus installable domain scenario packs.
- Dashboard with summary cards, gauges and charts. Submodules can add their own metrics.
- Multi-provider benchmarking with CSV export, including system-cost metrics: latency, output tokens, throughput, memory and energy.
- Training mode, human rating and rating analysis, with inter-rater reliability (Cohen's and Fleiss' kappa), difficulty calibration and blind comparison.
- Scheduled evaluations on cron, with email alerts when a metric falls below its threshold or the provider changes.
- AI request tags on every provider call, so guardrails, logging and observability can tell the model under evaluation apart from the judge.
Empathy checks where AI already runs (new in 1.5.0)
- AI agents and the AI Assistant: a "Score empathy" tool lets an agent check a reply it is drafting and revise it before sending.
- ECA: an action that scores a text into a token, and a condition that is true when a text scores below a threshold. Flag cold-sounding content, notify an editor or stop a publish without code.
- Guardrails: stop AI chat responses below an empathy threshold, with a sample rate and score caching to keep the cost down.
- Text fields: a one-click "Check Empathy" button for editors, with an optional publish gate.
- One-step setup: a recipe that enables the field action and the guardrail and creates an "Empathy" guardrail set.
AI governance policies (new in 1.4.0)
The AI Empathy Governance submodule includes an updatable catalogue of AI governance frameworks and principles: the EU AI Act, OECD AI Principles, UNESCO, UN, the Council of Europe Framework Convention, the G7 Hiroshima Process, ISO/IEC 42001, India, the US NIST AI RMF, the UK, Singapore, Canada, China, and general trustworthy-AI principles.
- Choose the policies that apply site-wide, by jurisdiction or as general principles, and override them per scenario.
- The active policies' guidance is added to the evaluation prompt, either directly or as Context Control Centre items.
- Each response gets a Policy Alignment score (0-100), shown on result pages and the dashboard.
- Policies are configuration: edit them, add your own, and export them.
Submodules
- AI Empathy Governance: Trust and Accountability dimensions (the Trust · Accountability · Empathy triad), the governance policy catalogue and policy alignment scoring.
- AI Empathy CCC Integration: brand and organisational context from the AI Context Context Control Centre, Tone Alignment scoring, and governance policies delivered as context items.
- AI Empathy HR Governance and AI Empathy Healthcare Governance: sector scoring for AI-assisted hiring and clinical decisions (EU AI Act, NYC LL144, DPDP, HIPAA, GDPR), with domain scenarios.
- AI Empathy Finance Scenario Pack: financial-services dilemmas.
- AI Empathy Guardrail: an Empathy Score Check plugin for the AI module's guardrails, with sampling and score caching.
- AI Empathy ECA: an action and a condition for ECA models (needs ECA 3 and Drupal 11.3 or later).
- AI Empathy Field Action: a one-click "Check Empathy" button on text fields and an optional content-moderation publish gate (needs Field Widget Actions).
- AI Empathy Observability: OpenTelemetry spans and a Grafana dashboard template (needs OpenTelemetry).
Requirements
Installation
composer require drupal/ai_empathy drush en ai_empathy
Then go to Configuration → AI → AI Empathy Evaluation → Settings and choose the evaluation and scoring models. See the README for full documentation.
To set up empathy checks for AI content in one step, apply the included recipe instead:
drush recipe modules/contrib/ai_empathy/recipes/ai_empathy_content
Research
This module is grounded in peer-reviewed research on empathetic decision-making in AI:
- Evaluating Empathetic Decision-Making in AI: A Comparative Study of Open-Source Models in High-Stakes Scenarios. IJFMR, 2025. doi:10.36948/ijfmr.2025.v07i06.63345
- Operationalising Empathy in AI Systems for High-Stakes Decision-Making. IJISRT, 2026. doi:10.38124/ijisrt/26jun931
- Scoring the Score: An Empathy, Trust and Accountability Lens for Auditing AI-Assisted Hiring Decisions. IJSR, 2026. doi:10.21275/sr26705230110
Ask your assistant about AI Empathy Evaluation
Check compatibility with your Drupal and PHP version, maintenance and security coverage, from current release data. How it works
Depends on
Dependencies of the latest stable release
Required by
Tracked projects that depend on this one
No tracked projects depend on this one yet.
More in the AI ecosystem
Most installed first