AI Empathy Evaluation
This module provides a framework for evaluating and improving how AI models handle ethical dilemmas with empathetic reasoning. It integrates with Drupal's AI module to benchmark LLM responses, acting as a testing and quality-assurance layer for AI in Drupal by scoring responses on empathy, accuracy, and explanation quality. It also offers tools for analysis, visual benchmarking across different AI providers and models, and optional integrations for guardrails, governance, observability, and content moderation.
AI Empathy Evaluation
The AI Empathy Evaluation module provides a framework for evaluating and improving how AI models handle ethical dilemmas with empathetic reasoning. It integrates with Drupal's AI module to benchmark LLM responses, and acts as a testing and quality-assurance layer for AI in Drupal.
Core evaluation
- Scores responses to ethical-dilemma scenarios on four research-backed metrics: decision accuracy, empathy alignment, explanation quality, and consistency.
- Ships with 20 default scenarios across Military, Medical, Emotion-based, Cultural — plus an Organizational category — each with a reference decision and difficulty level (1–5). Create your own through the admin UI or the AI-assisted scenario builder.
- Send scenarios to any configured AI provider (OpenAI, Anthropic, etc.) via the AI module and receive structured responses analysed for empathetic reasoning.
Analysis & tooling
- Interactive dashboard (Chart.js): metric gauges, provider/model comparison, category performance, and trends.
- Multi-provider comparison / benchmarking with rankings, per-scenario tables, historical trends, and CSV export.
- Training mode with cumulative feedback across a session.
- Human rating system with inter-rater reliability (Cohen's / Fleiss' kappa), blind comparison, and rating analysis.
- Scheduled cron-based evaluation with email alerts on threshold breaches and provider-model changes.
Ecosystem integrations (optional submodules)
- AI Empathy Guardrail — an Empathy Score Check guardrail for the AI guardrails system.
- AI Empathy Governance — adds Accountability and Trust dimensions (the Trust · Accountability · Empathy triad) with a triad visualization.
- AI Empathy Observability — exports metrics as OpenTelemetry spans with a Grafana dashboard template.
- AI Empathy Field Action — a one-click "Check Empathy" button on text fields, plus an optional content-moderation publish gate.
- Context Control Centre integration — injects brand / organisational context and adds a tone-alignment metric.
- Domain scenario packs (Finance, Healthcare, HR) with domain-specific scenarios and scoring.
All integrations are backward-compatible and opt-in — enable only what you need.
Research & references
This module is grounded in peer-reviewed research on evaluating empathetic decision-making in AI:
- Evaluating Empathetic Decision-Making in AI: A Comparative Study of Open-Source Models in High-Stakes Scenarios — International Journal For Multidisciplinary Research (IJFMR), 2025-12-13.
DOI: 10.36948/ijfmr.2025.v07i06.63345
(ISSN 2582-2160). - Operationalising Empathy in AI Systems for High-Stakes Decision-Making — International Journal of Innovative Science and
Research Technology (IJISRT), 2026-06-23.
DOI: 10.38124/ijisrt/26jun931
(ISSN 2456-2165). - Scoring the Score: An Empathy, Trust and Accountability Lens for
Auditing AI-Assisted Hiring Decisions — International Journal of Science and Research (IJSR), 2026-07-08.
DOI: 10.21275/sr26705230110
(ISSN 2319-7064).
See also the Drupal AI module documentation, which this module builds on.