rl

This module is included in DXPR CMS.

The Reinforcement Learning (RL) module implements A/B testing in the most efficient and effective way possible, minizing lost conversions using machine learning.

Thompson Sampling is a learning-while-doing method. Each time a visitor lands on your site the algorithm "rolls the dice" based on what it has learned so far. Variants that have performed well roll larger numbers, so they are shown more often, while weak copies still get a small chance to prove themselves. This simple trick means the system can discover winners very quickly without stopping normal traffic.

Traditional A/B tests run for a fixed horizon—say two weeks—during which half your visitors keep seeing the weaker version. Thompson Sampling avoids this waste. As soon as the algorithm has even a little evidence it quietly shifts most traffic to the better variant, saving conversions and shortening the wait for useful insights.

For full details of what goes on behind the curtains check the source code:
ThompsonCalculator.php.

RL Modules

• AI Sorting - Intelligent content ordering/switching for Drupal Views

Features

• Thompson Sampling Algorithm - Pure PHP implementation
• Fast HTTP REST API - Optimized JSON endpoints for tracking and decisions
• Administrative Reports - Experiment analysis interface
• Service-based Architecture - Extensible design
• Data Sovereignty, Privacy First - No cloud, just Drupal

You need RL if

• A/B Testing - Test content variations
• Content Optimization - Track content engagement
• Feature Selection - Choose features to show users
• Recommendations - Optimize content recommendations
• Resource Allocation - Distribute resources across options

Thompson Sampling

Unlike traditional A/B testing, Thompson Sampling:

• Adapts automatically - Shifts traffic to better options
• Handles multiple options - Works with 2+ variations, even thousands of arms are handled well
• Continuous learning - No fixed test duration
• Bayesian approach - Incorporates uncertainty

Prefer a turnkey demo site?

Spin up DXPR CMS—Drupal pre-configured with DXPR Builder, DXPR Theme, RL (Reinforcement Learning) module, and security best practices.

Get DXPR CMS »

Installation

composer require drupal/rl
drush en rl

Verify rl.php Access

The RL module includes a .htaccess file that allows direct access to rl.php (following the same pattern as Drupal 11's contrib statistics module). Test that it's working:

curl -X POST -d "action=ping" http://example.com/modules/contrib/rl/rl.php

If the test fails:

• Apache: Ensure .htaccess files are processed (AllowOverride All)
• Nginx: Copy the rewrite rules from .htaccess to your server config
• Security modules: Whitelist /modules/contrib/rl/rl.php

If server policies prevent direct access to rl.php, use the Drupal Routes API instead.

Drush Command Reference

Category Commands Description Discovery rl:list, rl:status, rl:performance, rl:trends List experiments, check phase/confidence, arm-level stats, historical trends Analysis rl:analyze, rl:export Full analysis with recommendations, export experiment data Experiment CRUD rl:experiment:create, rl:experiment:update, rl:experiment:delete Create, update, and delete experiments with --dry-run support Configuration rl:config:get, rl:config:set, rl:config:list, rl:config:reset Get/set module settings, list all with current values, reset to defaults Setup rl:setup-ai Install AI skill files for Claude Code, Codex, Gemini, Copilot, Cursor

AI Coding Assistant Integration

The RL module ships with a built-in Agent Skills file that teaches AI coding assistants how to manage experiments through natural language. Compatible with Claude Code, Codex CLI, Gemini CLI, GitHub Copilot, Cursor, and other tools supporting the standard.

After installing the module, run drush rl:setup-ai to enable AI assistant support. Your AI will then respond to natural language like:

• "List all running experiments"
• "Analyze the hero_cta_test experiment"
• "Create a new A/B test for the homepage banner"
• "What's the conversion rate for variant B?"

API

// Get the experiment manager
$experiment_manager = \Drupal::service('rl.experiment_manager');

// Record a trial (content shown)
$experiment_manager->recordTurn('my-experiment', 'variant-a');

// Record a success (user clicked)
$experiment_manager->recordReward('my-experiment', 'variant-a');

// Get Thompson Sampling scores
$scores = $experiment_manager->getThompsonScores('my-experiment');

// Select the best option
$ts_calculator = \Drupal::service('rl.ts_calculator');
$best_option = $ts_calculator->selectBestArm($scores);

// Override page cache for web components (optional)
$cache_manager = \Drupal::service('rl.cache_manager');
$cache_manager->overridePageCacheIfShorter(60); // 60 seconds

JavaScript API

Attach the rl/api library to make Drupal.rl available on the page. It exposes a thin transport layer that coalesces every RL call on the page into a single batched POST to rl.php:

// Ask for a Thompson Sampling decision - resolves to the winning arm id.
Drupal.rl.decide('hero_cta', ['v0', 'v1', 'v2']).then(function (armId) {
  renderVariant(armId);
});

// Record an impression.
Drupal.rl.turn('hero_cta', 'v0');

// Record a conversion.
Drupal.rl.reward('hero_cta', 'v0');

Decides flush on the next tick so all modules that register during Drupal.behaviors.attach share one request. Turns and rewards flush in a 500 ms window and on pagehide via navigator.sendBeacon.

HTTP API (rl.php)

rl.php is the low-level HTTP endpoint reachable directly from any client that can make an HTTP POST, not only in-browser Drupal pages. Native mobile apps, server-side workers, other CMSes, and edge functions can record turns and rewards or request Thompson Sampling decisions against the same experiments used by in-browser code. The JavaScript Drupal.rl is simply a thin batching proxy that speaks this protocol.

Two actions are supported:

• action=ping - liveness probe used by hook_requirements(). Returns pong with HTTP 200. No database touch, no Drupal bootstrap.
• action=batch - zero or more decide, turn, and reward events in a single JSON body. All three sections are optional.

The endpoint requires no authentication. Experiment IDs must already be registered in the rl_experiment_registry table (normally via a module's __construct()) - unknown IDs are silently dropped so garbage writes cannot create arbitrary registry entries.

Request

POST /modules/contrib/rl/rl.php?action=batch
Content-Type: application/json

{
  "decides": [
    {"id": "hero_cta", "arms": ["v0", "v1", "v2"]},
    {"id": "page_title_123", "arms": ["v0", "v1"]}
  ],
  "turns": [
    {"id": "menu_main_5", "arm": "v1"},
    {"id": "hero_cta", "arm": "v0"}
  ],
  "rewards": [
    {"id": "hero_cta", "arm": "v0"}
  ]
}

Experiment IDs and arm IDs must match ^[a-zA-Z0-9_-]+$. Decides require at least 2 arms. Invalid or unregistered entries are dropped silently so one bad event does not poison the rest of the batch.

Response

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: no-store, private, max-age=0

{
  "decisions": {
    "hero_cta": {"armId": "v1"},
    "page_title_123": {"armId": "v0"}
  }
}

Only successful decide lookups appear in the map. Callers should always hold a local fallback for missing keys. Turns and rewards produce no response payload beyond HTTP 200 - they are fire-and-forget writes.

Curl examples

Impression from a server-side job:

curl -X POST 'https://example.com/modules/contrib/rl/rl.php?action=batch' \
  -H 'Content-Type: application/json' \
  -d '{"turns":[{"id":"hero_cta","arm":"v1"}]}'

Decision lookup from a native mobile client:

curl -X POST 'https://example.com/modules/contrib/rl/rl.php?action=batch' \
  -H 'Content-Type: application/json' \
  -d '{"decides":[{"id":"hero_cta","arms":["v0","v1","v2"]}]}'
# => {"decisions":{"hero_cta":{"armId":"v1"}}}

Liveness probe:

curl -X POST 'https://example.com/modules/contrib/rl/rl.php?action=ping'
# => pong

Performance note: rl.php bootstraps a minimal Drupal kernel per request (same pattern as core's statistics.php). One boot processes the whole batch, so non-browser callers should coalesce events into a single request whenever they can.

Cache Management

RL provides optional cache management for web components:

// Override page cache if experiment cache is shorter than site cache
\Drupal::service('rl.cache_manager')->overridePageCacheIfShorter(30);

How it works:

• If site cache is 300s and experiment needs 30s → overrides to 30s
• If site cache is 60s and experiment needs 300s → leaves at 60s
• If site cache is disabled → no override

Use cases:

• Views plugins using RL for content sorting
• Blocks displaying A/B tested content
• Components needing frequent RL score updates

Ready to get started? Install the module and begin implementing intelligent, adaptive decision-making in your Drupal applications today!

Resources

• Free Micro-Course on Thompson Sampling
• Multi-Armed Bandit Problem Wiki
• On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two Samples
• Finite-time Analysis of the Multiarmed Bandit
  Problem