botshield

BotShield helps Drupal sites detect bot traffic, apply per-bot/IP rate limits or blocks, enrich events with geo data, and provide reports to safely tune crawler access.

BotShield is a Drupal module that gives site administrators visibility and control over automated traffic. It classifies bots, enforces configurable policies, and provides reports for safe tuning.

Features

• Classifies known and custom bots using User-Agent rules.
• Supports per-bot actions: allow, rate_limit, and block.
• Supports IP/CIDR rules and override file support for always-allow IPs.
• Adds path-group rules and per-group thresholds.
• Includes flood safety-net controls, including stricter unknown-bot handling.
• Enriches events with geo data from headers, GeoLite2 MMDB, and optional free API fallback.
• Provides report pages: dashboard, status, log, map, and alerts.
• Supports customizable 429 response messaging.
• Includes retention cleanup via cron.

Use cases

• High-traffic sites needing crawler control without blocking all bots.
• Editorial/operations teams needing visibility into bot behavior and origin.
• Sites that want Drupal-native bot controls and reporting.

Post-Installation

1. Grant permissions at /admin/people/permissions:
   • administer botshield for settings managers.
   • view botshield reports for report viewers.
2. Configure BotShield at /admin/config/system/botshield.
3. Start with Traffic scope = anonymous while tuning.
4. Set thresholds, flood controls, and bot policies.
5. If using MMDB upload, configure $settings['file_private_path'] and upload GeoLite2-City.mmdb.
6. Verify dependencies and health at /admin/reports/botshield/status.
7. Tune behavior with /admin/reports/botshield/log and /admin/reports/botshield/map.
8. Review setup guidance at /admin/reports/botshield/help.
9. Ensure cron runs for retention cleanup.

Additional Requirements

• Drupal core 10/11.
• Required core modules: file, system.
• Composer dependency for local GeoLite2 lookups: geoip2/geoip2.
• Private files configured for MMDB and override-file uploads.
• Optional outbound HTTP access if free API geo fallback is enabled.
• Mail transport configured if alert email is enabled.

Recommended modules/libraries

• Redis module (plus cache-bin mapping) for better performance at scale.
• An SMTP/mail module for reliable alert delivery.
• Drush for cache clear and cron operations.
• GeoLite2 City MMDB for improved geo coverage.

Similar projects

• CAPTCHA/Honeypot modules focus on form spam; BotShield focuses on request-level bot traffic across routes.
• Manual IP ban tools are static/manual; BotShield adds automated per-bot and per-group enforcement.
• Edge/CDN bot controls are external; BotShield provides Drupal-native policy logic and reporting.

Community Documentation

• Project README (installation, settings, dependencies).
• In-module guide: /admin/reports/botshield/help.