This module allows you to cache content differently based on whether a site is being accessed by a known crawler. This enables you to tailor content delivery for search engines and other bots, such as providing a "next page" link only to them.
If you want to cache the output of a render array differently depending on if the site is visited by a known crawler or not, this is the module you need.
Use cases
Say you want Google to index all pages of a search result, even if you want the average user to get an infinite scroll. You can then want to display a "next page" link to google only. Or maybe to crawlers only:
$build['next'] = [
'#type' => 'link',
'#url' => $some_url
'#title' => t('Next page'),
'#cache' => [
'contexts' => [
// This will limit it to google bot only.
'crawlers_cache_context:googlebot',
],
],
'#access' => $access,
];
$build['next'] = [
'#type' => 'link',
'#url' => $some_url
'#title' => t('Next page'),
'#cache' => [
'contexts' => [
// This will limit it to crawlers in general.
'crawlers_cache_context',
],
],
'#access' => $access,
];
What are the crawlers
This module relies on the package jaybizzle/crawler-detect. If you download the module with composer, this should be downloaded automatically for you.
Currently you can get a general idea of what crawlers are detected with this link
Ask your assistant about Crawler cache context
Check compatibility with your Drupal and PHP version, maintenance and security coverage, from current release data. How it works
Depends on
Dependencies of the latest stable release
No dependencies recorded for this project.
Required by
Tracked projects that depend on this one
No tracked projects depend on this one yet.
Activity
Release Timeline
Releases
| Version | Type | Core | PHP | Release date | |
|---|---|---|---|---|---|
| 8.x-1.1 | Stable | 8–11 | Apr 1, 2024 | ||
| 8.x-1.0 | Stable | 8–10 | Nov 8, 2023 | ||
| 8.x-1.0-beta1 | Pre-release | 8–10 | Nov 8, 2023 |