This module provides a connection to AI models through an OpenAI-compatible server like llama.cpp, Ollama, or vLLM. It allows you to configure multiple AI servers and supports various AI operations such as chat, embeddings, and text-to-image.
⚠️ This module is no longer actively developed.
Development has moved to AI Provider Universal, which supports llama.cpp along with Ollama, vLLM, LM Studio, LiteLLM, Fireworks, OpenAI and any other OpenAI-compatible server — with multiple server instances, models as config entities and per-model capability overrides.
No new features will be added here. Only critical security fixes will be considered until a stable release of AI Provider Universal is available.
Existing users: you can keep using this module for now, but we recommend migrating to AI Provider Universal. Configure your llama.cpp server at Configuration → AI → AI Servers (/admin/config/ai/providers/universal).
llama.cpp AI Provider (Multi-instance)
1.2.0-beta1 is now available! This is the first beta of the 1.2 series with full multi-server support.
A powerful, multi-instance provider for the AI module. While built with llama.cpp as its primary focus, it natively supports any OpenAI-compatible /v1 server (Ollama, vLLM, LiteLLM, LM Studio, etc.).
Key Features
- Multi-server architecture: Configure and use multiple servers at the same time. Each server appears as an independent provider (e.g.
llama_cpp:gpu,llama_cpp:embeddings). - Secure credentials: API keys are stored using the Key module (no more plaintext).
- Broad operation support: Chat, Embeddings, Speech-to-Text, Rerank, Moderation (LlamaGuard3 + ShieldGemma with multiple safety policies), and Text-to-Image.
- Per-server control: Model filtering with glob patterns, manual capability overrides, and configurable timeout + authentication per server.
- Smart model detection: Auto-detects capabilities from server metadata, Hugging Face tags, and name heuristics.
- Robust & cache-friendly: Model lists and types are cached per server in Drupal State.
Supported Operations
- Chat completions (
/v1/chat/completions) - Embeddings (
/v1/embeddings) - Speech to Text (Whisper and similar)
- Rerank
- Moderation (native LlamaGuard3 and ShieldGemma support)
- Text to Image (
/v1/images/generations)
Installation
Install via Composer:
composer require drupal/ai_provider_llama_cppImportant: Composer installs the latest stable release by default (currently 1.0.0). To install the new 1.2.0-alpha1 (recommended for new projects), run:
composer require 'drupal/ai_provider_llama_cpp:^2.0@alpha'Then enable the module:
drush pm:enable ai_provider_llama_cppRequirements
- Drupal 10.2, 11 or 12
- AI module ^1.2
- Key module ^1.18 (for secure API key storage)
- An OpenAI-compatible server (Ollama, vLLM, llama.cpp, LiteLLM...)
See the release notes for full details and migration information from previous versions.
Note: This is a beta release. Please test in development/staging