AI Knowledge Connector

Overview

AI Knowledge Connector bridges Drupal and modern AI architectures by transforming Drupal entities into reusable knowledge documents suitable for embeddings, vector databases, Retrieval-Augmented Generation (RAG), and AI agents.

Large Language Models do not need more web pages. They need structured, contextualized knowledge. Drupal already stores knowledge in a highly structured format through entities, fields, taxonomies, relationships, workflows, and permissions. This module leverages that structure to feed AI systems with high-quality, organized information.

The Problem

Most AI integrations treat websites as collections of rendered HTML pages. This approach loses valuable information:

• Entity relationships
• Structured fields
• Taxonomies
• Metadata
• Editorial workflows
• Access control rules

As a result, AI systems often receive incomplete or poorly structured information.

What This Module Provides

• ✅ Structured knowledge extraction from Drupal entities
• ✅ AI-ready knowledge documents
• ✅ Foundation for RAG architectures
• ✅ Compatibility with local and cloud AI providers
• ✅ Extensible architecture for future integrations

Dependencies

Before installing this module, you need the following:

• AI module: Provides the core AI framework and API. Learn more about the AI module.
• AI Search module: Provides semantic search and vector database integration. Learn more about AI Search.
• Key module: For secure API key management. Learn more about the Key module.
• At least one AI Provider (e.g., OpenAI, Hugging Face, Mistral).
• At least one Vector Database Provider (e.g., Milvus, Pinecone, Qdrant).

Installation

Install the module via Composer:

composer require 'drupal/ai_knowledge_connector'
drush en ai_knowledge_connector

For a complete setup, you'll need to install and configure the full AI ecosystem. Refer to the AI module documentation for detailed instructions.

Configuration

1. Configure AI Provider and API Key

Follow the AI module setup guide to configure your AI provider and API key using the Key module.

2. Create a Search API Server and Index

Navigate to /admin/config/search/search-api and:

1. Create a new server with "AI Search Database Service (Vector Database)" as the backend.
2. Create a new index associated with this server, selecting the content types and fields you want to index.
3. Add the "AI Knowledge Connector" processor to the index.

The AI Search documentation provides detailed instructions for each step.

3. Configure Embedding Strategies

AI Search provides several embedding strategies for chunking content:

• Contextual Chunks (Recommended): Enriches each content chunk with contextual information for improved accuracy.
• Average Pool Embedding: Creates a single composite vector from multiple chunks using average pooling.
• Pluggable Strategy System: Extensible architecture for defining custom embedding approaches.

4. Index Your Content

Once everything is configured, click "Index now" on your Search API index page. The AI Knowledge Connector will process your entities, break them into chunks, generate embeddings, and store them in your vector database.

How to Use the Structured Data

Once your content is indexed, there are several ways to leverage this structured knowledge:

Creating a RAG Chatbot

Install the AI RAG Assistant module to get a ready-to-use chatbot. It provides:

• A dedicated chatbot page at /rag-chat
• A configurable block that can be placed anywhere on your site
• Context-aware responses based on your indexed content

Building Advanced Searches with Views

AI Search integrates with Drupal's Views module. You can:

• Create Views directly from your vector database
• Use fulltext search filters to query the vector database
• Combine vector search with traditional database or Solr searches using boost processors

For detailed examples, check the AI Search Views documentation.

Finding Similar Content

Use the AI Related Content module to find semantically similar content based on vector embeddings, enabling automatic recommendations without manual tagging.

Programmatic Access

Developers can access the vector database programmatically:

// Load your LLM provider
$llm = \Drupal::service('ai.provider');
$vector_input = $llm->embeddings($search_words, $model_id)->getNormalized();

// Load your VDB provider
$vdb = \Drupal::service('ai.vdb_provider');
$results = $vdb->vectorSearch($collection_name, $vector_input, []);

// Use the results with an LLM
$answer = $llm->chat("Here is a question, answer it using this content: " . $content_from_results);

Refer to the AI Search programmatic documentation for more details.

Key Features

Structured Knowledge Extraction

Convert Drupal entities into AI-ready documents while preserving context and relationships. The module respects entity structures, field relationships, and taxonomy hierarchies.

Provider-Agnostic Design

Compatible with any AI provider supported by the Drupal AI module, including local (Ollama) and cloud-based (OpenAI, Anthropic, Mistral) services. Also works with any vector database provider (Milvus, Pinecone, Qdrant, etc.).

Incremental Indexing

Process only entities that have changed, minimizing resource consumption. The module uses content hashing to detect changes and only re-indexes when necessary.

Queue-Based Processing

Built using Drupal Queue API for scalability and reliability, ensuring large sites can index thousands of entities without performance degradation.

Extensible Plugin Architecture

Add new knowledge sources, retrievers, and integrations without modifying core functionality. Developers can create custom plugins for specific entity types or data sources.

AI Search Tracker Integration

The AI Search tracker ensures Search API can generate vector embeddings at scale, providing efficient tracking of indexed content.

Use Cases

• Higher Education: Academic assistants, course discovery, program recommendations
• Government: Citizen service assistants, policy search, regulation discovery
• Drupal Commerce: Semantic product search, AI-powered shopping assistants, knowledge-based product recommendations
• Enterprise Knowledge Bases: Internal documentation assistants, corporate knowledge retrieval, AI-powered information discovery

Required Modules Summary

To use this module effectively, you need to install and configure the following modules:

• AI (core framework)
• AI Search (semantic search and vector database)
• Key (API key management)
• One AI Provider (e.g., OpenAI, Mistral, Hugging Face)
• One Vector Database Provider (e.g., Milvus, Pinecone, Qdrant)

Additional Resources

• AI Knowledge Connector project page
• Source code repository
• AI RAG Assistant - Ready-to-use chatbot interface
• AI Related Content - Semantic content recommendations
• Search API - Core search framework

Compatibility

Requires Drupal 10.3 or 11, PHP 8.3 or higher.

For any issues or feature requests, please use the issue queue.