Postgres VDB Provider

The Postgres VDB Provider is a database module that works together with the AI module, to allow you to do vector searches with AI Search or other modules that implement it.

The module provides integration with a Postgres database using pgvector, a vector database, and the AI module.

This module provides features such as creating the tables to store vectors and other indexed fields, the indexing of data, the deletion of indexed data, vector searches with and without filters and a basic Postgres query search.

Features

Postgres is a powerful, open source object-relational database system with a strong reputation for reliability, feature robustness, and performance. It is supported already by the majority of cloud providers and can easily be run locally.

Using the open source extension pgvector, Postgres can store and search upon vectors.

The open source nature and wide availability of this vector database management solution makes this a good candidate, especially for small to medium sized vector databases.

This module currently provides features such as creating the tables to store vectors and other indexed fields, the indexing of data, the deletion of indexed data, vector searches with and without filters and a non-vector query search.

Installation

Use composer to install this module and its dependencies.

Post-Installation

• Enable the module
• Configure the database connection
• /admin/config/ai/vdb_providers/postgres
• Create the Search API server
• /admin/config/search/search-api/add-server
• Select AI Search as the Backend
• Configure the backend
• Select Postgres vector DB as the Vector Database
• Enter the database name
• Enter the collection name (main table name)
• Select the similarity metric to use in vector searches
• Configure the rest of the AI Search backend as desired (see AI Search documentation)
• When saved this will attempt to create the collection main table
• Setup an index that uses the new server and configure as desired (see AI Search documentation)

Additional Requirements

• PHP extension pgsql enabled on the web server
• AI module
• AI Search module
• Key module
• Search API module

Roadmap/TODO

• Handle the changes to the table schema better
• Work out where we can make these schema changes but keep user data for as long as possible
• Potentially look to search_api_db to see how it solves this problem
• Currently tables may have to be dropped by clearing the index for big schema changes
• Investigate if this can use core’s db connection
• Lots of calls are made to sanitise what could be user input, can these calls be cached for performance
• Tests
• Support for facets

Similar projects

Other AI Search VDB providers:

• Pinecone VDB Provider
• Milvus VDB Provider

Thanks

Thanks to the maintainers of the AI and AI Search modules for building the underlying infrastructure.