MySQL VDB Provider
This experimental module allows AI Search to perform vector searches using your existing MySQL database, eliminating the need for a specialized vector database. It leverages a custom MySQL library to efficiently retrieve similar vectors through a two-step process of approximate nearest neighbor search and a final ranking by cosine similarity.
The MySQL VDB Provider is an experimental database module that works together with the AI module, to allow you to do vector searches in AI Search using just your existing MySQL database instead of specialised vector database.
It uses the mysql-vector class (https://github.com/allanpichardo/mysql-vector)
which does the actual vector search cosine similarity.
However, to improve performance on mysql it does the search in two steps:
1. First it uses quantization and filters using the Hamming Distance
to retrieve vectors that are "close" (fast).
2. Second it ranks the remaining results by cosine similarity (slow)
More information: albert.skibinski.nl/en/blog/experimenting-vector-search-mysql
Post-Installation
See a demo video here: https://youtu.be/kFDChP_NBZA
Important: do not use more than 2040 dimensions on your embedder of choice, because at this time this translates to a length of
255 bytes (2040/8) which is the limit of the field.
Additional Requirements
- Needs AI patch: #3507269-8
Also the mysql-vector fork currently in use does not have a packagist repo,
so you will need to add it to your composer repositories:
{
"type": "vcs",
"url": "https://github.com/askibinski/mysql-vector"
},