Canvas AI Image Search

This recipe is based on some changes that are being worked on in Canvas AI under the roadmap: https://git.drupalcode.org/project/canvas/-/work_items/3591777

Until the roadmap is complete, it is not recommended to use this recipe in production, but please feel free to try it out and report bugs.

For the Canvas AI agent to search your media library, you need two things: first, generate a description for each media entity in the site, and second, index those descriptions in a vector database.

The AI Image Classification recipe provides the first part. It adds a description field to media entities and adds an automator that populates the description field with what the image is. So apply this recipe first and resave the media entities to generate descriptions. This will require you to configure a chat with vision model.

Step two is that these descriptions must be indexed in a vector database. You can use any of the vector database providers:

• amazee.io
• Azure
• MariaDB
• Milvus
• Pinecone 
• Postgres
• SQLite

Visit the README of this recipe to see how to set up the Postgres VDB provider and index the items.

Then apply this recipe. While applying, it will ask for the name of the index where you have indexed the image descriptions. Provide the name there. And that's it.
 

Demo: https://youtu.be/J10rTu5PIAM