How it stores vectors¶
One core holds every collection¶
Each document names its collection and its kind. One document records a collection's vector field, similarity and dimension, and each chunk is a document of its own. Solr has no databases, so AI Search's database name is not used.
Fields¶
A chunk's text, its Drupal ids and its metadata use the _default
configset's dynamic fields: *_s, *_t, *_ss and *_ds.
- Metadata is kept as strings, and numbers and booleans also as numbers, so filters can compare them.
- The vector field is named for its dimension, similarity and quantization,
such as
vector_1536_cosineorvector_1536_cosine_scalar. - Vectors are indexed but not stored, so the raw vector cannot be returned.
Quantization¶
| Setting | Each dimension | Memory, against none | Solr |
|---|---|---|---|
| None | 32-bit float | all of it | 9.6 or later |
| Scalar | 7 bits | about a quarter | 10 |
| Binary | 1 bit | about a thirty-second | 10 |
The setting applies to collections created after it is changed; an existing collection keeps its vector field.
Filters¶
AI Search's conditions become Solr filter queries. Solr applies them before
the kNN search, so a filtered search still returns the nearest matches.
Supported operators: =, <>, IN, NOT IN, <, <=, >, >=,
BETWEEN and NOT BETWEEN.
Scores¶
Lucene scores cosine and dot product similarity as (1 + s) / 2, and Euclidean distance as 1 / (1 + d²). The provider turns them back into the similarity, or the squared distance, as Milvus returns them, so AI Search's thresholds mean the same.
Grouping¶
When AI Search checks access to each result, the provider collapses chunks by item, so a page of results is a page of distinct items.