Vector Search
Enable AI-powered semantic search with high-dimensional vector similarity. Perfect for RAG, recommendations, and intelligent content discovery.
Vector search enables finding semantically similar items by comparing high-dimensional embeddings. Instead of exact keyword matching, vector search understands meaning - finding documents that are conceptually related even if they use different words.
Semantic Search
Find by meaning, not keywords
RAG Applications
Power AI with relevant context
Recommendations
Similar items & content
How Vector Search Works
Generate Embeddings
Use an AI model (OpenAI, Ollama, etc.) to convert text/images into vectors
Store in SoliDB
Save embeddings as array fields in your documents
Create Vector Index
Index the embedding field for fast similarity search
Query by Similarity
Find nearest neighbors using SDBQL or REST API
HNSW: Fast Approximate Search
SoliDB uses HNSW (Hierarchical Navigable Small World) graphs for fast approximate nearest neighbor search on large vector datasets. HNSW provides 10-100x faster searches compared to brute-force while maintaining high recall (>95%).
Automatic Activation
HNSW automatically activates when your index exceeds 10,000 vectors. Below this threshold, exact brute-force search is used for 100% accuracy.
# Vectors >= 10K: HNSW approximate
Query-Time Tuning
Use the ef_search parameter to trade off between speed and recall at query time.
ef_search: 100 // Balanced
ef_search: 200 // High recall, slower
Performance Characteristics
O(log n) vs O(n) brute-force. 10-100x faster at 100K+ vectors.
Graph stored alongside vectors. ~20-30% overhead for neighbor links.
>95% recall with default settings. Tunable via ef_search.
Tip: HNSW graph is built incrementally as vectors are inserted and persisted to disk. No manual rebuild is required.
Scalar Quantization (4x Memory Reduction)
Scalar Quantization compresses vectors from f32 (4 bytes/dim) to u8 (1 byte/dim), reducing memory usage by 4x while maintaining high search accuracy (~98% recall).
How It Works
- 1. Compute per-dimension min/max across all vectors
- 2. Scale each value to 0-255 range (u8)
- 3. Search uses asymmetric comparison (query f32, DB u8)
Memory Impact
Enable Quantization
Create index with quantization:
{
"name": "embedding_idx",
"field": "embedding",
"dimension": 768,
"metric": "cosine",
"quantization": "scalar"
}Or quantize an existing index:
curl -X POST \
/_api/database/:db/vector/:collection/:index/quantizeWhen to use: Quantization is ideal for large datasets (100K+ vectors) where memory is a constraint. The ~2% recall drop is negligible for most RAG and search applications.
Distance Metrics
SoliDB supports three distance metrics for comparing vectors. Choose based on your embedding model's requirements.
Cosine
Measures the angle between vectors. Best for text embeddings where magnitude doesn't matter.
similarity = cos(θ) = A·B / (|A||B|)
✓ Sentence transformers
Euclidean
Straight-line distance in vector space. Best when magnitude matters.
distance = √Σ(Ai - Bi)²
✓ Normalized vectors
Dot Product
Raw dot product. Fastest metric, requires normalized vectors for meaningful results.
score = Σ(Ai × Bi)
✓ Maximum performance
Creating a Vector Index
Create a vector index on any field containing embedding arrays. The index enables fast similarity search.
{
"name": "embedding_idx",
"field": "embedding",
"dimension": 1536,
"metric": "cosine"
}Parameters
| name | Unique index name |
| field | Document field containing vectors |
| dimension | Vector dimension (e.g., 1536 for OpenAI) |
| metric | "cosine", "euclidean", or "dot" |
Common Dimensions
-
OpenAI text-embedding-3-small
1536 -
OpenAI text-embedding-3-large
3072 -
Ollama nomic-embed-text
768 -
Sentence Transformers
384-768
Storing Documents with Vectors
Store embedding vectors as array fields in your documents. The vector index automatically updates when documents are inserted, updated, or deleted.
{
"_key": "article_001",
"title": "Introduction to Machine Learning",
"content": "Machine learning is a subset of AI...",
"embedding": [0.0123, -0.0456, 0.0789, ...]
}Tip: Generate embeddings before insertion using your preferred embedding model (OpenAI, Ollama, HuggingFace, etc.). SoliDB stores and indexes the vectors but doesn't generate them.
Querying with SDBQL
Use vector functions in SDBQL to compute similarity scores, filter by similarity threshold, and sort by relevance.
FOR doc IN articles
LET similarity = VECTOR_SIMILARITY(doc.embedding, @query_vector)
FILTER similarity > 0.8
SORT similarity DESC
LIMIT 10
RETURN {
title: doc.title,
score: similarity
}FOR doc IN products
FILTER doc.category == "electronics"
FILTER doc.price < 500
LET sim = VECTOR_SIMILARITY(doc.embedding, @query_vec)
FILTER sim > 0.7
SORT sim DESC
LIMIT 20
RETURN docAvailable Functions
VECTOR_SIMILARITY(a, b)- Cosine similarity (0-1)VECTOR_DISTANCE(a, b, metric)- Distance valueVECTOR_NORMALIZE(v)- Normalize to unit length
Performance Tips
- • Apply non-vector filters first to reduce candidates
- • Use
LIMITto cap result size - • Pre-filter by metadata when possible
REST API Search
Use the dedicated vector search endpoint for direct similarity queries without writing SDBQL.
Request:
{
"vector": [0.0123, -0.0456, ...],
"limit": 10,
"ef_search": 100
}Response:
{
"results": [
{
"doc_key": "article_001",
"score": 0.9823
},
...
]
}Use Cases
🤖 RAG (Retrieval Augmented Generation)
Power LLM applications with relevant context. Store document chunks with embeddings, retrieve the most relevant passages for each query.
Semantic document retrieval
🔍 Semantic Search
Find documents by meaning, not keywords. "comfortable office chair" finds "ergonomic desk seating" even without matching words.
Natural language queries
💡 Recommendations
Suggest similar products, articles, or content. Users who viewed X might like these similar items.
Content-based filtering
🖼️ Image Search
Find visually similar images using CLIP or other vision embeddings. Search photos by description or by example.
Multimodal search