Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs-site/content/30.2/api/vector-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -2860,6 +2860,40 @@ For this example, when you index a document:
```
The text used to generate embeddings for the `embedding` field will be `passage: ABCD` instead of `ABCD`. And when you query, if your query is `EFGH`, it will be embedded as `query: EFGH` instead of `EFGH`.

### Providing pre-generated document embeddings for auto-embedding fields

If a document you index already contains a value for an auto-embedding field, Typesense skips embedding generation for that document and indexes the vector you provided as-is. This applies to create, upsert and update requests.

This lets you split the embedding work between your pipeline and Typesense: you generate **document** embeddings in your own indexing pipeline and include them in the documents you send, while Typesense still generates the **query** embedding at search time using the model configured on the field. Semantic search and hybrid search on the field keep working exactly as described below, with no changes to your search parameters.

This is particularly useful when:

- You are bulk-importing a large number of documents. Generating document embeddings inside Typesense requires a model inference per document before it can be indexed, which slows down large imports.
- Your cluster serves highly concurrent semantic or hybrid searches while documents are being imported. With built-in models, document embedding and query embedding share the same model instance on each node (and the same GPU, when one is used), so heavy import-time embedding can slow down search-time query embedding.
- You want to generate document embeddings in batches on your own infrastructure, which can be more cost-effective than sizing your Typesense cluster (or GPU) for import-time embedding.

For example, using the `products` collection defined [above](#creating-an-auto-embedding-field), you can import documents that already carry their embedding vector:

```bash
curl "http://localhost:8108/collections/products/documents/import?action=create" \
-H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
-X POST \
-d '{"product_name": "Cell phone", "categories": ["electronics"], "embedding": [0.241, 0.909, ...]}
{"product_name": "Laptop", "categories": ["electronics"], "embedding": [0.394, 0.481, ...]}'
```

Documents that include the `embedding` field are indexed with the vector given, and only documents that omit the field have their embedding generated by Typesense.

:::warning Use the same model and prefixes
Since Typesense embeds the query string with the model configured in `model_config` at search time, the document embeddings you provide must be generated with that **same model** (and have the same number of dimensions), otherwise vector distances will be meaningless.

Also make sure you apply the model's indexing prefix to the document text yourself before embedding, if the model uses one. For example, the `ts/e5-*` family of models embeds documents as `passage: <text>` and queries as `query: <text>`, as described above. Typesense applies these prefixes automatically only when it generates the embedding.
:::

:::tip Updates
If you later update any of the `embed.from` source fields of a document without including the embedding field in the update, Typesense will regenerate the embedding for that document. Include the vector field in the update request to keep using your own embeddings.
:::

## Nearest-neighbor vector search

Once you've indexed your embeddings in a vector field, you can now search for documents that are "closest" to a given query vector.
Expand Down Expand Up @@ -3439,6 +3473,10 @@ curl 'http://localhost:8108/multi_search' \
Typesense will do a keyword search using the `q` parameter, and a nearest neighbor search
using the `vector_query` field and combine the results into a ranked set of results using rank fusion as described earlier.

:::tip
If you only want to move **document** embedding generation outside Typesense (for example, to keep bulk imports from competing with search-time query embedding), you don't need to embed the query yourself. Define the field as an auto-embedding field and provide the document vectors at import time, as described in [Providing pre-generated document embeddings for auto-embedding fields](#providing-pre-generated-document-embeddings-for-auto-embedding-fields). Typesense will then generate only the query embedding, and hybrid search works with the regular `query_by` syntax shown above.
:::

:::warning Performance Tip
If you expect users to use several-words-long queries in the `q` parameter when doing hybrid search (which is common during [conversational search](./conversational-search-rag.md) for eg),
you want to set `drop_tokens_threshold: 0` as an additional search parameter to avoid redundant internal keyword searches and excessive CPU usage. Read more about what this parameter does under [this table](./search.md#typo-tolerance-parameters).
Expand Down