From 3a7dbdaead57972a064928f4461211d2723ff221 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alan=20Galv=C3=A3o=20Martini?= Date: Tue, 21 Jul 2026 15:47:18 -0300 Subject: [PATCH] docs: providing pre-generated document embeddings for auto-embedding fields Documents that already contain a value for an auto-embedding field are indexed with the provided vector and skip embedding generation, while Typesense still embeds the query at search time. Documents when to use this (bulk imports, GPU/model contention with concurrent hybrid search, cost) and the same-model + indexing-prefix requirements, with a cross-reference from the hybrid search section. Claude-Session: https://claude.ai/code/session_01Vvtp2Z3KRphwTNrepC6EuU --- docs-site/content/30.2/api/vector-search.md | 38 +++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/docs-site/content/30.2/api/vector-search.md b/docs-site/content/30.2/api/vector-search.md index 692eea8d..ce33428c 100644 --- a/docs-site/content/30.2/api/vector-search.md +++ b/docs-site/content/30.2/api/vector-search.md @@ -2860,6 +2860,40 @@ For this example, when you index a document: ``` The text used to generate embeddings for the `embedding` field will be `passage: ABCD` instead of `ABCD`. And when you query, if your query is `EFGH`, it will be embedded as `query: EFGH` instead of `EFGH`. +### Providing pre-generated document embeddings for auto-embedding fields + +If a document you index already contains a value for an auto-embedding field, Typesense skips embedding generation for that document and indexes the vector you provided as-is. This applies to create, upsert and update requests. + +This lets you split the embedding work between your pipeline and Typesense: you generate **document** embeddings in your own indexing pipeline and include them in the documents you send, while Typesense still generates the **query** embedding at search time using the model configured on the field. Semantic search and hybrid search on the field keep working exactly as described below, with no changes to your search parameters. + +This is particularly useful when: + +- You are bulk-importing a large number of documents. Generating document embeddings inside Typesense requires a model inference per document before it can be indexed, which slows down large imports. +- Your cluster serves highly concurrent semantic or hybrid searches while documents are being imported. With built-in models, document embedding and query embedding share the same model instance on each node (and the same GPU, when one is used), so heavy import-time embedding can slow down search-time query embedding. +- You want to generate document embeddings in batches on your own infrastructure, which can be more cost-effective than sizing your Typesense cluster (or GPU) for import-time embedding. + +For example, using the `products` collection defined [above](#creating-an-auto-embedding-field), you can import documents that already carry their embedding vector: + +```bash +curl "http://localhost:8108/collections/products/documents/import?action=create" \ + -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \ + -X POST \ + -d '{"product_name": "Cell phone", "categories": ["electronics"], "embedding": [0.241, 0.909, ...]} +{"product_name": "Laptop", "categories": ["electronics"], "embedding": [0.394, 0.481, ...]}' +``` + +Documents that include the `embedding` field are indexed with the vector given, and only documents that omit the field have their embedding generated by Typesense. + +:::warning Use the same model and prefixes +Since Typesense embeds the query string with the model configured in `model_config` at search time, the document embeddings you provide must be generated with that **same model** (and have the same number of dimensions), otherwise vector distances will be meaningless. + +Also make sure you apply the model's indexing prefix to the document text yourself before embedding, if the model uses one. For example, the `ts/e5-*` family of models embeds documents as `passage: ` and queries as `query: `, as described above. Typesense applies these prefixes automatically only when it generates the embedding. +::: + +:::tip Updates +If you later update any of the `embed.from` source fields of a document without including the embedding field in the update, Typesense will regenerate the embedding for that document. Include the vector field in the update request to keep using your own embeddings. +::: + ## Nearest-neighbor vector search Once you've indexed your embeddings in a vector field, you can now search for documents that are "closest" to a given query vector. @@ -3439,6 +3473,10 @@ curl 'http://localhost:8108/multi_search' \ Typesense will do a keyword search using the `q` parameter, and a nearest neighbor search using the `vector_query` field and combine the results into a ranked set of results using rank fusion as described earlier. +:::tip +If you only want to move **document** embedding generation outside Typesense (for example, to keep bulk imports from competing with search-time query embedding), you don't need to embed the query yourself. Define the field as an auto-embedding field and provide the document vectors at import time, as described in [Providing pre-generated document embeddings for auto-embedding fields](#providing-pre-generated-document-embeddings-for-auto-embedding-fields). Typesense will then generate only the query embedding, and hybrid search works with the regular `query_by` syntax shown above. +::: + :::warning Performance Tip If you expect users to use several-words-long queries in the `q` parameter when doing hybrid search (which is common during [conversational search](./conversational-search-rag.md) for eg), you want to set `drop_tokens_threshold: 0` as an additional search parameter to avoid redundant internal keyword searches and excessive CPU usage. Read more about what this parameter does under [this table](./search.md#typo-tolerance-parameters).