Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 0 additions & 46 deletions docs-site/content/guide/syncing-data-into-typesense.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,52 +96,6 @@ This buffer-based approach provides several benefits:
- The buffer provides an audit trail of changes
- You can tune the processing frequency based on your real-time requirements

### Using Sequin

[Sequin](https://sequinstream.com) streams data from your Postgres database to Typesense in real-time. Any change to your database (whether from your application, an internal tool, or another process) will be immediately reflected in Typesense.

This approach comes with several benefits:

- Sequin uses logical replication, which adds virtually no overhead to your database (unlike polling and triggers)
- The direct integration with Typesense leverages the bulk import API to efficiently load every create and update. It also supports deletes out of the box.
- You can tune Sequin's batching behavior to your real-time requirements
- Sequin comes with transforms, backfills, filtering, and built-in retries to ensure your Postgres tables are perfectly replicated into Typesense.

#### Setup overview

:::tip
Read Sequin's Typesense [Quickstart](https://sequinstream.com/docs/quickstart/typesense) for a step-by-step guide.
:::

1. Setup Sequin locally or create a cloud account.
2. Connect your Postgres database to Sequin.
3. Create a Typesense sink for each table you want to replicate to a Typesense collection.

Here's an [example `Sequin.yaml`](https://sequinstream.com/docs/reference/sequin-yaml#typesense-sink) showing how to sink a `products` table to Typesense:

```yaml
databases:
- name: "prod-db"
username: "postgres"
password: "postgres"
hostname: "my-database-instance.abcd1234wxyz.us-east-1.rds.amazonaws.com"
database: "prod"
port: 5432
slot_name: "sequin_slot"
publication_name: "sequin_pub"
sinks:
- name: "typesense-sink"
database: "prod-db"
table: "public.products"
destination:
type: "typesense"
endpoint_url: "https://your-typesense-server:8108"
collection_name: "products"
api_key: "your-api-key"
batch_size: 1000
timeout_seconds: 5
```

## Full re-indexing

In addition to the above strategies, you could also do a re-index of your entire dataset on say a nightly basis, just to make sure any gaps in the synced data due to schema validation errors, network issues, failed retries etc are addressed and filled.
Expand Down