Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/cloud/manage/backup-restore.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ If you want to do a PITR for a cluster, you can follow the steps below:
- Select the target time that you want to restore to.
![](../images/target-time.png)
::: warning
The target time should be within the first first recoverability point and last archived wal time.
The target time should be within the first recoverability point and last archived wal time.
:::


Expand Down
2 changes: 1 addition & 1 deletion src/cloud/manage/terraform.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Terraform Integration Overview with VectorChord Cloud

This guide walks you through the process of installing and configuring the PGVetco.rs Cloud provider.
This guide walks you through the process of installing and configuring the PGVecto.rs Cloud provider.

## 1. Prerequisites

Expand Down
2 changes: 1 addition & 1 deletion src/pgvecto_rs/admin/fdw.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ services:
networks:
localnet:

forigen_db:
foreign_db:
image: tensorchord/pgvecto-rs:pg15-v0.2.0
ports:
- "5433:5432"
Expand Down
2 changes: 1 addition & 1 deletion src/pgvecto_rs/admin/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ SHOW search_path;
SET search_path = "$user, public";
```

#### 2. Check exist indexes
#### 2. Check existing indexes

For indexes created at vector columns, please retrieve the definition and record it in a convenient location. This information is necessary for the recovery of the indexes at a later stage.

Expand Down
2 changes: 1 addition & 1 deletion src/pgvecto_rs/admin/upgrading.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ You may get this error if some [indexes](../usage/indexing.md) were created befo

```
ERROR: index public.this_is_index depends on operator class vector_cos_ops for access method vectors
index public.this_is_index depends on operator class vector_cos_ops for access method vectorscannot drop operator class vector_cos_ops for access method vectors because other objects depend on it
index public.this_is_index depends on operator class vector_cos_ops for access method vectors cannot drop operator class vector_cos_ops for access method vectors because other objects depend on it

ERROR: cannot drop operator class vector_cos_ops for access method vectors because other objects depend on it
SQL state: 2BP01
Expand Down
2 changes: 1 addition & 1 deletion src/pgvecto_rs/integration/langchain.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ Score: 0.29536954
- 💃 **Easy to use**: pgvecto.rs is a Postgres extension, which means that you can use it directly within your existing database. This makes it easy to integrate into your existing workflows and applications.
- 🔗 **Async indexing**: pgvecto.rs's index is asynchronously constructed by the background threads and does not block insertions and always ready for new queries.
- 🥅 **Filtering**: pgvecto.rs supports filtering. You can set conditions when searching or retrieving points. This is the missing feature of other postgres extensions.
- 🧮 **Quantization**: pgvecto.rs supports scalar quantization and product qutization up to 64x.
- 🧮 **Quantization**: pgvecto.rs supports scalar quantization and product quantization up to 64x.
- 🦀 **Rewrite in Rust**: Rust's strict compile-time checks ensure memory safety, reducing the risk of bugs and security issues commonly associated with C extensions.

## Comparison with pgvector
Expand Down
2 changes: 1 addition & 1 deletion src/pgvecto_rs/use-case/adaptive-retrieval.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ But you could safely remove some numbers from the end of the sequence and still

::: details

You may need normalization to ensure that the shortened embeddings compatible with some distance calculation e.g. dot poroduct. OpenAI's API will help you on this if you are calling `text-embedding-3-large` to generate a lower dimension embedding directly, instead of truncating the original embeddings on your own.
You may need normalization to ensure that the shortened embeddings compatible with some distance calculation e.g. dot product. OpenAI's API will help you on this if you are calling `text-embedding-3-large` to generate a lower dimension embedding directly, instead of truncating the original embeddings on your own.

```python
embedding_3072_dim_from_openai = [-0.00611, -0.00269, ...]
Expand Down
4 changes: 2 additions & 2 deletions src/vectorchord/admin/kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ You can use `kubectl port-forward` to connect to PostgreSQL cluster.
```shell
$ kubectl port-forward services/vchord-rw 5432:5432
$ psql -h 127.0.0.1 -d tensorchord -U tensorchord
ttensorchord=> \dx
tensorchord=> \dx
List of installed extensions
Name | Version | Schema | Description
---------+---------+------------+---------------------------------------------------------------------------------------------
Expand Down Expand Up @@ -225,7 +225,7 @@ spec:
- "vchord"
```

We can check the extension is alroady installed successfully.
We can check the extension is already installed successfully.

```shell
$ kubectl port-forward services/vchord-rw 5432:5432
Expand Down
6 changes: 3 additions & 3 deletions src/vectorchord/admin/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ CREATE EXTENSION vchord CASCADE;

::: info

This will install vectorchord and pgvector at public schema, if other schema is picked, please subsititue `public` with your schema at the migration command.
This will install vectorchord and pgvector at public schema, if other schema is picked, please substitute `public` with your schema at the migration command.

:::

Expand All @@ -80,7 +80,7 @@ postgres=# \dx

:::

#### 2. Check exist indexes
#### 2. Check existing indexes

For indexes created at vector columns, please retrieve the definition and record it in a convenient location.
This information is necessary for the recovery of the indexes at a later stage.
Expand Down Expand Up @@ -145,7 +145,7 @@ spherical_centroids = false
$$);
```

For more information of create index at `VectorChord`, see [indexing](../usage/indexing).
For more information on creating index at `VectorChord`, see [indexing](../usage/indexing).

#### 5. Clean the environment

Expand Down
2 changes: 1 addition & 1 deletion src/vectorchord/usage/graph-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ ef_construction = 128
$$);
```

The index building can be sped up by using multiple processes. Refer to [PostgreSQL Tuning](performance-tuning.md#indexing).
The index building can be sped up by using multiple processes. Refer to [PostgreSQL Tuning](performance-tuning.md).

## Reference

Expand Down
2 changes: 1 addition & 1 deletion src/vectorchord/usage/indexing.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ SET vchordrq.probes TO '10';
SELECT * FROM items ORDER BY embedding <-> '[3,1,2]' LIMIT 10;
```

The second step, inserting rows, can be parallelized using multiple processes. Refer to [PostgreSQL Tuning](performance-tuning.md#indexing).
The second step, inserting rows, can be parallelized using multiple processes. Refer to [PostgreSQL Tuning](performance-tuning.md).

For most datasets using cosine similarity, enabling `residual_quantization` and `build.internal.spherical_centroids` improves both QPS and recall.

Expand Down
33 changes: 18 additions & 15 deletions src/vectorchord/usage/postgresql-tuning.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
# PostgreSQL Tuning

## Indexing

Indexing may be parallelized. Parallelism increases server load and is therefore not enabled by default in PostgreSQL. To enable it, refer to the following SQL.

```sql
-- Sets the maximum number of background processes that the cluster can support.
-- It's recommended to set it to the sum of your desired parallelism plus 8.
-- For example, if you want to use 16 threads for parallel processing, set the value to 24.
-- It's recommended to set it to at least the sum of your desired parallelism plus 8.
-- For example, if you want to use 16 threads for parallel processing, set the value at least to 24.
-- Depending on your workload, this value might be useful to be higher.
-- Note: A restart is required for this setting to take effect.
ALTER SYSTEM SET max_worker_processes = 24;

Expand All @@ -28,27 +27,31 @@ The number of parallel workers also depends on the table's configuration. By def
ALTER TABLE items set (parallel_workers = 15);
```

## Search

To ensure the search performs efficiently, you may need to adjust some PostgreSQL parameters.
You need to tune a few settings to improve I/O performance.

```sql
-- Sets the amount of memory the database server uses for shared memory buffers.
-- For typical database workloads, this value is recommended to be 40% of the total memory.
-- For vector database workloads, this value is recommended to be 80% of the total memory.
-- Note: A restart is required for this setting to take effect.
ALTER SYSTEM SET shared_buffers = '16GB';

-- Sets the number of concurrent disk I/O operations that PostgreSQL expects can be executed simultaneously.
-- It's recommended to set it to 200 when using SSDs for storage.
SET effective_io_concurrency = 200;
SET maintenance_io_concurrency = 200;
```

We have observed that in certain cases, malfunctioning JIT can negatively impact performance. If you do not rely on it, you may choose to disable it.

-- We have observed that in certain cases, malfunctioning JIT can negatively impact performance.
-- If you do not rely on it, you may choose to disable it.
```sql
SET jit = off;
```

Finally, you need to set the size of PostgreSQL's shared buffers.

```sql
-- Sets the amount of memory the database server uses for shared memory buffers.
-- Depending on your workload, this value is recommended to be 25%~40% of the total memory.
-- For vector database workloads, this value might be useful to be 80% of the total memory.
-- Note: A restart is required for this setting to take effect.
ALTER SYSTEM SET shared_buffers = '16GB';
```

## Asynchronous I/O {#asynchronous-i-o}

PostgreSQL 17 introduced the streaming I/O API for the first time. Building on this, PostgreSQL 18 introduces asynchronous I/O. With asynchronous I/O, PostgreSQL can initiate multiple read operations and continue processing while waiting for the results. This can improve the performance of queries that need to read large amounts of data from disk. PostgreSQL offers multiple asynchronous I/O methods. You can adjust them as needed.
Expand Down
2 changes: 1 addition & 1 deletion src/vectorchord/use-case/colqwen2-modal.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Think about the limitations of traditional OCR: complex layouts get mangled, tab

> T*he ColPali architecture (Image from the ColPali paper)*

**ColQwen2** builds upon this powerful ColPali architecture but swaps the underlying VLM for the more recent **Qwen2-VL-2B**. It generates [**ColBERT**\-style](https://arxiv.org/abs/2004.12832) multi-vector representations, capturing fine-grained details from both text and images. As seen on the [**vidro-leaderboard**,](https://huggingface.co/spaces/vidore/vidore-leaderboard) ColQwen2 delivers impressive performance with practical model size.
**ColQwen2** builds upon this powerful ColPali architecture but swaps the underlying VLM for the more recent **Qwen2-VL-2B**. It generates [**ColBERT**\-style](https://arxiv.org/abs/2004.12832) multi-vector representations, capturing fine-grained details from both text and images. As seen on the [**vidore-leaderboard**,](https://huggingface.co/spaces/vidore/vidore-leaderboard) ColQwen2 delivers impressive performance with practical model size.

## How Does VectorChord Enable ColQwen2 in Postgres?

Expand Down
2 changes: 1 addition & 1 deletion src/vectorchord/use-case/hybrid-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ For embedding, we can use a pre-trained embedding model like `BGE-M3` to generat

For validation, we use the [BEIR](https://github.com/beir-cellar/beir) dataset, a heterogeneous benchmark for information retrieval. It is easy to use and allows you to evaluate your models across 15+ diverse IR datasets.

First, you need to load your data into PostgreSQL.Then you can use the following SQL query to generate embeddings for the documents.
First, you need to load your data into PostgreSQL. Then you can use the following SQL query to generate embeddings for the documents.

```python
with self.conn.cursor() as cursor:
Expand Down