You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**ETL** is a Rust framework by [Supabase](https://supabase.com) that enables you to build high-performance, real-time data replication applications for PostgreSQL. Whether you're creating ETL pipelines, implementing CDC (Change Data Capture), or building custom data synchronization solutions, ETL provides the building blocks you need.
21
23
22
-
This crate builds abstractions on top of Postgres's [logical streaming replication protocol](https://www.postgresql.org/docs/current/protocol-logical-replication.html) and pushes users towards the pit of success without letting them worry about low level details of the protocol.
24
+
Built on top of PostgreSQL's [logical streaming replication protocol](https://www.postgresql.org/docs/current/protocol-logical-replication.html), ETL handles the low-level complexities of database replication while providing a clean, Rust-native API that guides you towards the pit of success.
23
25
24
26
## Table of Contents
25
27
@@ -35,27 +37,88 @@ This crate builds abstractions on top of Postgres's [logical streaming replicati
35
37
36
38
## Features
37
39
38
-
The `etl` crate supports the following destinations:
40
+
**Core Capabilities:**
41
+
- 🚀 **Real-time replication**: Stream changes from PostgreSQL as they happen
42
+
- 🔄 **Multiple destinations**: Support for various data warehouses and databases (coming soon)
To use `etl` in your Rust project, add the core library and desired destinations via git dependencies in `Cargo.toml`:
54
+
Add ETL to your Rust project via git dependencies in `Cargo.toml`:
47
55
48
56
```toml
49
57
[dependencies]
50
58
etl = { git = "https://github.com/supabase/etl" }
51
-
etl-destinations = { git = "https://github.com/supabase/etl", features = ["bigquery"] }
52
59
```
53
60
54
-
The `etl` crate provides the core replication functionality, while `etl-destinations` contains destination-specific implementations. Each destination is behind a feature of the same name in the `etl-destinations` crate. The git dependency is needed for now because the crates are not yet published on crates.io.
61
+
> **Note**: ETL is currently distributed via Git while we prepare for the initial crates.io release.
55
62
56
63
## Quickstart
57
64
58
-
To quickly get started with `etl`, see the [etl-examples](etl-examples/README.md) crate which contains practical examples and detailed setup instructions.
65
+
Get up and running with ETL in minutes using the built-in memory destination:
**Need production destinations?** Add the `etl-destinations` crate with specific features:
114
+
115
+
```toml
116
+
[dependencies]
117
+
etl = { git = "https://github.com/supabase/etl" }
118
+
etl-destinations = { git = "https://github.com/supabase/etl", features = ["bigquery"] }
119
+
```
120
+
121
+
For comprehensive examples and tutorials, visit the [etl-examples](etl-examples/README.md) crate and our [documentation](https://supabase.github.io/etl).
59
122
60
123
## Database Setup
61
124
@@ -110,3 +173,9 @@ This limits performance for large tables. We plan to address this once the ETL s
110
173
## License
111
174
112
175
Distributed under the Apache-2.0 License. See `LICENSE` for more information.
176
+
177
+
---
178
+
179
+
<palign="center">
180
+
Made with ❤️ by the <ahref="https://supabase.com">Supabase</a> team
0 commit comments