You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Added pg_binary_protocol attribute to derive send and receive functions for PostgresType (#2068)
This PR generally addresses issues such as #2060, #1446, #1364, and
pksunkara/pgx_ulid#27, and supersedes PR #887.
- [x] Introduces the generation of the `SEND` and `RECEIVE` bindings.
- [x] Introduces the generation of the `*_recv` and `*_send` functions
in rust, based on the existing `serde_cbor`-based methods
[`cbor_encode`](https://github.com/pgcentralfoundation/pgrx/blob/231acce2448ce7df2b9f01cb0962077122a0cf83/pgrx/src/datum/varlena.rs#L380-L397)
and
[`cbor_decode`](https://github.com/pgcentralfoundation/pgrx/blob/231acce2448ce7df2b9f01cb0962077122a0cf83/pgrx/src/datum/varlena.rs#L399-L409)
- [x] Introduces the `#[pg_binary_protocol]` decorator to be used
alongside the `PostgresType` derive
This PR is not API breaking, as even if people have created functions
with naming clashes they would still need to add the attribute on the
interested structs.
Many thanks to @YohDeadfall and @zommiommy for their help in
understanding better pgrx internals and postgres's binary protocol.
## The generated Rust & SQL
Given a custom rust type such as:
```rust
#[derive(serde::Serialize, serde::Deserialize, pgrx::PostgresType)]
#[pg_binary_protocol]
pub struct PositiveU32 {
pub field: i32,
}
```
The resulting generated rust bindings look like:
```rust
#[doc(hidden)]
#[::pgrx::pgrx_macros::pg_extern(immutable, strict, parallel_safe)]
pub fn positiveu32_recv(
internal: ::pgrx::datum::Internal,
) -> PositiveU32 {
let buf = unsafe { internal.get_mut::<::pgrx::pg_sys::StringInfoData>().unwrap() };
let mut serialized = ::pgrx::StringInfo::new();
serialized.push_bytes(&[0u8; ::pgrx::pg_sys::VARHDRSZ]);
serialized.push_bytes(unsafe {
core::slice::from_raw_parts(
buf.data as *const u8,
buf.len as usize
)
});
let size = serialized.len();
let varlena = serialized.into_char_ptr();
unsafe{
::pgrx::set_varsize_4b(varlena as *mut ::pgrx::pg_sys::varlena, size as i32);
buf.cursor = buf.len;
::pgrx::datum::cbor_decode(varlena as *mut ::pgrx::pg_sys::varlena)
}
}
#[doc(hidden)]
#[::pgrx::pgrx_macros::pg_extern(immutable, strict, parallel_safe)]
pub fn positiveu32_send(input: PositiveU32) -> Vec<u8> {
use ::pgrx::datum::{FromDatum, IntoDatum};
let Some(datum): Option<::pgrx::pg_sys::Datum> = input.into_datum() else {
::pgrx::error!("Datum of type `{}` is unexpectedly NULL.", stringify!(#name));
};
unsafe {
let Some(serialized): Option<Vec<u8>> = FromDatum::from_datum(datum, false) else {
::pgrx::error!("Failed to CBOR-serialize Datum to type `{}`.", stringify!(#name));
};
serialized
}
}
```
And the associated generated SQL looks like:
```sql
/* <begin connected objects> */
/*
This file is auto generated by pgrx.
The ordering of items is not stable, it is driven by a dependency graph.
*/
/* </end connected objects> */
/* <begin connected objects> */
-- utils/diesel-pgrx/example_extension/src/lib.rs:16
-- example_extension::PositiveU32
CREATE TYPE PositiveU32;
-- utils/diesel-pgrx/example_extension/src/lib.rs:16
-- example_extension::positiveu32_in
CREATE FUNCTION "positiveu32_in"(
"input" cstring /* core::option::Option<&core::ffi::c_str::CStr> */
) RETURNS PositiveU32 /* core::option::Option<example_extension::PositiveU32> */
IMMUTABLE PARALLEL SAFE
LANGUAGE c /* Rust */
AS 'MODULE_PATHNAME', 'positiveu32_in_wrapper';
-- utils/diesel-pgrx/example_extension/src/lib.rs:16
-- example_extension::positiveu32_out
CREATE FUNCTION "positiveu32_out"(
"input" PositiveU32 /* example_extension::PositiveU32 */
) RETURNS cstring /* alloc::ffi::c_str::CString */
IMMUTABLE STRICT PARALLEL SAFE
LANGUAGE c /* Rust */
AS 'MODULE_PATHNAME', 'positiveu32_out_wrapper';
-- utils/diesel-pgrx/example_extension/src/lib.rs:16
-- example_extension::positiveu32_recv
CREATE FUNCTION "positiveu32_recv"(
"internal" internal /* pgrx::datum::internal::Internal */
) RETURNS PositiveU32 /* example_extension::PositiveU32 */
IMMUTABLE STRICT PARALLEL SAFE
LANGUAGE c /* Rust */
AS 'MODULE_PATHNAME', 'positiveu32_recv_wrapper';
-- utils/diesel-pgrx/example_extension/src/lib.rs:16
-- example_extension::positiveu32_send
CREATE FUNCTION "positiveu32_send"(
"input" PositiveU32 /* example_extension::PositiveU32 */
) RETURNS bytea /* alloc::vec::Vec<u8> */
IMMUTABLE STRICT PARALLEL SAFE
LANGUAGE c /* Rust */
AS 'MODULE_PATHNAME', 'positiveu32_send_wrapper';
-- utils/diesel-pgrx/example_extension/src/lib.rs:16
-- example_extension::PositiveU32
CREATE TYPE PositiveU32 (
INTERNALLENGTH = variable,
INPUT = positiveu32_in, /* example_extension::positiveu32_in */
OUTPUT = positiveu32_out, /* example_extension::positiveu32_out */
RECEIVE = positiveu32_recv, /* example_extension::positiveu32_recv */
SEND = positiveu32_send, /* example_extension::positiveu32_send */
STORAGE = extended
);
/* </end connected objects> */
```
## On Diesel traits
I use [`diesel`](https://docs.rs/diesel/latest/diesel/) extensively in
my projects, and it uses the binary protocol to send data to PostgreSQL.
This PR makes it possible to use Postgres's binary protocol with `pgrx`
type, so I can now implement diesel's
[`ToSql`](https://docs.rs/diesel/latest/diesel/serialize/trait.ToSql.html)
and
[`FromSql`](https://docs.rs/diesel/latest/diesel/deserialize/trait.FromSql.html)
traits as follows:
```rust
#[derive(
Debug, diesel::FromSqlRow, diesel::AsExpression,
serde::Serialize, serde::Deserialize, pgrx::PostgresType
)]
#[pg_binary_protocol]
#[diesel(sql_type = MyCustomTypeDiesel)]
pub struct MyCustomType {
pub field1: String,
pub field2: i32,
}
#[derive(
Debug, Clone, Copy, Default, diesel::query_builder::QueryId, diesel::sql_types::SqlType,
)]
#[diesel(postgres_type(name = "my_custom_type"))]
pub struct MyCustomTypeDiesel;
impl diesel::serialize::ToSql<MyCustomTypeDiesel, diesel::pg::Pg> for MyCustomType
{
fn to_sql<'b>(&'b self, out: &mut diesel::serialize::Output<'b, '_, diesel::pg::Pg>) -> diesel::serialize::Result {
use std::io::Write;
serde_cbor::to_writer(out, self)?;
Ok(diesel::serialize::IsNull::No)
}
}
impl diesel::deserialize::FromSql<MyCustomTypeDiesel, diesel::pg::Pg> for MyCustomType
{
fn from_sql(raw: diesel::pg::PgValue<'_>) -> diesel::deserialize::Result<Self> {
Ok(serde_cbor::from_slice(raw.as_bytes())?)
}
}
```
*P.S. Sorry for the very small commits, but I could only figure out how
to compile pgrx inside a Docker and therefore to test it in there I had
to push every single small test edit.*
*P.P.S I have also tried to add support for fixed-size types, but there
is too much varlena-specific methods at this time and the amount of code
it would be needed for that makes it necessarily a separate PR*
0 commit comments