Skip to content

Conversation

@mullimanko
Copy link
Contributor

Closes #24558.

For example, this PR enables the to write to the database multiple times within the same transaction:

import adbc_driver_postgresql.dbapi as pg_dbapi
import polars as pl

conn_str ="postgresql://username:password@host:port/database"

df = pl.DataFrame({"a": [1], "b": [4]})

conn = pg_dbapi.connect(conn_str, autocommit=False)
print(f"Inspect autocommit status: {conn.adbc_connection.get_option("adbc.connection.autocommit")}")

df.write_database("public.table1", connection=conn, engine="adbc")
(df*2).write_database("public.table2", connection=conn, engine="adbc")
conn.commit()

Add autocommit=True to adbc engine
Delete `conn.commit()` becuase `autocommit=True`
@github-actions github-actions bot added A-io-database Area: reading/writing to databases fix Bug fix python Related to Python Polars labels Sep 21, 2025
@codecov
Copy link

codecov bot commented Sep 21, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.74%. Comparing base (664656e) to head (252592c).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24559      +/-   ##
==========================================
- Coverage   81.76%   81.74%   -0.02%     
==========================================
  Files        1684     1684              
  Lines      229458   229457       -1     
  Branches     2952     2952              
==========================================
- Hits       187616   187579      -37     
- Misses      41099    41135      +36     
  Partials      743      743              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@alexander-beedie alexander-beedie self-assigned this Sep 22, 2025
@ritchie46 ritchie46 force-pushed the main branch 3 times, most recently from ddf5907 to d0914d4 Compare September 27, 2025 11:06
@henryharbeck
Copy link
Contributor

While I appreciate this change (and it is arguably the correct behaviour), I believe that it is breaking longstanding behaviour.

For example, the code below currently writes a table to the database, but under these changes, would no longer do so.
It forces the user to change connect(uri) to connect(uri, autocommit=True) or to add conn.commit() before exiting the context manager - that being the breaking change.

from adbc_driver_postgresql.dbapi import connect
import polars as pl

df = pl.DataFrame({"a": 1})
uri = "postgresql://postgres:postgres@localhost:5433"
# autocommit defaults to False in `connect()`
with connect(uri) as conn, conn.cursor() as cur:
    df.write_database("public.my_table", conn)

I appreciate that by default, the ADBC connections have autocommit off (for PEP 249 compliance) (Source), but at the moment Polars is still committing the user's behalf.

Perhaps there is a way to introduce this additional flexibility without being a breaking change. A way to specify that you do not want Polars to commit on your behalf.

@mullimanko
Copy link
Contributor Author

Ok, I understand that this would be a breaking change. My opinion is that having autocommit=False and then polars still doing the conn.commit() on the users behalf undermines somewhat the purpose of the autocommit=False default of PEP249 because for the user the commit happens automatically anyway.
However, perhaps compliance with PEP 249 can be maintained with the following suggestion? Introduce a boolean Config option to give the possibility to turn off the conn.commit() in frame.py, e.g. something like Config.set_autocommit for ADBC. If other db engines have the same issue, they should also stick to this.

@ritchie46 ritchie46 force-pushed the main branch 3 times, most recently from 90ceb7b to e9fce55 Compare October 26, 2025 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-io-database Area: reading/writing to databases fix Bug fix python Related to Python Polars

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow manual commit of write_database with engine="adbc"

3 participants