Maintain compatibility with CPython 3.13 #1242

daskol · 2025-03-27T17:18:15Z

See changes in Python C API in CPython 3.13 changelog. Initial discussion is in AUR.

medaminezghal · 2025-03-28T07:05:10Z

@daskol You need to edit the files that use asyncore (Removed in python 3.13) to make it use asyncio
This what I did in the prepare function in PKGBUILD file:

    sed -i 's/_try_asyncore_import/_try_asyncio_import/g' cassandra/cluster.py
    sed -i 's/from cassandra.io.asyncorereactor import AsyncoreConnection/from cassandra.io.asyncioreactor import AsyncioConnection/g' cassandra/cluster.py
    sed -i 's/from cassandra.io.asyncorereactor import AsyncoreConnection/from cassandra.io.asyncioreactor import AsyncioConnection/g' tests/integration/long/test_ipv6.py
    sed -i 's/(AsyncoreConnection,None)/(AsyncioConnection,None)/g' cassandra/cluster.py
    sed -i '/import asynctest/d' tests/unit/io/test_asyncioreactor.py
    sed -i 's/asynctest.TestSelector()/AsyncMock()/g' tests/unit/io/test_asyncioreactor.py
    sed -i '/from unittest.mock import patch/a from unittest.mock import AsyncMock' tests/unit/io/test_asyncioreactor.py

Maybe be you could edit the files to use asyncore or asyncio by case (python>=3.13 or not)

medaminezghal · 2025-03-28T07:07:34Z

@daskol and even after I edit for some reason I can't import the cassandra.io.libevwrapper

daskol · 2025-03-28T08:08:39Z

@medaminezghal What about cqlsh utility? It does not work without this fix because of missing cp313 wheels.

medaminezghal · 2025-03-28T08:24:57Z

@daskol if you build the package without the C extensions it should work (of course after editing the files that use asyncore) and then you can build cqlsh in python 3.13

medaminezghal · 2025-03-28T08:32:56Z

@daskol you can replace any code that use asyncore with asyncio safely https://docs.python.org/3/library/asyncore.html

daskol · 2025-03-28T09:30:59Z

Do pacman -S python-pyasyncore or pip install pyasyncore solve your issue? Initially, I faced with the issue of missing cp313 wheels and build failure of this dreiver from scratch.

medaminezghal · 2025-03-28T10:54:08Z

I think it’s not useful to use unmaintained package as provided in the description. So it’s better to use asyncio

medaminezghal · 2025-04-07T15:51:46Z

@daskol I think you need to fix the problem related to this issue from other project.
There is only 1 file that use asynctest package.
Also the distutils (used to build the C extensions) is deprecated and will be removed soon.

absurdfarce · 2025-05-28T22:50:15Z

Thanks for the PR @daskol!

Have you signed the Contributor License Agreement for contributions to DataStax open source projects? If not you can find it at https://cla.datastax.com/. Thanks!

I mentioned this in another recently submitted PR but it's worth restating here: because the Python driver aims to support all Python runtimes that aren't end-of-life we still have to support Python 3.9.x and up. And since the asyncore reactor is more stable than the current asyncio version I have no plans to remove it anytime soon. There's a lot more detail in my comment.

For the record: if you want to use the libev reactor you'll need libev installed and accessible via the lib path before the Python driver will even attempt to use it. You can find some additional info in our docs on installing the Python driver.

daskol · 2025-05-28T23:19:13Z

@absurdfarce Yes, I signed agreement as soon as you posted this requirement in #1244.

we still have to support Python 3.9.x and up.

This PR exactly improves support for CPython 3.9+.

bschoening · 2025-06-18T12:42:43Z

@daskol since the driver only supports 3.8+, is there any reason the deprecated initializer code cannot be simply removed entirely?

Note: 3.8 was EOL on Oct 2024, so should really be 3.9+

bschoening · 2025-08-14T15:34:18Z

@daskol is there any reason the deprecated initializer code cannot be simply removed entirely?

andy-slac · 2025-09-25T16:50:38Z

Could this be merged relatively soon and a new release created with this fix? Python 3.13 is sort of broken because of this.

absurdfarce · 2025-10-01T21:04:15Z

I agree with this PR but it's worth stating that Python 3.13.x isn't "broken" without this change. Plenty of gcc instances will just report these as warnings about undefined method calls without breaking anything. In fact that's exactly what our Jenkins build does against Python 3.13:

[2025-10-01T04:53:44.415Z] gcc -pthread -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/usr/include/libev -I/usr/local/include -I/opt/local/include -I/usr/include -I/home/jenkins/.pyenv/versions/3.13.5/include/python3.13 -c cassandra/io/libevwrapper.c -o build/temp.linux-x86_64-cpython-313/cassandra/io/libevwrapper.o
[2025-10-01T04:53:44.415Z] cassandra/io/libevwrapper.c: In function ‘PyInit_libevwrapper’:
[2025-10-01T04:53:44.415Z] cassandra/io/libevwrapper.c:668:10: warning: implicit declaration of function ‘PyEval_ThreadsInitialized’ [-Wimplicit-function-declaration]
[2025-10-01T04:53:44.415Z]   668 |     if (!PyEval_ThreadsInitialized()) {
[2025-10-01T04:53:44.415Z]       |          ^~~~~~~~~~~~~~~~~~~~~~~~~
[2025-10-01T04:53:44.415Z] cassandra/io/libevwrapper.c:669:9: warning: ‘PyEval_InitThreads’ is deprecated [-Wdeprecated-declarations]
[2025-10-01T04:53:44.415Z]   669 |         PyEval_InitThreads();
[2025-10-01T04:53:44.415Z]       |         ^~~~~~~~~~~~~~~~~~
[2025-10-01T04:53:44.415Z] In file included from /home/jenkins/.pyenv/versions/3.13.5/include/python3.13/Python.h:124,
[2025-10-01T04:53:44.415Z]                  from cassandra/io/libevwrapper.c:1:
[2025-10-01T04:53:44.415Z] /home/jenkins/.pyenv/versions/3.13.5/include/python3.13/ceval.h:114:37: note: declared here
[2025-10-01T04:53:44.415Z]   114 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
[2025-10-01T04:53:44.415Z]       |

That's against Focal which is using 9.3.0 IIRC. I agree it's not great but it's not the case that Python 3.13.x support is broken across the board.

Also worth mentioning that the wheels for 3.29.2 work just fine. Those were also built on Focal runners.

absurdfarce · 2025-10-01T21:08:52Z

I'm not unsympathetic to your argument that we should remove these initializers entirely @bschoening but at the moment my inclination is to go with something like what's in this PR. Taking this approach rather than removing the calls entirely at least allows for the possibility that you could build this against something other than the officially supported versions of Python. We're obviously not changing the Python versions we support and I don't want to get in the habit of bending the rules to keep older versions happy... but in this case we have a completely acceptable PR which enables that kind of backwards support.

I readily agree it's a judgement call but for now I'm inclined to go with it. This support will likely go away at some point in the future... but for now it seems like a reasonable compromise.

absurdfarce · 2025-10-01T21:10:48Z

Kicked off a Jenkins build just to make sure there aren't any unexpected regressions here. Hard to see how there could be but, you know... computers and all that.

andy-slac · 2025-10-01T21:34:31Z

@absurdfarce, build does work (without building libevwrapper), but runtime is definitely broken. Here is with fresh Python and cassandra-driver from PyPI:

root@6285e3970dc1:/# pip install cassandra-driver
Collecting cassandra-driver
  Downloading cassandra-driver-3.29.2.tar.gz (293 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting geomet<0.3,>=0.1 (from cassandra-driver)
  Downloading geomet-0.2.1.post1-py3-none-any.whl.metadata (1.0 kB)                                                                                                                                                                                                            Collecting click (from geomet<0.3,>=0.1->cassandra-driver)
  Downloading click-8.3.0-py3-none-any.whl.metadata (2.6 kB)                                                                                                                                                                                                                   Collecting six (from geomet<0.3,>=0.1->cassandra-driver)                                                                                                                                                                                                                         Downloading six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB)
Downloading geomet-0.2.1.post1-py3-none-any.whl (18 kB)
Downloading click-8.3.0-py3-none-any.whl (107 kB)                                                                                                                                                                                                                              Downloading six-1.17.0-py2.py3-none-any.whl (11 kB)                                                                                                                                                                                                                            Building wheels for collected packages: cassandra-driver                                                                                                                                                                                                                         Building wheel for cassandra-driver (pyproject.toml) ... done
  Created wheel for cassandra-driver: filename=cassandra_driver-3.29.2-cp313-cp313-linux_x86_64.whl size=344079 sha256=c2fc2ac6ec807ee1acf55454b7b0f3089834351aec1ecb67d97a93b174ddde0a
  Stored in directory: /root/.cache/pip/wheels/ab/e6/16/a8ddddf3d6c34eaa48769f4bc80ea75ad46afcad10e5b4634d
Successfully built cassandra-driver
Installing collected packages: six, click, geomet, cassandra-driver
Successfully installed cassandra-driver-3.29.2 click-8.3.0 geomet-0.2.1.post1 six-1.17.0

root@6285e3970dc1:/# python -c "import cassandra.cluster"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import cassandra.cluster
  File "/usr/local/lib/python3.13/site-packages/cassandra/cluster.py", line 173, in <module>
    raise DependencyException("Unable to load a default connection class", excs)
cassandra.DependencyException: Unable to load a default connection class
The following exceptions were observed:
 - The C extension needed to use libev was not found.  This probably means that you didn't have the required build dependencies when installing the driver.  See http://datastax.github.io/python-driver/installation.html#c-extensions for instructions on installing build dependencies and building the C extension.
 - Unable to import asyncore module.  Note that this module has been removed in Python 3.12 so when using the driver with this version (or anything newer) you will need to use one of the other event loop implementations.

I guess extra packages needed to be installed to avoid this crash, but they are not declared as dependencies.

absurdfarce · 2025-10-01T21:43:33Z

Hey @andy-slac, thanks for the clarification! Yeah, you're seeing those errors because libev itself isn't available on your box. The libevwrapper code referenced in this PR provides interop between the driver and libev but we still require the library itself. This is mentioned in the installation docs, specifically the section on libev.

For what it's worth part of the motivation for my earlier comments was the mess around asyncio support in Python 3.12 and up. There's been a lot of confusion around that point and we've indicated to a number of folks that libev is the recommended reactor for Python 3.12+ (until we can get to a prod-ready asyncio reactor). So I didn't want users to see a suggestion that libev wasn't functionality on Python 3.13 without providing the additional context.

andy-slac · 2025-10-01T22:01:54Z

@absurdfarce, libev is installed on my system, but your wrapper module fails to build:

root@766c7cd5c72c:/# ls -l /usr/lib/x86_64-linux-gnu/libev.so*
lrwxrwxrwx 1 root root    14 Oct 28  2024 /usr/lib/x86_64-linux-gnu/libev.so -> libev.so.4.0.0
lrwxrwxrwx 1 root root    14 Oct 28  2024 /usr/lib/x86_64-linux-gnu/libev.so.4 -> libev.so.4.0.0
-rw-r--r-- 1 root root 67504 Oct 28  2024 /usr/lib/x86_64-linux-gnu/libev.so.4.0.0
root@766c7cd5c72c:/# ls -l /usr/local/lib/python3.13/site-packages/cassandra/io/
total 100
-rw-r--r-- 1 root root   570 Oct  1 21:58 __init__.py
drwxr-xr-x 2 root root  4096 Oct  1 21:58 __pycache__
-rw-r--r-- 1 root root  7655 Oct  1 21:58 asyncioreactor.py
-rw-r--r-- 1 root root 14754 Oct  1 21:58 asyncorereactor.py
-rw-r--r-- 1 root root  6729 Oct  1 21:58 eventletreactor.py
-rw-r--r-- 1 root root  4277 Oct  1 21:58 geventreactor.py
-rw-r--r-- 1 root root 12848 Oct  1 21:58 libevreactor.py
-rw-r--r-- 1 root root 22348 Oct  1 21:58 libevwrapper.c
-rw-r--r-- 1 root root 10398 Oct  1 21:58 twistedreactor.py

absurdfarce · 2025-10-01T22:45:32Z

@andy-slac I'm not 💯 sure what the root cause might be in your case but a couple things do occur to me.

First, you need to have both libev and it's corresponding dev package installed; without the dev package you won't get the headers necessary to build everything.

Second, the directories where libev is installed on your system might not be on the default search path. setup.py will search a set of default paths to try and find libev (details here) and at least the paths you mention in your comment are different from those defaults. You can always try defining CASS_DRIVER_LIBEV_INCLUDES and CASS_DRIVER_LIBEV_LIBS to point to the correct directories on your system to see if that helps.

absurdfarce · 2025-10-01T22:48:50Z

Jenkins run has finished and looks good, saw improvement in the expected area around the libev extension build on Python 3.13. No other errors were introduced.

andy-slac · 2025-10-02T01:13:46Z

@absurdfarce, in my case everything was installed properly, but wrapper extension failed to build because gcc generated an error for an implicit declaration of PyEval_ThreadsInitialized. I think this is a correct behavior, implicit declarations should not be allowed by a modern compiler. Here is the log from pip install:

2025-10-01T22:04:21,136   gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/usr/include/libev -I/usr/local/include -I/opt/local/include -I/usr/include -I/usr/local/include/python3.13 -c cassandra/io/libevwrapper.c -o build/temp.linux-x86_64-cpython-313/cassandra/io/libevwrapper.o
2025-10-01T22:04:21,186   cassandra/io/libevwrapper.c: In function <E2><80><98>PyInit_libevwrapper<E2><80><99>:
2025-10-01T22:04:21,186   cassandra/io/libevwrapper.c:668:10: error: implicit declaration of function <E2><80><98>PyEval_ThreadsInitialized<E2><80><99> [-Wimplicit-function-declaration]
2025-10-01T22:04:21,186     668 |     if (!PyEval_ThreadsInitialized()) {
2025-10-01T22:04:21,186         |          ^~~~~~~~~~~~~~~~~~~~~~~~~
2025-10-01T22:04:21,186   cassandra/io/libevwrapper.c:669:9: warning: <E2><80><98>PyEval_InitThreads<E2><80><99> is deprecated [-Wdeprecated-declarations]
2025-10-01T22:04:21,186     669 |         PyEval_InitThreads();
2025-10-01T22:04:21,186         |         ^~~~~~~~~~~~~~~~~~
2025-10-01T22:04:21,186   In file included from /usr/local/include/python3.13/Python.h:125,
2025-10-01T22:04:21,186                    from cassandra/io/libevwrapper.c:1:
2025-10-01T22:04:21,186   /usr/local/include/python3.13/ceval.h:114:37: note: declared here
2025-10-01T22:04:21,186     114 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
2025-10-01T22:04:21,186         |                                     ^~~~~~~~~~~~~~~~~~
2025-10-01T22:04:21,188   command '/usr/bin/gcc' failed with exit code 1

Thanks for merging the patch, when should we expect a new release?

bschoening · 2025-10-02T12:26:57Z

#if PY_MAJOR_VERSION < 3 && PY_MINOR_VERSION < 7

Should this be PY_MAJOR_VERSION == 3? Isn't this checking for version < 2.7?

absurdfarce · 2025-10-02T20:42:17Z

Great catch @bschoening... you're absolutely right. I'm angry at myself for missing that!

Changing that to:

#if PY_MAJOR_VERSION < 3 || (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION < 7)

in a ninja fix. I'll then run the full test suite on it just to make sure it's good.

absurdfarce · 2025-10-02T20:45:34Z

Fix in a5c6c5f

absurdfarce · 2025-10-02T23:03:26Z

Jenkins run is clean, calling this good

Maintain compatibility with CPython 3.13

27969fc

kheina mentioned this pull request May 23, 2025

CPython 3.13 Compat and Asyncore Removal #1244

Closed

absurdfarce added the cla-missing label May 28, 2025

absurdfarce removed the cla-missing label Jun 17, 2025

xylar mentioned this pull request Aug 5, 2025

Rebuild for python 3.13 conda-forge/cassandra-driver-feedstock#49

Closed

absurdfarce merged commit b144a84 into datastax:master Oct 1, 2025
2 of 3 checks passed

Maintain compatibility with CPython 3.13 #1242

Maintain compatibility with CPython 3.13 #1242

Uh oh!

Conversation

daskol commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

medaminezghal commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

medaminezghal commented Mar 28, 2025

Uh oh!

daskol commented Mar 28, 2025

Uh oh!

medaminezghal commented Mar 28, 2025

Uh oh!

medaminezghal commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daskol commented Mar 28, 2025

Uh oh!

medaminezghal commented Mar 28, 2025

Uh oh!

medaminezghal commented Apr 7, 2025

Uh oh!

absurdfarce commented May 28, 2025

Uh oh!

daskol commented May 28, 2025

Uh oh!

bschoening commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bschoening commented Aug 14, 2025

Uh oh!

andy-slac commented Sep 25, 2025

Uh oh!

absurdfarce commented Oct 1, 2025

Uh oh!

absurdfarce commented Oct 1, 2025

Uh oh!

absurdfarce commented Oct 1, 2025

Uh oh!

andy-slac commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

absurdfarce commented Oct 1, 2025

Uh oh!

andy-slac commented Oct 1, 2025

Uh oh!

absurdfarce commented Oct 1, 2025

Uh oh!

absurdfarce commented Oct 1, 2025

Uh oh!

Uh oh!

andy-slac commented Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bschoening commented Oct 2, 2025

Uh oh!

absurdfarce commented Oct 2, 2025

Uh oh!

absurdfarce commented Oct 2, 2025

Uh oh!

absurdfarce commented Oct 2, 2025

Uh oh!

Uh oh!

daskol commented Mar 27, 2025 •

edited

Loading

medaminezghal commented Mar 28, 2025 •

edited

Loading

medaminezghal commented Mar 28, 2025 •

edited

Loading

bschoening commented Jun 18, 2025 •

edited

Loading

andy-slac commented Oct 1, 2025 •

edited

Loading

andy-slac commented Oct 2, 2025 •

edited

Loading