persistent release 6.2 seems broken

[dataflake]

While updating dependencies for Zope I noticed that tons of tests do not import when using persistent version 6.2 instead of 6.1.1. The import failures happen on all Python versions on the GHA Windows and macOS runners and I can confirm these failures locally on macOS/arm64.

GHA test output: https://github.com/zopefoundation/Zope/actions/runs/18198493940

Example traceback:

Traceback (most recent call last):
  File "/Users/runner/work/Zope/Zope/src/App/tests/testImageFile.py", line 7, in <module>
    from Testing.ZopeTestCase.warnhook import WarningsHook
  File "/Users/runner/work/Zope/Zope/src/Testing/ZopeTestCase/__init__.py", line 22, in <module>
    from Testing.ZopeTestCase import ZopeLite as Zope2
  File "/Users/runner/work/Zope/Zope/src/Testing/ZopeTestCase/ZopeLite.py", line 30, in <module>
    import App.ProductContext  # NOQA
  File "/Users/runner/work/Zope/Zope/src/App/ProductContext.py", line 20, in <module>
    from AccessControl.Permission import registerPermissions
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/AccessControl/__init__.py", line 15, in <module>
    from AccessControl.Implementation import setImplementation
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/AccessControl/Implementation.py", line 106, in <module>
    setImplementation(_default_implementation)
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/AccessControl/Implementation.py", line 58, in setImplementation
    from AccessControl import ImplC as impl
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/AccessControl/ImplC.py", line 16, in <module>
    from AccessControl.cAccessControl import PermissionRole
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/AccessControl/SimpleObjectPolicies.py", line 48, in <module>
    from BTrees.IIBTree import IIBTree
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/BTrees/__init__.py", line 62, in <module>
    mod = create_module(family)
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/BTrees/_module_builder.py", line 229, in create_module
    populate_module(vars(mod), key_type, val_type, iface, mod)
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/BTrees/_module_builder.py", line 137, in populate_module
    from ._base import _fix_pickle
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/BTrees/_base.py", line 17, in <module>
    from persistent import Persistent
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/persistent/__init__.py", line 36, in <module>
    from persistent import picklecache as _picklecache
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/persistent/picklecache.py", line 29, in <module>
    from persistent.ring import Ring
  File "/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/persistent/ring.py", line 18, in <module>
    from persistent import _ring
ImportError: cannot import name '_ring' from partially initialized module 'persistent' (most likely due to a circular import) (/Users/runner/work/Zope/Zope/.tox/py39-universal2/lib/python3.9/site-packages/persistent/__init__.py)

[dataflake]

The problem doesn't show in the persistent unit tests themselves, it seems to affect packages that depend on it. So for a simpler test case than Zope I manually ran the GHA tests in the Persistence package:

https://github.com/zopefoundation/Persistence/actions/runs/18218313237

The only difference is that tests on Python 3.14, which Zope does not test yet, actually pass. But all other Python versions fail on macOS and Windows runners with the ImportError similar to the one above.

[dataflake]

More debugging data points from inside a Persistence checkout:

If I force-install the persistent dependency inside the tox.init [testenv] deps list as persistent == 6.2 the tests fail. This will grab the wheel from PyPI.

If I use git+https://github.com/zopefoundation/persistent.git@6.2#egg=persistent instead, which grabs the sources for 6.2 and then builds the extensions locally, the tests pass.

This could be an issue with the 6.2 wheels on PyPI?

[dataflake]

When I compare the contents of the wheels between version 6.1.1 and 6.2 for Python 3.9 on macOS/arm64 (persistent-6.1.1-cp39-cp39-macosx_11_0_arm64.whl and persistent-6.2-cp39-cp39-macosx_11_0_arm64.whl) I see that the 6.2 wheel is missing the file persistent/_ring.abi3.so. When pulling the sources from GitHub so that these extensions are built locally the file is built and the tests succeed.

[jamadden]

I suspect the problem is that the "build_package" part of GHA isn't installing CFFI. So we get a bunch of distutils warnings about "unknown option cffi_modules" and _ring.so isn't built.

[jamadden]

Indeed, #217 removed the installation of CFFI

[dataflake]

Please take a look at #223, from the GHA output I can see that the wheels do contain the missing library now.

[jamadden]

LGTM