Cargo workspace where more than one crate is an extension module?

[elenakrittik]

I am binding a decoupled ecosystem of rust libraries to Python using PyO3 and maturin. The original rust libraries are in a single workspace/monorepo, but are published to crates.io independently from each other. Since there's a lot of crates to port and keep in sync, i'd like to also use a single repository/cargo workspace for all of my python extension modules.

However, i ran into an issue: maturin seems to be able to build/install only a single extension at a time, and for each it requires a Cargo.toml and a pyproject.toml. This is not very convenient since while the crates are somewhat independent, they are still meant to be used together and as such i want to install them all by running a single maturin develop.

I wonder if anyone already figured a solution for this, or will i have to duplicate manifests and write some wrapper script to run maturin on all of my crates?

[elenakrittik]

If anyone's interested, here's my simple script i currently use:

import sys
import subprocess as sp

PACKAGES = ("cydus-http", "cydus-model")  # List of python extension modules you wish to install

if sys.prefix == sys.base_prefix:
    print("Please run this from a venv.", file=sys.stderr)
    sys.exit(1)

for pkg in PACKAGES:
    if sp.run(["maturin", "develop"], cwd=f"./crates/{pkg}").returncode != 0:
        print("Something went wrong. Aborting.", file=sys.stderr)
        sys.exit(1)
    print()

It assumes python and maturin to be available (which should already be installed if you're building python modules anyway) and the following workspace structure:

Cargo.toml # workspace manifest that has `members = ["crates/*"]` under `[workspace]` in it.
crates/
  cydus-http/
    Cargo.toml # package manifest, the one generated by maturin new should be good
    pyproject.toml # python manifest, same requirements as above
    src/
      ...
  cydus-model/
    ... # same as above
  ... # optionally some non-python-extension crates here

[messense]

IMO this should be solved by a Python package manager like poetry/hatch/pdm and etc instead of building such feature in a PEP 517 build backend like maturin.