Skip to content

✨: add CanArrayX protocols #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ classifiers = [
]
dependencies = [
"typing-extensions>=4.14.1",
"optype>=0.9.3; python_version < '3.11'",
"optype>=0.12.2; python_version >= '3.11'",
Comment on lines +30 to +31
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yikes... Do you think we really need 3.10 support, or could we get away with dropping it? I mean, I could add py310 support back to optype, but I'd really like to avoid that if I can 😅.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering. This wasn't my commit...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lol I wasn't sure what I was thinking when I did that.

But either way, my question still stands.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree we can drop 3.10!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since CI doesn't seem to mind this, we can do that in a follow-up then.

"tomli>=1.2.0 ; python_full_version < '3.11'",
]

[project.urls]
Expand Down Expand Up @@ -121,9 +124,12 @@ ignore = [
"D107", # Missing docstring in __init__
"D203", # 1 blank line required before class docstring
"D213", # Multi-line docstring summary should start at the second line
"D401", # First line of docstring should be in imperative mood
"FBT", # flake8-boolean-trap
"FIX", # flake8-fixme
"ISC001", # Conflicts with formatter
"PLW1641", # Object does not implement `__hash__` method
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"PLW1641", # Object does not implement `__hash__` method

"PYI041", # Use `float` instead of `int | float`
]

[tool.ruff.lint.pylint]
Expand All @@ -137,10 +143,13 @@ allow-dunder-method-names = [
]

[tool.ruff.lint.flake8-import-conventions]
banned-from = ["array_api_typing"]
banned-from = ["array_api_typing", "optype", "optype.numpy", "optype.numpy.compat"]

[tool.ruff.lint.flake8-import-conventions.extend-aliases]
array_api_typing = "xpt"
optype = "op"
"optype.numpy" = "onp"
"optype.numpy.compat" = "npc"

[tool.ruff.lint.isort]
combine-as-imports = true
Expand Down
3 changes: 2 additions & 1 deletion src/array_api_typing/__init__.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
"""Static typing support for the array API standard."""

__all__ = (
"Array",
"HasArrayNamespace",
"__version__",
"__version_tuple__",
)

from ._namespace import HasArrayNamespace
from ._array import Array, HasArrayNamespace
from ._version import version as __version__, version_tuple as __version_tuple__
88 changes: 88 additions & 0 deletions src/array_api_typing/_array.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
__all__ = (
"Array",
"BoolArray",
"HasArrayNamespace",
"NumericArray",
)

from pathlib import Path
from types import ModuleType
from typing import Literal, Never, Protocol, TypeAlias
from typing_extensions import TypeVar

import optype as op

from ._utils import docstring_setter

# Load docstrings from TOML file
try:
import tomllib
except ImportError:
import tomli as tomllib # type: ignore[import-not-found, no-redef]

_docstrings_path = Path(__file__).parent / "_array_docstrings.toml"
with _docstrings_path.open("rb") as f:
_array_docstrings = tomllib.load(f)["docstrings"]
Comment on lines +24 to +25
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit of a mouthful, but it helps with namespace pollution (I'm talking about the f).

Suggested change
with _docstrings_path.open("rb") as f:
_array_docstrings = tomllib.load(f)["docstrings"]
_array_docstrings = tomllib.loads(_docstrings_path.read_text())["docstrings"]

And _array_docstrings will now be inferred as Any, so it could use an annotation (e.g. one wrapped as Final).

Also, since these are constants (the path too), uppercase names seem appropriate.


NS_co = TypeVar("NS_co", covariant=True, default=ModuleType)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about calling this NamespaceT_co instead?

Personally, I've kinda grown attached to having a trailing T in type parameters. That way you can always tell at a glance if something is a type parameter or something else. Especially in case of numpy, where the _co suffix is also used for array-likes with "coercible" dtypes, such as numpy._typing._ArrayLikeInt_co.

But that being said, in this context here, it's probably won't be much of a problem. So it'd mostly be for the sake of consistency I suppose.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me.

T_contra = TypeVar("T_contra", contravariant=True)
R_co = TypeVar("R_co", covariant=True, default=Never)


class HasArrayNamespace(Protocol[NS_co]):
"""Protocol for classes that have an `__array_namespace__` method.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should mention that this is only intended for static typing, not for runtime things. And maybe also that this is intended as purely structural type, not a nominal one (i.e. not intended as base class).


Example:
>>> import array_api_typing as xpt
>>>
>>> class MyArray:
... def __array_namespace__(self):
... return object()
>>>
>>> x = MyArray()
>>> def has_array_namespace(x: xpt.HasArrayNamespace) -> bool:
... return hasattr(x, "__array_namespace__")
>>> has_array_namespace(x)
True

"""

def __array_namespace__(
self, /, *, api_version: Literal["2021.12"] | None = None
) -> NS_co: ...


@docstring_setter(**_array_docstrings)
class Array(
HasArrayNamespace[NS_co],
op.CanPosSelf,
op.CanNegSelf,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

np.bool is messing things up again...

>>> -np.array([True])
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    -np.array([True])
TypeError: The numpy boolean negative, the `-` operator, is not supported, use the `~` operator or the logical_not function instead.

I'm guessing that something like op.CanNegSelf["Array[NS_co]"] could work.

op.CanAddSame[T_contra, R_co],
op.CanSubSame[T_contra, R_co],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's another annoying np.bool one:

>>> np.array([True]) - 1
array([0])
>>> np.array([True]) - True
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    np.array([True]) - True
    ~~~~~~~~~~~~~~~~~^~~~~~
TypeError: numpy boolean subtract, the `-` operator, is not supported, use the bitwise_xor, the `^` operator, or the logical_xor function instead.
>>> np.array([True]) - np.array([True])
Traceback (most recent call last):
  File "<python-input-3>", line 1, in <module>
    np.array([True]) - np.array([True])
    ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
TypeError: numpy boolean subtract, the `-` operator, is not supported, use the bitwise_xor, the `^` operator, or the logical_xor function instead.

The fact that it accepts int but not bool (which subclasses int) is especially annoying (and type unsafe, btw).

So instead, something like op.CanSubSelf[T_contra, "Array[NS_co]"] should do the trick

op.CanMulSame[T_contra, R_co],
op.CanTruedivSame[T_contra, R_co],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one also won't work for integer arrays:

>>> np.array([1]) / np.array([1])
array([1.])

so how about

Suggested change
op.CanTruedivSame[T_contra, R_co],
op.CanTruedivSame[T_contra, "Array[NS_co]"],

or perhaps

Suggested change
op.CanTruedivSame[T_contra, R_co],
op.CanTruedivSame[T_contra, "Array[NS_co, T_contra]"],

op.CanFloordivSame[T_contra, R_co],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

>>> np.array([True]) // np.array([True])
array([1], dtype=int8)

so maybe

Suggested change
op.CanFloordivSame[T_contra, R_co],
op.CanFloordivSame[T_contra, "Array[NS_co, T_contra]"],

or something?

Copy link
Collaborator Author

@nstarman nstarman Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work, using the existing type parameters?

Array[bool, Array[float | int, _, _], _]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that depends on the _ of the inner array

Copy link
Collaborator Author

@nstarman nstarman Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's Self, which should be correct for floating dtype arrays.
It's maybe not 100% correct for integer dtypes... since 1/2 is floating.
Array[bool, Array[Any, _, _], _] is correct but erases information.

op.CanModSame[T_contra, R_co],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

>>> np.array([True]) % np.array([True])
array([0], dtype=int8)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is covered by R_co, right?

Array[bool, Array[float | int, _, _], _]

op.CanPowSame[T_contra, R_co],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

>>> np.array([True]) ** np.array([True])
array([1], dtype=int8)

Copy link
Collaborator Author

@nstarman nstarman Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is covered by R_co, right?

Array[bool, Array[float | int, _, _], _]

Copy link
Member

@jorenham jorenham Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But R_co defaults to Never, in which case it would reject boolean arrays.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean the inner R_co?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant in general

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering the specific case of Array[bool, Array[Any, _, _], _]
This should work for everything, but erases a lot of information on the return type.

x: Array[bool, Array[Any, _, _], _] = np.array([True])
y = x ** x  # Array[Any, _, _] 

Something more specific would be

x: Array[bool, Array[float | int, _, _], _] = np.array([True])
y = x ** x  # Array[Any, _, _] 

Protocol[T_contra, R_co, NS_co],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put NS_co first?

And what is the purpose of R_co here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put NS_co first?

It's the least likely to be used.

And what is the purpose of R_co here?

Return types. e.g BoolArray = Array[DType_co, bool, Array[Any, float, Never, NS_co], NS_co]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the least likely to be used.

The array namespace is where most of the static information lives, so I expect it will be used quite a lot, actually.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I'm also fine with separating the two, i.e. the CanArrayNamespace and Array

Copy link
Collaborator Author

@nstarman nstarman Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expect it will be used quite a lot, actually.

In what way? In #34 we get the DType_co parameter

Array[+DType, -Other = Never, +R = Never, +NS = ModuleType] = Array[DType, Self | Other, Self | R, NS]

So the parameters are dtype, other allowed types for binary ops, the return types, and the array namespace. This will predominantly just be a Module, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In what way?

getting rid of the ND_co typar

This will predominantly just be a Module, right?

It'll be something that we can match against with protocols, so that we can obtain a bunch of juicy static typing details, like for example the return type of their abs() function.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'll be something that we can match against with protocols, so that we can obtain a bunch of juicy static typing details, like for example the return type of their abs() function.

That will be very nice.

):
"""Array API specification for array object attributes and methods."""


BoolArray: TypeAlias = Array[bool, Array[float, Never, NS_co], NS_co]
"""Array API specification for boolean array object attributes and methods.

Specifically, this type alias fills the `T_contra` type variable with
`bool`, allowing for `bool` objects to be added, subtracted, multiplied, etc. to
the array object.

"""

NumericArray: TypeAlias = Array[float | int, NS_co]
"""Array API specification for numeric array object attributes and methods.

Specifically, this type alias fills the `T_contra` type variable with `float
| int`, allowing for `float | int` objects to be added, subtracted, multiplied,
etc. to the array object.

"""
151 changes: 151 additions & 0 deletions src/array_api_typing/_array_docstrings.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
[docstrings]
__pos__ = '''
Evaluates `+self_i` for each element of an array instance.

Returns:
Self: An array containing the evaluated result for each element.
The returned array must have the same data type as `self`.

See Also:
array_api_typing.Positive

'''

__neg__ = '''
Evaluates `-self_i` for each element of an array instance.

Returns:
Self: an array containing the evaluated result for each element in
`self`. The returned array must have a data type determined by Type
Promotion Rules.

See Also:
array_api_typing.Negative

'''

__add__ = '''
Calculates the sum for each element of an array instance with the respective
element of the array `other`.

Args:
other: addend array. Must be compatible with `self` (see
Broadcasting). Should have a numeric data type.

Returns:
Self: an array containing the element-wise sums. The returned array
must have a data type determined by Type Promotion Rules.

See Also:
array_api_typing.Add

'''

__sub__ = '''
Calculates the difference for each element of an array instance with the
respective element of the array other.

The result of `self_i - other_i` must be the same as `self_i +
(-other_i)` and must be governed by the same floating-point rules as
addition (see `CanArrayAdd`).

Args:
other: subtrahend array. Must be compatible with `self` (see
Broadcasting). Should have a numeric data type.

Returns:
Self: an array containing the element-wise differences. The returned
array must have a data type determined by Type Promotion Rules.

See Also:
array_api_typing.Subtract

'''

__mul__ = '''
Calculates the product for each element of an array instance with the
respective element of the array `other`.

Args:
other: multiplicand array. Must be compatible with `self` (see
Broadcasting). Should have a numeric data type.

Returns:
Self: an array containing the element-wise products. The returned
array must have a data type determined by Type Promotion Rules.

See Also:
array_api_typing.Multiply

'''

__truediv__ = '''
Evaluates `self_i / other_i` for each element of an array instance with the
respective element of the array `other`.

Args:
other: Must be compatible with `self` (see Broadcasting). Should have a
numeric data type.

Returns:
Self: an array containing the element-wise results. The returned array
should have a floating-point data type determined by Type Promotion
Rules.

See Also:
array_api_typing.TrueDiv

'''

__floordiv__ = '''
Evaluates `self_i // other_i` for each element of an array instance with the
respective element of the array `other`.

Args:
other: Must be compatible with `self` (see Broadcasting). Should have a
numeric data type.

Returns:
Self: an array containing the element-wise results. The returned array
must have a data type determined by Type Promotion Rules.

See Also:
array_api_typing.FloorDiv

'''

__mod__ = '''
Evaluates `self_i % other_i` for each element of an array instance with the
respective element of the array `other`.

Args:
other: Must be compatible with `self` (see Broadcasting). Should have a
numeric data type.

Returns:
Self: an array containing the element-wise results. Each element-wise
result must have the same sign as the respective element `other_i`.
The returned array must have a floating-point data type determined
by Type Promotion Rules.

See Also:
array_api_typing.Remainder

'''

__pow__ = '''
Calculates an implementation-dependent approximation of exponentiation by
raising each element (the base) of an array instance to the power of
`other_i` (the exponent), where `other_i` is the corresponding element of
the array `other`.

Args:
other: array whose elements correspond to the exponentiation exponent.
Must be compatible with `self` (see Broadcasting). Should have a
numeric data type.

Returns:
Self: an array containing the element-wise results. The returned array
must have a data type determined by Type Promotion Rules.

'''
30 changes: 0 additions & 30 deletions src/array_api_typing/_namespace.py
Original file line number Diff line number Diff line change
@@ -1,30 +0,0 @@
__all__ = ("HasArrayNamespace",)

from types import ModuleType
from typing import Literal, Protocol
from typing_extensions import TypeVar

T_co = TypeVar("T_co", covariant=True, default=ModuleType)


class HasArrayNamespace(Protocol[T_co]):
"""Protocol for classes that have an `__array_namespace__` method.

Example:
>>> import array_api_typing as xpt
>>>
>>> class MyArray:
... def __array_namespace__(self):
... return object()
>>>
>>> x = MyArray()
>>> def has_array_namespace(x: xpt.HasArrayNamespace) -> bool:
... return hasattr(x, "__array_namespace__")
>>> has_array_namespace(x)
True

"""

def __array_namespace__(
self, /, *, api_version: Literal["2021.12"] | None = None
) -> T_co: ...
64 changes: 64 additions & 0 deletions src/array_api_typing/_utils.py
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit worried about whether tools like Pylance will be able to figure out the docstrings like this, given the dynamic nature. Did you check that already?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet. It worked in Jupyter, but that's not static.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it you're a data scientist :P ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Computational astrophysicist. Close enough. 🤷.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity; are you on team "dark matter", or on one of the other ones?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dark matter. Though that doesn't stop me writing papers on the other ones 😆.

Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
"""Utility functions."""

from collections.abc import Callable
from enum import Enum, auto
from typing import Literal, TypeVar

ClassT = TypeVar("ClassT")
DocstringTypes = str | None


class _Sentinel(Enum):
SKIP = auto()


def set_docstrings(
obj: type[ClassT],
main: DocstringTypes | Literal[_Sentinel.SKIP] = _Sentinel.SKIP,
/,
**method_docs: DocstringTypes,
) -> type[ClassT]:
"""Set the docstring for a class and its methods.

Args:
obj: The class to set the docstring for.
main: The main docstring for the class. If not provided, the
class docstring will not be modified.
method_docs: A mapping of method names to their docstrings. If a method
is not provided, its docstring will not be modified.

Returns:
The class with updated docstrings.

"""
if main is not _Sentinel.SKIP:
obj.__doc__ = main

for name, doc in method_docs.items():
method = getattr(obj, name)
method.__doc__ = doc
return obj


def docstring_setter(
main: DocstringTypes | Literal[_Sentinel.SKIP] = _Sentinel.SKIP,
/,
**method_docs: DocstringTypes,
) -> Callable[[type[ClassT]], type[ClassT]]:
"""Decorator to set docstrings for a class and its methods.

Args:
main: The main docstring for the class. If not provided, the
class docstring will not be modified.
method_docs: A mapping of method names to their docstrings. If a method
is not provided, its docstring will not be modified.

Returns:
A decorator that sets the docstrings for the class and its methods.

"""

def decorator(cls: type[ClassT]) -> type[ClassT]:
return set_docstrings(cls, main, **method_docs)

return decorator
Loading
Loading