Define value range of scalars in doctypes?

[lagru]

This was brought up in scikit-image/scikit-image#7853 (comment).

In scikit-image we frequently document the expected value range for scalars in our docstrings. E.g.

p0 : float in interval [0, 1], optional

@stefanv points out that it might be nice to support this syntax in docstub.

[lagru]

(Pasting from my comment in #7853)

Hmm, with some modification of docstub's grammar this would be totally doable. We'd have to come up with a consistent expectation. E.g.

interval: qualname "in interval" INTERVAL_BRACE number "," number INTERVAL_BRACE

INTERVAL_BRACE: "[" | "]" | "(" | ")"

might work. We could also support "in range". What I don't like is that we have to hardcode the "in interval" string. Otherwise it could be potentially confused with other expressions in the grammar.

There's a cost to doing this. Python's type system doesn't support this (yet, as far as I know). Docstub could either drop it or encode it with typing.Annotated.

Maybe another solution might make this more readable? E.g. the docstub expects the following form

name : annotation, optional, extra_info

Maybe we could allow switching "optional" and "extra_info" around. That way one could write

 p0, p1 : float, in interval [0, 1], optional

Doesn't feel like an immediate priority though. Curious what you think.

[stefanv]

I think the main point being that optional, which doesn't add much useful information (can look at function signature; keywords are optional anyway).

The interval could also be float in [0, 1], which reads a bit better from a math perspective, but may be more confusing to users.

[mkcor]

I think the main point being that optional, which doesn't add much useful information (can look at function signature; keywords are optional anyway).

I agree. I would support moving optional to the end, i.e.,

name : annotation, extra_info, optional

By the way, is 'annotation' used instead of 'type' to avoid confusion with the strict meaning of 'type' in Python?

The sentence preceding the prescribed form refers to the NumPyDoc style and, just looking up numpy.sin, I can see an instance of optional at the end:

out : ndarray, None, or tuple of ndarray and None, optional

Oh, no! I can't (easily) find a direct link to the source code... So maybe it's just rendered with optional at the end? 🤔

The interval could also be float in [0, 1], which reads a bit better from a math perspective, but may be more confusing to users.

@lagru pointed out that it would be confusing to the parser, iiuc: "What I don't like is that we have to hardcode the 'in interval' string. Otherwise it could be potentially confused with other expressions in the grammar."

[lagru]

While the optional expression is currently defined and parsed,

docstub/src/docstub/doctype.lark

Lines 12 to 15 in b1448b5

	// The basic structure of a full docstring annotation as it comes after the
	// `name : `. It includes additional meta information that is optional and
	// currently ignored.
	?annotation_with_meta: type ("," optional)? ("," extra_info)?

its content or presence is not actually used in docstub. E.g.

def foo(a=1, b=1):
    """
    Parameters
    ----------
    a : int
    b : int, optional

will extract the same annotation for a and b. Since docstub is meant to generate stubs and not meant as a linter for the NumPyDoc style, I'm actually fine with that.

So we could sidestep this whole thing by simplifying the expected form to name : annotation, extra_info where extra_info swallows everything after the comma, be it optional or in interval [0, 1].

[lagru]

In reply to #65 (comment)

By the way, is 'annotation' used instead of 'type' to avoid confusion with the strict meaning of 'type' in Python?

Yes, even after having read the typing guide (and glossary) more than once I'm still somewhat fuzzy about the distinction between "type" and "(type) annotation". I think an "annotation" is a synonym for "type" when it is used to annotate something. I'm not married to it.

What I actually care about is differentiating types and their form in docstrings and in Python code. I've been informally using "doctype" to refer to types in docstrings. Maybe I should introduce a glossary to make that clearer.

The sentence preceding the prescribed form refers to the NumPyDoc style and, just looking up numpy.sin, I can see an instance of optional at the end:

I don't think the NumPyDoc style guide says anything about extra information besides "optional" and "default: ...". The implementation also just parses everything after the name : as the "type".

[lagru]

With #70 merged, it's now possible to write

p0 : float, in interval [0, 1], optional

Note the first comma. It's important to differentiate the actual doctype from the optional information that has now grammar restrictions placed on it.

[lagru]

I'll close this as addressed for now. If Python ever adds first class support for something like defining value constraints for int, I'm happy to revisit if docstub can or wants to take advantage in another issue.