ExperimentalAPI.jl

public says who may call a name. Nothing says whether the name is finished.

Julia already checks half of that. Docs.undocumented_names has been public API in Base since 1.11, and Aqua.test_undocumented_names ships it as a test: every public name must carry a docstring. What neither can express is the third option —

this name is public, it has no docstring, and that is deliberate: the shape is not settled, and here is why.

ExperimentalAPI adds that option at the definition site, and makes it something a tool reads rather than prose a human might happen to notice.

using ExperimentalAPI

@experimental "reads Test's internal result tree; not dogfooded in CI yet" \
function render_test_report(records)
    # ...
end

julia> ExperimentalAPI.audit(MyPackage)
Public surface of MyPackage — 46 names
  documented      45
  experimental     1
  unaccounted      0

The mark costs nothing at run time: @experimental emits your definition unchanged plus one push! at load time. Calls are not wrapped.

The check is the point

A marker nobody compares against anything is a claim. Put this in runtests.jl and it becomes a contract:

using MyPackage, ExperimentalAPI, Test

ExperimentalAPI.test_surface(MyPackage)

It fails, naming the symbol, when a public name has neither a docstring nor a mark — and also when a mark points at a name that was never made public, which is the module contradicting itself.

Adopting it on a package that already has a backlog:

ExperimentalAPI.test_surface(MyPackage; skip = [:legacy_one, :legacy_two])

A stale skip entry fails. A name that has since been documented, declared, or deleted is reported, so the list can only shrink.

Install

Not in the General registry yet — install by URL:

pkg> add https://github.com/QAtlasHub/ExperimentalAPI.jl

It is loaded by the package being marked, so it is a normal dependency — but it pulls in nothing beyond TOML, and Test only through a package extension.

[deps]
ExperimentalAPI = "fd2d14cb-3a46-42a9-afd8-e8499236f05e"

Declaring

Attached to a definition, or as a list of names defined elsewhere:

# the definition site
@experimental "signature will be wrapped once the write-back refactor settles" \
function ingest(config; doc, kwargs...)
    # ...
end

# names an included file defines
@experimental "reads Test's internal result tree; not dogfooded in CI" \
    render_test_report dump_test_report load_test_dump

# with the issue where the shape is being decided
@experimental("export format is a guess until someone consumes it",
    since = v"0.4.0", tracking = "https://github.com/org/Pkg.jl/issues/12",
    registry_entry(x) = x)

A docstring and a mark are not exclusive — a name can be documented and declared unfinished, and the two accounts answer different questions.

Querying

experimental(MyPackage)              # Vector{Mark}, sorted — the reason travels with the name
isexperimental(MyPackage, :foo)      # the one-bit form
stable(MyPackage)                    # the complement: what you cannot change quietly
audit(MyPackage)                     # the check, as data

Release decisions

Write the covenant down at each release, and compare the next one against it:

write_snapshot("api.toml", MyPackage)          # at release time, committed

d = compare(read_snapshot("api.toml"), MyPackage)
isbreaking(d) && error("breaking: $(d.removed_stable) removed, $(d.demoted) demoted")

Removing a name you declared experimental lands in removed_experimental and is not breaking. That is the whole contract: the mark was the notice, given in the source, at the definition, before the removal. Demoting a settled name to experimental is breaking — a promise withdrawn is a change to what callers were told.

Names, not signatures. compare reads name sets. A name present in both snapshots whose arguments changed is a breaking change it cannot see. Read the diff as a floor on breakage, never as a clearance.

Why this cannot be a feature of Aqua

A mark is written in src/, on the line above the definition, so the package being marked has to depend on whatever provides @experimental at run time. Aqua is a test-only dependency. It can own the check; it structurally cannot own the declaration.

The check here is also not the same set difference. Docs.undocumented_names reports every public name without a docstring — including names re-exported from a dependency, whose prose is somebody else's job:

julia> names(Down)                      # `up` and `undoc_up` come from a dependency
4-element Vector{Symbol}:
 :Down, :own_undoc, :undoc_up, :up

julia> Docs.undocumented_names(Down)    # the dependency's gap, reported as yours
3-element Vector{Symbol}:
 :Down, :own_undoc, :undoc_up

julia> audit(Down).unaccounted           # only what this module actually owns
1-element Vector{Symbol}:
 :own_undoc

audit separates those as foreign, and adds dangling — a mark on a name that was never made public, which is the module contradicting itself and needs no reference to be wrong.

What this is not

axis	already solved by	this package
who may call a name	export, public (1.11)	orthogonal — a name can be public and unfinished
a name on its way out	@deprecate	opposite direction
type stability	DispatchDoctor	unrelated
every public name has a docstring	Docs.undocumented_names (Base 1.11+), Aqua.test_undocumented_names	the same check, plus a third answer
generating documentation	Documenter	only ever checks whether prose exists
run-time behaviour	—	calls are untouched

What the audit cannot see

Stated up front rather than in a footnote, because a check whose blind spots are undocumented reads as if it had none:

• Signatures. A public name whose arguments change under it is invisible.
• Prose quality. A docstring reading TODO counts as documented.
• Methods on other packages' functions. They are not in names(m).
• Names public only inside an extension, which is a separate module.
• Whether a name appears in your guide, README or docs site. Docstring presence is not documentation-page presence, and those two gaps are usually different sets.

License

MIT