Making it easier for DTS consumers

[avivkeller]

This was discussed in today's Web Team Meeting.

Supersedes: #437

---

We'd like to make our AST, and possibly a generator, to make it easy to consume Node.js docs by the DTS maintainers. We want them to have a good DX.

As discussed in today's meeting, this means finding and resolving quirks that require these maintainers to manually scour our documentation, which, on it's own, has many fallacies.

[Renegade334]

Some musings on the state of play, as discussed. Apologies for the slight tardiness.

I've deliberately omitted things that cause us obstacles, but which aren't really fixable within the documentation system that exists. Hopefully everything here is either actionable, or at the very least food for thought.

Objects

• A significant proportion of the documentation references bare {Object} types, with no elaboration. In rare occasions, the API in question is truly agnostic as to the object being accepted or returned. In most of these cases, however, the shape of the output or expected input is known but is described verbally, which isn't useful from a typing perspective.
  • This is probably partially down to the documentation not having any way of expressing JSDoc-style "typedefs" to describe anonymous objects with known shapes, with the only way of documenting the type of such objects being inline, which can impact on human readability.
• Inheritance relationships:
  • Verbal inheritance relationships: this applies primarily to objects. Many options interfaces, for example, have description blocks that state "Also accepts any options from...", but this isn't expressed in the type declaration, and there's no canonical way to express an interface-style inheritance relationship for object definitions.
  • There are also examples of classes that express inheritance without using the canonical extends clause (example).
• There is no way to indicate property optionality in the documentation, beyond verbal description. This is a massive pain for us, as property definedness is something very observable in TypeScript environments, particularly with the exactOptionalPropertyTypes compiler option enabled.
  • In an input context (eg. options objects), there's no indication of which properties are mandatory and which are optional, other than verbally.
  • In an output context (eg. a class instance property), there's no indication of which properties are conditionally present. In addition, there's an observable difference between properties that are conditionally present (?: T) and those that are unconditionally present but might be undefined (: T | undefined) in TypeScript when exactOptionalPropertyTypes is enabled, and there is absolutely no way to express this in the docs at present.
• Readonly properties: again, no way to indicate this except for verbally.
• Function properties: it's extremely rare to see {Function} qualified with its parameter and return types other than verbally, and it's unclear whether there is a canonical convention for this.

Buffers

• The vast majority of Buffer-accepting APIs will accept any ArrayBufferView, as this is the behaviour of validateBuffer(), but it's extremely variable as to which APIs are documented as such.
• A lot of instances of {ArrayBuffer} actually refer to "any array buffer" (ie. {ArrayBuffer} | {SharedArrayBuffer}), but this is again variably documented.
• These are more issues with the source material than with the tooling.

Promises

• All promises are typed as {Promise}, with a verbal description of what the Promise actually resolves to. This isn't useful for us.
  • The doc tooling would need to support some notion of generic types (ie. Promise<...>) to unlock improvements here.

Arrays

• Again, quite a few examples of {Array}, which aren't useful.
• There's no way of expressing an array of a union of element, as opposed to a union of arrays of a specific element. For example, input that accepts (string | URL)[] is expressed as string[] | URL[], which isn't accurate.
  • Again, this would probably require some sort of support for generics (eg. Array<string | URL>) in order to be improved upon.

I know I've said it a lot, but I feel like it's worth reiterating: the greatest hurdle for DT is the robustness and accuracy of the source material, not the tooling itself. However, that's not to say that QOL improvements wouldn't be well-received.

[avivkeller]

Thank you! We have JSConf and the Collaborator Summit next week, so I can definitely get some input from the core collaborators!

[Renegade334]

The existential question at the heart of all of this is what the documentation is intended to be. Is it a somewhat relaxed, best-effort human guidebook, or is it intended to be a robust, structured resource that's machine-consumable? At the moment, it seems to be a hybrid of both, which is where a lot of the friction lies.

I suppose a related question is whether the project and associated documentation have grown to the extent that it's worth considering documenting the API in a structured language, rather than a set of glorified READMEs... 😉

[flakey5]

Thank you @Renegade334!

I'm hoping that with the new JSON representation for the docs (re #214, #287) solves some of the QOL issues you mentioned, but regardless there's still some outstanding.

Iirc, one of the next steps once all the new tooling is in Node core is to review the actual structure of the docs and see what changes would be good to make, which hopefully also means resolving even more of the issues you mentioned

[avivkeller]

(I'm marking this "Blocked" until we integrate the new tooling into core, since, once that's done, we have a lot more control over that content)