De-duplicate comments shared by multiple definitions

[dduugg]

Motivation

Fixes #2702.

tapioca gem repeats a constant's comments once per definition: selenium-webdriver@4.47.0 emits the Apache/SFC license header 145 times in front of module Selenium; end, 2,336 of that file's 24,508 lines. documentation_comments concatenates the comments of every definition, and the existing comments.uniq! cannot collapse them. Rubydex::Comment exposes only string and location, so ==/hash come from BasicObject/Kernel and uniq! compares object identity.

Implementation

Render and filter the comments per definition, then de-duplicate those blocks before flattening. Per block rather than per line keeps repeated lines within a block (the bare # separators in a header); after filtering rather than before means headers differing only by an ignored comment (# frozen_string_literal: true) still collapse. The dropped return [] if comments.empty? was redundant: no definitions, or all comments filtered out, both fall through to [].

Only byte-identical blocks collapse, so a header embedding a per-file copyright line still repeats (ffi goes from 14x to 12x, kramdown from 22x to 3x), unchanged from main. 2deb3fa also dropped yard_doc.rb's return [] if /(copyright|license)/i.match?(docstring), which is why preambles are documented at all now; restoring it would clear most of the remainder but also drops real docs mentioning "license", so I left it for a follow-up.

Tests

Added one spec: a module reopened in three files (two sharing a header, one distinct), a singleton method defined twice with the same doc, and a constant defined twice with the same doc. It fails on main with all three doubled, and fails against a de-duplicate-before-filter variant, so it pins the ordering too.

Also generated each of these twice, unpatched main worktree vs. this branch, with versions pinned by lockfile reuse:

Gem	before	after	removed
elasticsearch-api@9.5.0	31,131	19,955	11,176 (35.9%)
selenium-webdriver@4.47.0	24,508	22,172	2,336 (9.5%)
aws-sdk-core@3.254.1	24,386	24,272	114 (0.5%)
opentelemetry-api@1.11.0	1,904	1,829	75 (3.9%)
googleauth@1.17.3	4,086	4,043	43 (1.1%)
signet@0.22.0	44	31	13 (29.5%)
activesupport@8.1.3.1	20,357	20,349	8
jwt@3.2.0	2,583	2,579	4
faraday@2.14.3	3,059	3,059	0
rack@3.2.7	4,633	4,633	0
Total	116,691	102,922	13,769 (11.8%)

For all ten, non-comment lines are byte-identical, the set of distinct comment lines is identical (no documentation lost), and "after" is an exact line-subsequence of "before". --no-doc output is SHA-256-identical, and a sweep over 124,328 declarations in 138 installed gems found no case where output grew or lost a distinct doc line.

[vinistock]

Thank you for the contribution!