Skip to content

epic: package work for the first two deployed-editor go-lives - #235

Merged
jpslav merged 38 commits into
mainfrom
integration-202608-b
Aug 15, 2026
Merged

epic: package work for the first two deployed-editor go-lives#235
jpslav merged 38 commits into
mainfrom
integration-202608-b

Conversation

@jpslav

@jpslav jpslav commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Standing integration branch for a go-live epic: getting CanopyCMS to a deployed
editor for the first two real adopter sites — a documentation/knowledge-base site
first, then a marketing site.

Kept in draft. This body is the running ledger — it is rewritten on every merge, so it
always answers "what is in this branch right now" without reading the commit log.

Why this epic

Both adopter sites run Canopy in mode: 'dev' locally and neither wires
CanopyCmsService. The thing that has never happened is a real site running the
deployed editor
. This branch closes the package-side half of that gap.

Two independent audits of those sites found they had converged on the same missing
capabilities — separately building search indexing, sitemap/SEO metadata, markdown
rendering, heading-annotation and table-of-contents handling, entry-filename parsing,
and trailing-slash URL normalization for static exports. Both repos had independently
created a file with the same name to solve the heading problem. That convergence is the
ranking signal for what belongs in the package: when two teams with no contact build
the same thing, the package is missing it.

Everything here is generally applicable — none of it is specific to those two sites.

Landed

All package work for this epic is merged. Full suite green on the merged result:
4064 tests passing, typecheck / lint / client-bundle-boundary / backlog-integrity
all clean.

Change Notes
docs/adopter-migration.md The adopter-facing record. Every entry names the pattern of code it supersedes, never a specific adopter's files
Backlog re-baseline Corrected an inverted premise in the task index; filed task files for adopter requests and audit findings that had none
required: false infers an optional property Breaking, type-level. Was a required T | undefined, now field?: T. Only an explicit required: false is affected — omitting required still infers required, pinned by tests in both directions
parseTypedFilename + defaultBuildPath exported Both existed internally and were merely unreachable. Four hand-rolled copies of the filename parser were found across two audited adopter repos, disagreeing on segment count and case-folding
read/readByUrlPath return meta.entryType + meta.entryId Already resolved internally; this is plumbing. Enables routing on entry type instead of re-parsing a path
listEntries carries updatedAt The stat already happened and was discarded. Documented as filesystem mtime, so a fresh CI clone reports checkout time
Sitemap + SEO metadata helpers Enumerates every routable entry type by default, so omission requires explicit opt-out. Ships as one change because noindex is a single predicate feeding both the page's robots directive and sitemap exclusion
Block-registry types, field fragments, shared-block recipe Exhaustive block→component mapped type instead of a renderer component; defineFieldFragment; documented shared-block recipe with its caveat
Build-context factory, resolveEntryTitle, toPlainText The three primitives genuinely duplicated across adopters

Two things reviewers should look at

The one breaking change is the optional-property inference. The break is narrow and
one-directional: reading, literal construction, keyof, in, and spreads are all
unaffected — only assignment to a hand-written interface declaring the key as
required-with-undefined, Required<T>, and exactOptionalPropertyTypes projects.
Measured in-repo blast radius was 2 lines.

meta.entryType is inferred, not read, for legacy files that predate embedded-ID
filenames: it reports the collection's default entry type, which may not be what the
file is. meta.entryId === undefined is the exact tell. Documented rather than changed,
since adopters are being encouraged to branch routing on that field.

Deliberately not built

  • A bundled markdown renderer. Would add three runtime dependencies and, to dodge
    the react-markdown-in-RSC trap, would have to be a client component — forcing every
    adopter's markdown fields into the client bundle. It could not cover the MDX half
    anyway, and the divergence that prompted the request (whether external links open in a
    new tab) is site policy, not CMS policy.
  • A generic search-document extractor. Comparing two real implementations, the
    derivation logic shares nothing but "walk strings, skip structural keys". A helper
    guessing which keys are prose would silently drop content from a search index — the
    same silent-divergence failure the request was raised about, relocated somewhere
    adopters cannot see it. The genuinely duplicated plumbing shipped instead.
  • A renderBlocks() component. It would have to fix a key strategy, an
    unknown-template policy and prop threading; any of those choices makes it unusable for
    someone. The type plus a documented loop is the better trade.

Decisions taken during this epic

Two were settled deliberately rather than left to the code, and they reinforce
each other:

Publish state is branch-only. No per-entry draft/published field. Merged to the
base branch ⇒ public; unmerged ⇒ not public; there is no third state. noindex means
public-but-unadvertised, not hidden. This is what the architecture always implied — the
gap was that nothing stated the negative half, so an adopter invented a draft
convention that lived in its docs and not in its code, and silently rotted. No
enumeration helper may invent a publish filter; the helper that shipped here is named
collectRoutableEntries for exactly that reason.

Path ACLs are enforced on the request-scoped listing, rather than deferring by
restricting listing to build time. This is what makes the first decision load-bearing:
with publish state branch-only there is no draft flag to fall back on, so on an unmerged
branch every entry is unpublished by definition and a request-time reader would see all
of it. That makes the exposure sharper, not weaker — which is why filtering won over the
cheaper deferral.

defaultBranchAccess: 'deny' is now genuinely usable, and is what canopycms init
scaffolds.
Previously the generator wrote 'allow' while the config schema defaulted
to fail-closed 'deny', so every scaffolded project opted out of secure-by-default. The
reason nobody had simply flipped it was that 'deny' did not work: the client granted a
branch's creator unconditionally while the server required an ACL match first, so under
'deny' the creator of a branch saw an enabled Submit and got a 403. Both halves are
fixed together — flipping the default without closing that divergence would have turned
a latent bug into a day-one one.

Review

Three rounds, all independent of the authors: one broad pass, four parallel specialists
(authorization, types, static generation, docs), then a final broad pass judging the
result as a whole. Every finding is fixed and merged. Rounds two and three each found a
defect introduced by the previous round's fixes, which is the case for having run them in
sequence.

The findings worth knowing about, because each was a silent failure rather than a broken
one:

  • A content file whose name did not parse was skipped through a debug-gated log, so a
    page could vanish from both the build and the sitemap with no output at all. Now a
    red build — and then narrowed, because the first fix also red-built a plain README and
    the sibling-artifact layout the README itself tells adopters to commit.
  • defineSeoFieldGroup({ group }) typed its wrapper key as required while the runtime
    treated it as optional, so data.seo.metaTitle type-checked and crashed on real
    content omitting the key.
  • The block-registry recipe crashed on data at rest: compile-time exhaustiveness covers
    the schema, not content carrying a renamed template. Confirmed as an HTTP 500 under
    next dev.
  • generateContentSitemap accepted a relative siteUrl, producing a sitemap search
    engines silently reject.
  • The noindex predicate had two independently-configured call sites and could disagree
    with itself; the field location is now set once on the context and bound into both.

CodeQL then caught one thing all three rounds missed: a polynomial-ReDoS regex in
siteUrl normalization, introduced with the sitemap helper. Measured at 2377ms versus
0.013ms on a 40k-slash input; replaced with a linear scan and pinned by a regression
test. Worth noting that static analysis found what three careful readings did not.

A normal merge commit is fine. An earlier draft of this body asked for a squash, on
the grounds that the branch's history carried detail that should not reach main. That
is no longer true — the branch history was rewritten and every commit, message and tree,
was verified clean. Squashing would now only cost the per-change commits, which are worth
keeping for git bisect and for reading the epic later.

Review guidance

Every change carries an entry in docs/adopter-migration.md naming what an adopter must
do and what local code the change makes deletable. That column is the deliverable: an
upgrade that adds a new API without removing what it supersedes leaves two
implementations to drift apart, which is the failure mode most of these changes exist to
end.

Each implementation branch was reviewed cold by an independent reviewer that did not
write it and re-ran the suite itself. That review confirmed the branches via exact test
arithmetic and a reverse-red proof, and found two real defects (a filename parser
accepting dotfiles, and the inferred-entryType gap above), both since fixed.

jpslav added 13 commits August 14, 2026 13:04
The two adopter sites are the consumers of everything this epic builds, so
each change records what an adopter must do AND what local code the change
makes deletable. Written as work lands rather than reconstructed at the end,
because the Phase 5 upgrade chips execute against this file.

The deletable half is the point: an upgrade that adds the new API without
removing the code it supersedes leaves two implementations to drift apart,
which is the failure mode most of these changes exist to end.
TypeFromEntrySchema emitted every field as a required property, adding
`| undefined` to the value type for `required: false` fields. Constructing
any schema-typed literal therefore meant spelling out every unset field as
`undefined`, and adding an optional field to a schema broke every existing
hand-written literal.

InferContentShape is now two key-remapped mapped types intersected — one
emitting required keys, one emitting optional keys via `?:` — collapsed by
a Simplify pass so the result is a single object type. RequiredValue is
gone; the `?` modifier carries optionality instead of duplicating it as
`| undefined`.

Only an EXPLICIT `required: false` becomes optional. A field that omits
`required` infers `boolean | undefined`, which does not extend `false`, so
it stays required — unchanged, and now pinned by a test covering all three
cases (true / false / absent) plus nested objects and block templates.

Type-only change: reading, `keyof`, `in`, spreads and Object.entries are
unaffected, and literal construction is strictly more permissive.

Also reconciles the docs, two of which already promised this behavior:
README's Type Inference section said `required: false` adds `| undefined`
(now a full Optional Fields section), while README's block example and the
defineBlockTemplate JSDoc already advertised `?:` and are now true.
…ning, file adopter-request gaps

The "hub premise is inverted" warning in index.md was itself wrong: both
the documentation site and the marketing site get a deployed editor, with the KB going
first, exactly as the hub's original E->F sequencing assumed. Retract the
warning, correct the preamble and sibling-repo versions (main is 0.0.62),
and re-sweep [MKT]-only tags to [BOTH]/[KB] wherever the KB deploying first
now makes them its blocker too.

File one task per untracked adopter-request gap (#11-#18) and per site-audit
finding with no backlog entry (draft/publish lifecycle, TOC/heading-ID
contract, content validation gate, resolved-reference URLs, trailing-slash
helpers, defaultBuildPath export, content-authoring API + ID generator,
script-runner entrypoint), recording each triage verdict so it isn't
re-derived. Cross-reference existing files (readbyurlpath-entry-type,
static-export-sitemap, authorization-enforcement-consolidation) with what
this epic is implementing now. Append a program-log.md entry recording the
corrected direction and the epic scope (integration-202608-b, PR #235).

pnpm lint:tasks and prettier --check both pass.
… updatedAt

Four adopter-facing primitives the package already computed internally but
never surfaced, each replacing hand-rolled duplicate logic in the two
adopter sites:

- Export parseTypedFilename from canopycms/server (entryTypes now optional,
  so callers without a schema on hand can still parse {type}.{slug}.{id}.{ext})
- Export defaultBuildPath from canopycms/server so buildContentTree's
  buildPath option can be extended instead of reimplemented
- read()/readByUrlPath() now return meta.entryType and meta.entryId,
  already resolved during path resolution, per
  .claude/future-tasks/readbyurlpath-entry-type.md
- listEntries() now carries updatedAt (mtime) through from
  listCollectionEntries, which already stat'd every entry

See docs/adopter-migration.md for adoption steps and what becomes
deletable at each site.
… updatedAt

Resolves the adopter-migration.md conflict by keeping both entry sets, and
rewrites every 'Now deletable' section to describe the PATTERN of superseded
code rather than naming a specific adopter's files.

canopycms is public; its adopters' repos generally are not. The entry template
previously invited per-site file naming, which guaranteed the leak would recur,
so the template itself now says to describe the shape and explicitly not to name
files, paths, branches or hosts. Stating findings as patterns is also simply
more useful to a third-party adopter, who cannot act on someone else's paths.
…log hygiene

- parseTypedFilename() rejected empty-string entry types when entryTypes was
  omitted (e.g. dotfiles/backup files parsed to type: ''). Now rejects any
  leading-dot filename outright, matching content-id-index.ts's existing
  extractEntryTypeFromFilename guard. entryTypes-supplied behavior unchanged.
- Extracted a shared ContentReadMeta type (content-reader.ts) used by
  ContentReader['read'] and CanopyBuildContext's read/readByUrlPath
  (context.ts), replacing three hand-written copies of the same shape.
  Documented, with two new regression probes, that entryType silently falls
  back to the collection's default entry type for legacy files (entryId
  undefined is the signal) and is read from the filename rather than
  re-validated against the current schema. Removed a dead `?? 'entry'`
  fallback in content-reader.ts now that buildPaths()'s entryTypeName is
  typed as always-populated.
- docs/adopter-migration.md: added the missing fallback caveat to the #3
  entry and fixed its now-stale link into resolved/.
- Moved two already-implemented future-tasks entries (readbyurlpath-entry-type,
  optional-property-inference) to resolved/, updating index.md and inbound
  links per the backlog rule.
- Filed entry-schema-eopt-reference-field-compile-error.md: verified
  entry-schema.ts's reference-field branch fails to compile under
  exactOptionalPropertyTypes (pre-existing, not a regression).
- parseTypedFilename rejects leading-dot filenames when entryTypes is omitted
- document that entryType is inferred (not read) when entryId is undefined
- extract a shared ContentReadMeta type; drop a dead fallback
- move two implemented task files to resolved/; file the eOPT compile gap
…w actually fires

Filed as a latent hazard; observed for real on 2026-08-14 during a full test
run, on a developer machine with no EFS and no cross-host contention. Two
consequences the fix should carry: it crashes the process separately from the
pass/fail tally, so a pipeline reading only pass counts calls a crashed run
green; and firing this readily locally means the hold-duration argument
understates production exposure rather than overstating it.
Both static-export surfaces the package previously disclaimed as 'coming
separately', shipped as one change because the noindex flag has to suppress a
page in BOTH of them.

Core (canopycms/server):
- collectRoutableEntries: collectStaticPaths' enumeration with each entry's
  data and updatedAt carried through. Both delegate to one internal pass, so
  the build-time schema-validity guard cannot apply to one and not the other.
- static/seo.ts: extractSeoFields (empty-string-is-unset, because optional
  fields are stored present-but-empty), isNoindexEntry, resolveSeoUrl /
  withTrailingSlash / isAbsoluteUrl.

Schema (canopycms): defineSeoFieldGroup() emits the seven recommended fields,
all optional; flat by default, nested via { group }.

Next adapter (canopycms-next): generateContentSitemap and entryToMetadata,
both bound onto NextCanopyContextResult so route modules never import the
admin build context.

Three decisions worth naming:
- Every routable entry type is enumerated by default. A sitemap built from a
  hand-maintained list of entry types omits whichever type nobody remembered
  to add, ships green, and takes those pages out of search results silently.
  Omission now needs an explicit exclude predicate or a noindex flag.
- trailingSlash is an explicit option. Canopy cannot read next.config, and a
  mismatch advertises URLs that redirect.
- noindex is ONE predicate feeding robots and sitemap exclusion; derived
  separately, the two surfaces drift. Enumeration deliberately does not
  filter on it -- noindex means 'do not advertise', not 'do not build'.

lastModified is pluggable and defaults to updatedAt, documented honestly as
filesystem mtime (a fresh CI clone reports checkout time). robots.txt is out
of scope. Sitemap generation is a new place a schema-invalid entry can turn a
build red, including in an app with no generateStaticParams.

Reference usage in apps/example1 (sitemap.ts + generateMetadata on the post
route); adopter-migration entry, README section, backlog tasks resolved.
Enumerates every routable entry type by default, so omitting one requires an
explicit opt-out rather than an act of memory. Ships with the SEO helper
because noindex is a single predicate feeding both surfaces -- the page's
robots directive and exclusion from the sitemap.
…hared-block docs

Adopter requests #13, #15, #16 from the go-live backlog re-baseline.

- entry-schema.ts: add BlockValueOf<Blocks, N> and BlockComponentRegistry<Blocks,
  ExtraProps> — a mapped type keyed off a block field's discriminated union that
  requires exactly one component per template, making a block-to-component
  registry exhaustive at compile time. Ship the types, not a renderBlocks()
  component: the adopter who requested this already arrived at the mapped-type
  solution independently, and a runtime helper would have to pick a key
  strategy, an unknown-template policy, and prop-threading shape that isn't
  right for everyone. Type-level tests prove exhaustiveness in both directions
  (missing template, unknown extra key both fail to compile).
- entry-schema.ts: add defineFieldFragment(), a 3-line const-inference identity
  helper beside defineBlockTemplate, for discoverability of the
  already-working field-array-spread pattern.
- README: document Block Component Registries, Reusable Field Fragments (both
  composition mechanisms plus a per-use-override example), and Shared /
  Referenced Blocks (recipe + a prominent listEntries-never-resolves-references
  caveat, called out in two places).
- apps/example1: refactor PostView.tsx from an if-chain ending in "Unknown
  block" to a BlockComponentRegistry; wire one real shared/referenced block
  (snippet entry type + sharedCta block template) into schemas.ts and content.
- docs/adopter-migration.md: add the three Unreleased entries.
- Move block-registry-types.md, field-fragments-docs.md and
  shared-blocks-listentries-caveat.md to .claude/future-tasks/resolved/,
  updating index.md and inbound links.

Verified: pnpm lint, lint:bundle, lint:tasks, typecheck, and the full test
suite (3910 tests across 5 packages) all pass.
Comment thread packages/canopycms/src/static/seo.ts Fixed
jpslav added 3 commits August 14, 2026 14:53
…uest #17)

Ships the three primitives that answer the search-document-extraction ask
without building the requested extractSearchDocuments API (the two real
derivations share nothing at that level; see docs/adopter-migration.md for
the full reasoning):

- createBuildCanopy(config, options) — canopycms/server: one-call
  build/admin context factory for standalone scripts outside Next.js,
  mirroring createNextCanopyContext(...).getCanopyForBuild()'s boot
  sequence without the Next.js pieces.
- resolveEntryTitle — now exported from canopycms/server and the root
  canopycms entry (verified client-safe: type-only dependency).
- toPlainText(markdown) — canopycms/ai: MDX/Markdown-to-plaintext,
  built on strip-mdx.ts's stripMdxImports as one pipeline step. Fixes the
  live bug in hand-rolled versions: a paired custom component keeps its
  inner text, losing only its tags.

Resolves .claude/future-tasks/search-document-extraction-primitives.md in
full (its other two items, parseTypedFilename/updatedAt, had already
shipped) — moved to resolved/.
Resolves additive conflicts in entry-schema.ts and its test against the
sitemap work, which landed defineSeoFieldGroup in the same regions; both
export sets are kept. The react import is type-only, so the client-bundle
boundary check still passes with entry-schema.ts reachable from the client
entry.
Resolves backlog cross-link conflicts: both branches moved a task file to
resolved/ while updating its link to the other on the assumption the other
was still open. Both now live in resolved/, so the links are siblings.

Also drops two stale open rows that a keep-both merge left alongside their
own resolved rows -- caught by lint:tasks, which is exactly the drift that
guard exists to catch.
jpslav added a commit that referenced this pull request Aug 15, 2026
…ning, file adopter-request gaps

The "hub premise is inverted" warning in index.md was itself wrong: both
docs-site-proto and website v2 get a deployed editor, with the KB going
first, exactly as the hub's original E->F sequencing assumed. Retract the
warning, correct the preamble and sibling-repo versions (main is 0.0.62),
and re-sweep [MKT]-only tags to [BOTH]/[KB] wherever the KB deploying first
now makes them its blocker too.

File one task per untracked adopter-request gap (#11-#18) and per site-audit
finding with no backlog entry (draft/publish lifecycle, TOC/heading-ID
contract, content validation gate, resolved-reference URLs, trailing-slash
helpers, defaultBuildPath export, content-authoring API + ID generator,
script-runner entrypoint), recording each triage verdict so it isn't
re-derived. Cross-reference existing files (readbyurlpath-entry-type,
static-export-sitemap, authorization-enforcement-consolidation) with what
this epic is implementing now. Append a program-log.md entry recording the
corrected direction and the epic scope (integration-202608-b, PR #235).

pnpm lint:tasks and prettier --check both pass.
@jpslav
jpslav force-pushed the integration-202608-b branch from fc7518e to 3b975eb Compare August 15, 2026 00:48
jpslav added 11 commits August 14, 2026 18:53
…vention

Rebased onto the rewritten integration branch. Three conflicts resolved:

- draft-publish-lifecycle.md: the decision supersedes the description of the
  open gap it replaces. Carried forward the one fact the chip could not have
  had, since the sitemap/SEO helpers landed after it branched -- they comply,
  filtering on noindex plus an explicit exclude predicate and reading no
  publish field.
- Corrected the decision's own API reference: it named collectPublishedEntries,
  which never shipped. What shipped is collectRoutableEntries, and that rename
  is this decision expressed in code -- 'routable' is what the helper answers,
  'published' is the question this decision says the package does not ask.
- resolved-references-url.md: kept both sides; they edited adjacent bullets.

Link paths repaired via lint:tasks --fix for two task files that moved to
resolved/ during this epic.
docs: publish state is branch-only — kill the phantom draft convention
…tentTree

`CanopyContext extends CanopyBuildContext`, so `getCanopy().listEntries()` and
`getCanopy().buildContentTree()` were already reachable at request time — and
neither took a user or checked path permissions. On a mode:'prod' +
deployedAs:'server' deployment that meant the context documented as
"request-scoped, ACL-enforcing" returned full entry `data` for paths the same
user could not have fetched through `read()`. `guardBuildContext` only ever
guarded the *build* context, so nothing was holding this closed.

Both methods now filter through a memoized predicate built from
`services.createContentAccessChecker` — the existing batch primitive that
api/entries.ts already uses, which hoists branch access, the permissions root
and the rule load out of the loop and returns a synchronous per-path check. So
this adds no new ACL matcher (it routes through authorization/path.ts, matcher
server-side, and costs an admin short-circuit or one minimatch per rule with no
extra I/O.

The predicate is applied at each `listCollectionEntries` call site, so denied
entries never reach `extract` — including as the `meta.indexEntry` handed to a
collection's extract callback, which emits no node of its own and is the leak a
listEntries-only fix would have missed. It is skipped entirely at build time and
on static deployments: those run as the synthetic admin, and building the
checker there would add a getSettingsBranchRoot() round trip (EFS in prod) to
every build-time listing.

Also adds the phase-selecting `listEntries` to NextCanopyContextResult, which is
what adopter request #11 asked for: one filesystem pass returning urlPath + data
+ schema, replacing "enumerate paths then read each" (an N+1 whose hand-built
URLs silently miss on multi-segment slugs). No new package entrypoint —
ListEntriesOptions.filter already covers the entryType/noindex cases.

Verified end-to-end in example1: as a real non-admin dev user, request-scoped
listEntries returned 14 entries before this change and 0 after, against a build
context that still returns all 14.

Backlog: resolves listentries-acl-awareness.md (decision recorded, with three
corrections to its original analysis); files context-listing-branch-pinning.md
for the half deliberately deferred — neither method takes a `branch` option, and
refreshActiveBranch is a no-op outside dev, so in prod both always list the base
branch. Also logs a second instance of the known tmpdir-cleanup race.
fix: enforce path ACLs on the request-scoped listEntries and buildContentTree
`canopycms init` scaffolded `defaultBranchAccess: 'allow'` while the config
schema defaults to fail-closed `'deny'`, so every generated project opted out
of secure-by-default. Flipping the template alone was not possible: `'deny'`
was unusable for any site with non-admin editors, in two independent ways.

1. A `'deny'` branch was inert, not merely un-submittable.
   `checkBranchAccessWithDefault` never considered creator status, and
   `createContentAccessChecker` ANDs it into every content check. Since the
   create form sends no ACL, a non-admin creating a branch got one that
   appeared in the list but permitted no reads, writes or comments -- plus the
   enabled-Submit-then-403 divergence between the client and server.

2. The protected base branch had no escape hatch. It takes no ACL by design
   (an entry there would confer Withdraw rights) and its creator is the
   system, so under `'deny'` the branch every user lands on was unreachable
   with no way to configure around it.

Adds two grants to `checkBranchAccessWithDefault`, both scoped to branches
with NO ACL so an explicit ACL still restricts -- including against a branch's
own creator, which is how an admin locks down a branch someone else created:

- the creator of an un-ACL'd branch
- the protected base branch, for anonymous users too

The base-branch grant is a fallback where the bare default would otherwise
decide, never a short-circuit: short-circuiting would replace `allowed_by_acl`
with `base_branch` and silently strip Withdraw rights from ACL-listed users.
Workflow actions on the base branch stay denied, prod writes stay read-only,
and the editor API still 401s anonymous callers before authorization runs.

This also lets a public-read `deployedAs: 'server'` site run `'deny'` with
`defaultPathAccess: { read: 'allow' }` and keep un-ACL'd work branches
private, which previously required the blunt `'allow'`. `dual-build-fixture`
adopts exactly that pairing as the regression test.

Also deletes `services.checkPathAccess`, which was bound with an empty rules
array -- a well-named, zero-consumer checker that ignored every configured
rule and answered from the default alone. `createContentAccessChecker` is the
correct API and already existed.
Adds an integration suite that exercises `defaultBranchAccess: 'deny'` through
the HTTP API as a real non-admin editor. This is the only way to exercise the
default at all -- admins bypass both the branch and path layers, and every
other integration suite runs under the permissive 'allow'/'allow' workspace
default, so nothing covered the value the CLI now scaffolds.

Each assertion was checked to fail with the grants removed, not merely to pass
with them: base-branch reachability, creator read/write, and creator submit all
go red without the fix. The fourth case (a non-admin is still denied on someone
else's branch) passes either way by design -- it guards the behavior 'deny' is
supposed to keep.

One of those checks was vacuous on the first pass: it posted to `/:branch/status`
rather than `/:branch/submit`, got a 404, and satisfied a loose
`.not.toBe(403)`. Corrected to the real route with an exact status assertion.

Docs: ARCHITECTURE gains the Layer 1 precedence order, both grants and why each
is load-bearing, plus a decision record for the scaffolded default. README's
"Public read on server deployments" recipe is rewritten as 'deny' +
`{ read: 'allow' }`, and its "not read-scoped" caveat is deleted rather than
reworded -- that tradeoff no longer exists.

Backlog: acl-defaults-and-dead-path-checker.md and
client-server-workflow-permission-divergence.md move to resolved/ with the
decision and what verification turned up; inbound links and index rows updated.
The live-site migration is split out to live-site-acl-migration.md, which is
NOT done -- both adopter sites still run unscoped `defaultPathAccess: 'allow'`
(read and edit and review), the larger of the two exposures.
CODEBASE_GUIDE: authorization/branch.ts's new 4th param and full precedence
order, the two new BranchAccessResult reasons, createCheckBranchAccess's
optional config (fail-closed when omitted), the scaffolded template defaults,
and dual-build-fixture's role as the base-branch-grant regression test.

DEVELOPING: how to test the fail-closed defaults, since createTestWorkspace
defaults to permissive 'allow'/'allow' and no suite covered what the CLI
actually scaffolds. Records the two traps that cost time here -- admins bypass
BOTH layers so an admin-persona test proves nothing (and neither does clicking
around a dev site, where the default user is a bootstrap admin), and loose
`.not.toBe(403)` assertions pass on a 404 from a wrong route.

Also tightens a README line: an explicit ACL outranks the creator and
base-branch grants, but not admins/reviewers, so "always wins" was wrong.
…r-grant

fix(auth): make defaultBranchAccess 'deny' usable, and scaffold it
…rhead, docs)

- Exclude `snippet` (content-for-embedding, no route) from the example
  app's sitemap, and document the mirror-omission failure (a type with
  no route being advertised) in README and adopter-migration.md.
- Guard PostView's block-registry lookup against a stale/removed
  template name: verified with a real probe that `next build` catches it
  via the schema-validity guard, but `next dev` / request-time SSR does
  not, causing a real "Element type is invalid" 500. Added the guard to
  both the example and the README recipe.
- Short-circuit context.ts's per-listing access-checker build for the
  synthetic STATIC_DEPLOY_USER outside build/static mode, so
  createBuildCanopy() (standalone scripts) no longer provisions a
  settings-branch git workspace for a no-op ACL filter. Verified with a
  real (non-mocked) script + git-call trace, and added a non-mocked
  regression test in context.test.ts.
- Document the removal of CanopyServices.checkPathAccess in
  adopter-migration.md.
- Sanitize adopter-repo-identifying details (repo names/paths, internal
  branch names, source file paths) out of future-tasks files this branch
  added or edited, per the public-repo rule. Pre-existing disclosures and
  live-site-acl-migration.md are left untouched per instructions.
- example sitemap advertised a 404: a later change added a snippets
  collection the app never routes, so the worked example that exists to show
  'omission should be deliberate' shipped a dead URL. Documents the mirror
  failure too -- advertising a type with no route, not just forgetting one.
- block-registry recipe crashed on data at rest: compile-time exhaustiveness
  covers the schema, not content carrying a renamed template. Confirmed a 500
  under next dev; production is saved by the schema-validity guard.
- createBuildCanopy provisioned a whole settings-branch git workspace to build
  a no-op filter for the synthetic admin, and would hard-throw where a Next
  build succeeds. Short-circuited, with a non-mocked test.
- documented the checkPathAccess removal; sanitized adopter references in the
  task files this branch added.
Pre-existing and unchanged by this epic, but the epic is what makes it
matter. Reference resolution embeds a referenced entry's full data without
checking access to that entry, so a reader of A sees B through a reference
field even when denied B.

Every mitigation it was implicitly leaning on just went away: path ACLs
became real on the listing and tree paths, publish state is branch-only so
there is no per-entry flag left as a second signal, and two multi-editor
deployments are imminent. The asymmetry is the tell -- the same content is
filtered when listed and unfiltered when resolved.

Filed rather than fixed here because the substantive question is what a
denied reference should resolve to (bare ID, null, or title/URL only), and
that is a design call with adopter-visible consequences.
jpslav added 7 commits August 14, 2026 22:59
…ibCheck, doc samples)

Fixes the type/docs findings from two independent reviews:

- entry-schema.ts: defineSeoFieldGroup({ group }) now emits required: false on the
  nested wrapper, matching the runtime validator (only required: true is enforced) --
  previously an entry with no `seo:` key validated but crashed typed `data.seo.x`
  access. Regression test added; entry-schema.test.ts's prior unguarded
  Content['seo']['metaTitle'] assertion updated.
- README/docs/adopter-migration.md: document the skipLibCheck: true requirement for
  exactOptionalPropertyTypes projects (verified directly: the built package's
  dist/entry-schema.d.ts fails TS2344 under skipLibCheck: false, independent of any
  reference-field usage; skipLibCheck: true avoids it). Updated and re-prioritized
  (P2->P1) the existing eOPT task file with this more severe, build-level finding.
- docs/adopter-migration.md: fixed a generateMetadata sample that didn't compile
  (missing readByUrlPath<T> generic left result.data as unknown); compiled all 8
  migration-doc samples plus 6 README-added samples (14 total) against the real
  package via apps/example1 to confirm.
- README.md: corrected an inverted justification for the required-omitted default,
  and hedged the block-registry stray-key claim (excess-property checks are literal-
  only).
- content-listing.ts: documented parseTypedFilename's unenforced bare-filename
  precondition and its unvalidated Slug branding, rather than changing behavior.
- ARCHITECTURE.md: fixed a stale "sitemap helper when it lands" line and corrected
  the noindex-filtering claim (enumeration never filters on noindex; only the sitemap
  does).
- .claude/future-tasks: removed a net-new auth-posture disclosure naming a private
  adopter's dev-only auth; moved the now-shipped default-build-path-export task to
  resolved/; fixed an "above" that should read "below".

Verification: pnpm lint, lint:bundle, lint:tasks, typecheck, and pnpm -r run test all
green (full suite: 3756+42+26+74+143 tests passed, 0 failures).
The soundness bug is the one that mattered: defineSeoFieldGroup({ group })
typed the wrapper key as required while the runtime treated it as optional,
so data.seo.metaTitle type-checked and crashed on real content omitting the
key. Now required:false in both the type and the emitted object.

Also: documented the skipLibCheck requirement for exactOptionalPropertyTypes
adopters, fixed a migration-doc sample that did not compile at the step
labelled 'To adopt', removed a net-new auth-posture disclosure about a private
adopter, and corrected a README justification that had the validator's default
backwards.
…name build guard)

Fixes six findings from an independent static-generation review:

- listEntries() now fails an actual production build when a collection
  directory contains a content-extension file whose name doesn't parse into
  {type}.{slug}.{id}.{ext} — previously the page vanished from the build and
  the sitemap with zero output unless CANOPYCMS_DEBUG=true. Chose a red build
  over a louder warning, matching the sibling schema-validity guard's own
  stated preference; scoped to listEntries() only (not buildContentTree),
  tracked as a follow-up.
- createNextCanopyContext now takes a shared `seo` field-location option,
  bound into both generateContentSitemap and entryToMetadata via a new
  mergeSeoFieldLocation helper, so the two surfaces can no longer drift on
  where noindex/fields live.
- generateContentSitemap now throws if siteUrl isn't an absolute URL, instead
  of silently emitting a sitemap with invalid <loc> values.
- withTrailingSlash no longer appends the slash inside a query string or
  fragment.
- generateContentSitemap dedupes colliding sitemap URLs and warns instead of
  emitting duplicate <url> entries.
- Fixed collectStaticParams so basePath has no effect with shape: 'single',
  matching its own doc (it was silently dropping entries outside the prefix).

Verified with a real `next build` of apps/example1, including reproducing
the unparseable-filename build failure and confirming the sitemap output.
The HIGH: a content file whose name did not parse was skipped via log.warn,
which is gated behind CANOPYCMS_DEBUG -- so a default build emitted no page,
no sitemap URL, and no output at all. That is the silent-unpublish failure
this epic exists to prevent, one layer below the guard it added. listEntries
now throws in build mode, matching static/index.ts's own stated preference
that a red build beats silent disappearance.

Also: siteUrl is validated (a relative one produced a sitemap search engines
reject, silently); the noindex field location is configured once on the Next
context and bound into both surfaces, so the page's robots directive and
sitemap exclusion can no longer disagree; sitemap URLs dedup and warn; and
trailing slashes stop landing after a query string.
The build-time guard added in an earlier review fix (any file with a
recognized content extension that fails to parse as
{type}.{slug}.{id}.{ext} throws during `next build`) was too broad: it
also fired on files that were never meant to be an entry, most notably
a colocated sibling artifact read via an entryTransforms readSibling()
call (documented convention: `{contentId}.suffix.ext`).

Narrow the guard to shape: a successfully-parsed entry always has 4+
dot-separated segments (type, slug, id, ext), so a file with fewer
segments could never have been an entry attempt regardless of content
-- it's silently skipped, same as before the guard existed. A file
with 4+ segments that still fails to parse (unknown type, invalid ID)
still red-builds, which is the original bug this guard exists to
catch. Dot-prefixed and underscore-prefixed filenames are now always
skipped outright, matching established "not an entry" conventions.

Also make the thrown error message name the sibling-artifact
convention as a likely cause, and add a defaultPathAccess: { read:
'allow' } line to apps/example1's config so the reference app
demonstrates the public-read posture the README recommends instead of
500ing for non-admin dev users on every route.

Documented in docs/adopter-migration.md under its own heading.
The guard that turned a silent unpublish into a red build was too broad: it
also red-built a plain README in a content directory, and the sibling-artifact
layout the README itself tells adopters to commit next to their entries.

The cut is shape, not an allowlist. A parsed entry always has at least four
dot-separated segments, so a file with fewer could never have been one and is
skipped; dot- and underscore-prefixed names are always skipped. A malformed
entry still fails the build, which is the case the guard exists for.

No config knob: the shape rule covers both legitimate cases with no adopter
action, so an escape hatch would be unused surface.

Also gives the reference app the defaultPathAccess counterpart to its
fail-closed branch default, so it works on first run instead of 500ing for a
non-admin.
…tle+URL

Decided by JP. A public page linking to restricted content still has to
render the link -- an anonymous reader needs to see what they are being taken
to, even if clicking lands them on a sign-in page. A bare ID cannot be
rendered and null is indistinguishable from a broken reference.

Tagged, because a partial that looks like a full resolution makes a renderer
emit a half-empty card with no explanation -- the silent-nothing failure this
backlog keeps rediscovering.

The static case is the sharper one and it inverts the question: a static build
resolves as the admin deploy user, so no ACL fires and a public page embeds
restricted content in full into public HTML. There the check cannot be 'can
this reader see B' -- the reader is an admin -- it has to be 'is B public',
evaluated against anonymous. Wiring it the obvious way passes tests and leaks
in production.

Related section now says what the decision means for each neighbour rather
than just naming them.
@jpslav
jpslav force-pushed the integration-202608-b branch from dd033c9 to 76afad3 Compare August 15, 2026 14:25
…read

An earlier draft said the partial-resolution decision let a denied reference
carry more information through read() than a permitted one does through
listEntries(). That compares across two APIs answering different questions
and is not a meaningful ordering.

Within each API the behaviour is monotonic: read() gives full data when
permitted and title+URL when denied; listEntries() gives a bare ID to
everyone because it does not resolve references at all, not because of any
access decision. Nobody gains by being denied.

What remains is a shape inconsistency that predates this decision, plus the
one constraint worth carrying: if listEntries ever resolves references it
inherits the phase-dependent question, and implementing it with the
request-time question only would pass every test and leak in a static build.
@jpslav jpslav changed the title epic: go-live for docs-site-proto and website epic: package work for the first two deployed-editor go-lives Aug 15, 2026
CodeQL js/polynomial-redos, high, and genuinely ours -- introduced with the
sitemap helper. `siteUrl.replace(/\/+$/, '')` retries the quantifier from
every position before failing the end anchor, so a value that is mostly
slashes but does not end in one costs quadratic time. siteUrl is
adopter-supplied and reaches this from config or an env var.

Measured on a 40k-slash input: 2377ms with the regex, 0.013ms with a
character scan, output identical across every parity case.

Regression test asserts a 250ms ceiling -- generous against the 0.01ms real
cost, but it fails loudly if a regex is reintroduced.

Three review rounds missed this; static analysis is genuinely covering a
different axis than reading does.
@jpslav
jpslav marked this pull request as ready for review August 15, 2026 14:44

@debshila debshila left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — PR #235 (epic: package work for the first two deployed-editor go-lives)

Branch: integration-202608-bmain · Scope: 96 files, +7,782 / −539, 34 commits
Reviewed at: 61c6e95 (merge base db4f871 = origin/main exactly — a clean fast-forward, zero commits on main not in this branch, so #229's landed work is already underneath this)
CI: all 10 checks green (validate/typecheck/test, dual-build, 4× e2e shards, merge-report, CodeQL ×3). One automated review (github-advanced-security); no human review, no comments.


Verdict

Approve with one fix first. This is the most adopter-facing epic in the repo so far and the shape of it is right: every change is generally applicable, docs/adopter-migration.md names what each one makes deletable, and the two deliberate decisions (publish-state-is-branch-only, ACLs enforced on the request-scoped listing) are load-bearing on each other rather than independently chosen. The authorization work in particular is the strongest part — checkBranchAccessWithDefault's new precedence ladder is correct at every rung I traced, and the tests pin the three cases that matter (creator wins over bare default, explicit allowlist wins over creator, managerOrAdminAllowed wins over both).

One finding should land before merge:

  • #1 — a confirmed, measured polynomial ReDoS in toPlainText's TAG_RE, added in this PR. 26.3 seconds on a 128 KB body, quadratic scaling, and the trigger is an unterminated <Tag followed by a run of whitespace — a plausible shape in real MDX. I verified a one-token fix that removes the blowup entirely (26,321 ms → 0.63 ms) with identical output on nine tag shapes.

The PR body closes with: "CodeQL then caught one thing all three rounds missed: a polynomial-ReDoS regex in siteUrl normalization… Worth noting that static analysis found what three careful readings did not." That lesson generalizes further than the PR takes it. Of my top three findings, all three are the same defect class as that CodeQL alert, all in this PR, and none was flagged: the regex above (#1), the startsWith('//') off-site check that #229 removed from sanitizeHref two weeks ago and this PR reintroduces in isAbsoluteUrl (#2), and the exact /\/+$/ pattern CodeQL flagged — still present, unchanged, in the example app that adopters copy (#3). One fix was applied where the alert pointed and nowhere else.

Everything else is follow-ups.


Coverage — what this review actually covered

Read in full and reasoned about:

  • authorization/branch.ts + authorization/types.ts (the whole precedence ladder), protected-branch.ts for the grant it keys off
  • content-listing.ts (parseTypedFilename, looksLikeMalformedEntry, the build-mode throw, updatedAt), content-tree.ts (visibility threading + collection pruning)
  • context.ts (resolveVisibilityImpl and its three short-circuits), services.ts, build-canopy.ts
  • static/seo.ts (whole file, adversarially), static/index.ts, canopycms-next/src/static.ts, context-wrapper.ts
  • entry-schema.ts (the optional-property inference, defineSeoFieldGroup overloads, BlockComponentRegistry)
  • content-reader.ts (ContentReadMeta), ai/to-plain-text.ts (regex pipeline)
  • docs/adopter-migration.md (build-guard + SEO sections in full, the rest skimmed for claims), the README's new sections, all four canopycms.config.ts files and the CLI template, apps/example1/app/{lib/canopy.ts,sitemap.ts}

Verified by running code rather than reading (details in the findings): the TAG_RE timings and the candidate fix; isAbsoluteUrl/resolveSeoUrl/withTrailingSlash against eight URL spellings including WHATWG backslash forms, checking what a browser actually resolves each output to. I did not run the full suite or the e2e shards — my local node_modules is missing several packages, and CI's green is the evidence for those.

Not covered: the 14 new/moved .claude/future-tasks/ files and program-log.md were read for claims about the code, not audited as documents; apps/example1/app/components/PostView.tsx; the large new test files (static.test.ts, entry-schema.test.ts, content-listing.test.ts, context.test.ts, to-plain-text.test.ts) were read for what they assert, not line-read; CODEBASE_GUIDE.md. Treat absence of findings there as "not looked at".


Claims I verified independently

Claim Result
'deny' is now genuinely usable — creator and base-branch grants close the divergence Holds, and the precedence is right at every rung. Creator is scoped to the no-ACL case (an explicit allowlist omitting them still denies — pinned by a test), managerOrAdminAllowed short-circuits ahead of both, and the base-branch grant sits below defaultAccess so it can never replace allowed_by_acl or override denied_by_acl. The four new branch.test.ts cases cover exactly these.
The base-branch grant "cannot widen anything dangerous" Holds on all three named guards. canPerformWorkflowAction disables its system-branch grant on the same flag (so access says yes and the action still resolves false — there is a test for it); getBranchWriteProtection().readOnly blocks prod writes; the path layer still decides reads and defaults to 'deny'.
Path ACLs are enforced on listEntries and buildContentTree, including meta.indexEntry Holds. The predicate is applied in listEntries before extract runs, and buildContentTree routes every read through listVisibleEntries, so a denied index entry never reaches extract. The "collections whose children are all filtered out are pruned" claim is real code (if (allChildren.length === 0) return null), not just a doc sentence.
The ContentVisibilityOptions predicate is not adopter-supplyable Holds. It is a separate positional parameter on listEntries/buildContentTree, not a key on ListEntriesOptions/BuildContentTreeOptions, and context.ts is the only producer. An adopter cannot widen or override it through the public options object.
The three visibility short-circuits are load-bearing, not just an optimization Holds. createContentAccessChecker calls getSettingsBranchRoot(), which provisions/clones the settings workspace — so for createBuildCanopy (a standalone script, where neither isBuildMode() nor isDeployedStatic fires) the user === STATIC_DEPLOY_USER guard is the only thing preventing a settings-workspace clone the script never needed.
required: false → optional property; omitting required stays required Holds. The two-mapped-type intersection keys on F['required'] extends false, which boolean | undefined does not satisfy, so the three-way distinction is real. Simplify is the standard homomorphic idiom and preserves ?. Both directions are pinned in entry-schema.test.ts.
defineSeoFieldGroup({ group })'s wrapper key is now optional, matching the runtime validator HoldsNestedSeoFieldGroup carries required: false, and the flat variant is type: 'group', which FlattenInlineGroups flattens so the seven fields land at top level as optional. Schema names come from DEFAULT_SEO_FIELD_NAMES, the same constant extractSeoFields reads, so the two ends cannot drift.
noindex is derived from ONE predicate feeding both surfaces Holds structurally, not just by convention: entryToMetadata and generateContentSitemap both call isNoindexEntry, and mergeSeoFieldLocation binds the field location from one context-wide default with a deliberate key-by-key merge (a naïve spread would clobber it with undefined, and the comment says so).
generateContentSitemap is bound to the build context, so ACL filtering never empties a sitemap Holds. boundGenerateContentSitemap calls getCanopyForBuild(), whose user is STATIC_DEPLOY_USER → visibility short-circuits to {}. A request-time reach on a prod server throws via guardBuildContext instead of silently returning a filtered sitemap.
resolveEntryTitle is client-safe enough to re-export from the root entry Holdsutils/title-field.ts's only imports are FieldConfig/InlineGroupFieldConfig types, erased at compile time.
basePath no longer silently drops entries under shape: 'single' Holds — the filter is now gated on shape !== 'single', matching the option's own documented "no effect with shape: 'single'".
The ReDoS fix in stripTrailingSlashes is correct Holds — a linear character scan, no regex. See finding #3 for where it did not get applied.

Findings

1. HIGH — toPlainText's TAG_RE is a polynomial ReDoS (measured: 26.3s on 128 KB)

packages/canopycms/src/ai/to-plain-text.ts:69

const ATTR_CONTENT = `(?:[^<>"']|"[^"]*"|'[^']*')*`
const TAG_RE = new RegExp(`<\\/?([A-Za-z][\\w.-]*)?(?:\\s${ATTR_CONTENT})?\\s*\\/?>`, 'g')

ATTR_CONTENT's [^<>"'] already matches whitespace, and the \s* that follows it matches whitespace too. Every whitespace character after an unterminated tag can therefore be split between the two in any proportion, so when the closing > never arrives the engine enumerates every split. Measured on '<Callout\n' + '\n'.repeat(n) + 'text':

input size current with \s* removed
8 KB 102 ms 0.28 ms
32 KB 1,691 ms 0.11 ms
128 KB 26,322 ms 0.63 ms

Clean quadratic (~16× per 4× input). Prose after the stray tag is not a trigger (330 KB of ordinary paragraphs: 2.6 ms) — the tail has to be whitespace-dominated, i.e. an unterminated <Tag followed by blank lines or a long indented block. That is an ordinary content-authoring accident in MDX (a literal < before a word, a typo'd component open), not a crafted payload.

Why this is worth blocking on rather than filing: it is the same defect class the PR's own review section says CodeQL caught and three review rounds missed, introduced in the same PR, on an input the PR's own stripTrailingSlashes comment already argues counts as uncontrolled ("siteUrl is adopter-supplied and can reach here from config or an env var"). Entry bodies are at least as uncontrolled as an env var, and toPlainText is pitched at search-index builders — i.e. code that walks every entry, so one bad file stalls the whole index build or next build. CodeQL is green here, so the alert that motivated the siteUrl fix does not cover this.

Fix, verified: delete the redundant \s*.

const TAG_RE = new RegExp(`<\\/?([A-Za-z][\\w.-]*)?(?:\\s${ATTR_CONTENT})?\\/?>`, 'g')

ATTR_CONTENT already absorbs the trailing whitespace, so nothing is lost. I checked output parity on <br />, <a>, </a>, <a >, <Foo bar="1 > 2" />, <>, </>, <Callout>hi</Callout>, <a\nhref="x"> — identical in every case. A regression test in the shape of the siteUrl one (assert a bounded runtime on a 128 KB whitespace tail) pins it.

2. MEDIUM — isAbsoluteUrl reintroduces the startsWith('//') off-site check that sanitizeHref removed two weeks ago

packages/canopycms/src/static/seo.ts:158

export function isAbsoluteUrl(url: string): boolean {
  return /^[a-z][a-z0-9+.-]*:\/\//i.test(url) || url.startsWith('//')
}

utils/sanitize-href.ts, already on main under this branch, carries this comment:

This is enforced by checking whether the input DECLARES a scheme, not by matching a // prefix: WHATWG URL treats backslash as equivalent to slash for special schemes, so /\evil.com, \\evil.com and \/evil.com are all protocol-relative in effect. An earlier version of this function checked startsWith('//') and let all three through…

The same spellings defeat isAbsoluteUrl. Measured, with each output resolved the way a browser would resolve it against the page origin:

entry field value isAbsoluteUrl resolveSeoUrl(v) (no siteUrl) browser resolves to
/\evil.com false /\evil.com https://evil.com/
\\evil.com false /\\evil.com https://evil.com/
\/evil.com false /\/evil.com https://evil.com/
/\t/evil.com false /\t/evil.com https://evil.com/

entryToMetadata documents omitting siteUrl as a supported mode ("Omit to leave them relative and let Next's metadataBase resolve them"), and that is exactly the path where this bites: an entry's canonical or ogImage reaches <link rel="canonical"> / og:image / openGraph.url pointing at an attacker-controlled origin. With a siteUrl supplied the value is prefixed and stays same-origin, so the exposure is the no-siteUrl configuration specifically.

The input is CMS-author-supplied, so this is not an anonymous XSS — it is SEO poisoning (a canonical tag handing your ranking to another origin) and a social-card/referrer leak, reachable by anyone with write access to one entry. Lower severity than #229's open redirect; identical mechanism.

There is a second, unrelated consequence of the same predicate: //cdn.example.com/x in extraUrls returns true, passes through verbatim, and lands in the sitemap as a non-absolute <loc> — the precise invalidity generateContentSitemap throws about for siteUrl ("search engines silently reject the entire file"), just not checked for the per-URL values.

Fix direction: apply sanitizeHref's own conclusion — test whether the input declares a scheme (/^[a-z][a-z0-9+.-]*:/i) and treat anything else that the URL parser resolves off-origin as not-absolute-but-not-safe-either. Better still, have one of the two functions call the other; two independently-maintained answers to "is this URL off-site" in one package is the drift this whole epic is about.

3. MEDIUM — the ReDoS fix was applied at the alert site only; the example app still ships the flagged pattern

apps/example1/app/lib/canopy.ts:90

export const SITE_URL = (process.env.NEXT_PUBLIC_SITE_URL ?? 'http://localhost:3000').replace(
  /\/+$/,
  '',
)

That is character-for-character the pattern the PR body describes replacing:

a polynomial-ReDoS regex in siteUrl normalization, introduced with the sitemap helper. Measured at 2377ms versus 0.013ms on a 40k-slash input; replaced with a linear scan and pinned by a regression test.

Same operation, same input (siteUrl), same env-var provenance the stripTrailingSlashes comment cites as the reason it counts as uncontrolled — in the file this repo holds up as the reference integration. CodeQL did not flag it (checks are green), which is the useful signal here: the alert was treated as the bug rather than as one instance of it.

The mechanical cause is that the fix was made unshareable: stripTrailingSlashes is a module-private function in seo.ts, not exported and not re-exported from canopycms/server, so there is no package-provided way to do this correctly. apps/example1 had to re-roll it, and so will every adopter who follows the README's SITE_URL pattern. By the PR's own ranking signal — "when two teams with no contact build the same thing, the package is missing it" — this qualifies before anyone has built it twice.

Fix direction: export stripTrailingSlashes (or fold it into resolveSeoUrl's contract so callers never need it) and use it in apps/example1. Practically the exposure here is nil — build-time, builder-controlled env var — but the example is copy-paste material, and shipping the flagged pattern in it while the changelog says it was replaced is the kind of claim/code gap this repo audits for.

4. MEDIUM — the unparseable-filename build guard misses the shape most likely to be a lost page

packages/canopycms/src/content-listing.ts:336, 468

The guard's stated purpose is that a page must never disappear from a build with no output. looksLikeMalformedEntry gates it on filename.split('.').length >= 4, reasoning that a valid entry always has four segments so anything shorter "could never have parsed as an entry regardless of its content".

That is true about parsing and false about intent. The likeliest way a file loses its parse is losing its ID segment — a hand-created file, a bad rename, a merge — and post.hello-world.md has three segments. It fails parseTypedFilename (parts.length < 3 after the extension strip), is skipped, and looksLikeMalformedEntry returns false, so it never reaches skippedFiles and the build stays green with the page gone. The guard fires on post.hello-world.BADID.md (wrong ID, 4 segments) but not on post.hello-world.md (no ID, 3 segments) — and the second is the more common accident.

I understand why the narrowing happened: the first version red-built a plain README.md and the documented {contentId}.suffix.ext sibling artifact, and docs/adopter-migration.md records that correction honestly. But segment count is a proxy for "was this meant to be an entry", and it is a proxy that fails on the motivating case. A shape test that also catches {knownEntryTypeName}.{anything}.{ext} would cover the ID-loss case without touching README.md (2 segments, first segment not an entry type) or 5NVkkrB1MJUv.profile.json (first segment not an entry type either).

Worth stating in the same breath: the guard is listEntries-only. buildContentTree calls listCollectionEntries with no onSkip, so an entry lost from the tree is still silent in every mode. .claude/future-tasks/build-content-tree-silent-skip.md files it, which is the right way to ship a partial fix — noting it here so the guard isn't read as covering more than it does.

5. MEDIUM — a freshly scaffolded project renders nothing until the adopter edits the config

packages/canopycms/src/cli/template-files/canopycms.config.ts.template

canopycms init now scaffolds defaultBranchAccess: 'deny' and defaultPathAccess: 'deny'. Flipping the branch default to match the schema is unambiguously right — the generator writing 'allow' against a fail-closed schema was the bug. But the path default now makes the first-run experience a wall: apps/example1 needed exactly this to keep working, and its own comment says why:

Without this, a non-admin visitor (including the dev-auth default user) gets "Forbidden: path access denied" on every route, since defaultPathAccess otherwise falls back to fully closed.

So canopycms initnpm run dev → every page 403s, for the scaffolded developer themselves, until they find defaultPathAccess in the README. That is a strictly worse first run than the pre-flip 'allow'/'allow' pairing, and "secure by default" doesn't require it: the two decisions are separable, and only the branch one was the divergence being fixed.

The template comment does point at README's Public read on server deployments, which is the correct pointer — the gap is that the scaffold's own smoke test is the failure. Three options, in my order of preference: scaffold defaultPathAccess: { read: 'allow' } (public read, edit/review still closed — the posture the template comment itself recommends and apps/example1 adopts); or keep 'deny' and add the working { read: 'allow' } line commented out directly above it; or leave it and make the 403 name the config key, so the first-run failure is self-diagnosing rather than a generic "Forbidden".

Also worth a line in docs/adopter-migration.md: an existing adopter who relied on the old 'allow' default and never wrote defaultPathAccess is unaffected (their config still says what it said), but one who copied the scaffold and later removes the key inherits the closed default. I didn't find this transition covered there.

6. LOW–MEDIUM — the epic's own "don't let two implementations drift" test, applied to the epic

The PR's review guidance is explicit:

an upgrade that adds a new API without removing what it supersedes leaves two implementations to drift apart, which is the failure mode most of these changes exist to end.

Three places in this diff now hold a second implementation of something the package already answers:

  1. isAbsoluteUrl vs. sanitizeHref's scheme test (finding #2) — two answers to "is this URL off-site", one already corrected.
  2. apps/example1's /\/+$/ vs. stripTrailingSlashes (finding #3) — two answers to "strip trailing slashes", one already corrected.
  3. canopycms-cdk's isValidDeploymentName vs. the runtime copy — pre-existing, and handled correctly (a fixture-driven drift test), which is the model the other two are missing.

Nothing here is individually urgent; together they suggest the migration guide's "now deletable" column deserves an internal counterpart — when a fix lands, grep for the pattern it replaced rather than only the file the alert named.


Smaller notes

  1. Sitemap ordering is locale-dependent. canopycms-next/src/static.ts:216 sorts with a.url.localeCompare(b.url) and no locale argument, while the doc directly above promises "Output is stably ordered … so a rebuild doesn't reshuffle the file." Two build agents with different ICU defaults can emit different orders, producing a spurious diff. localeCompare(b.url, 'en'), or a plain </> comparison, makes the promise true. (This is the same machine-dependence class as the timezone-hardcoded test #229 documented.)

  2. mergeSeoFieldLocation has no way to say "flat, just this once". By design an omitted or explicitly undefined group falls through to the context-wide default — correct for the common case, and documented. But there is then no per-call override back to the flat convention; group: '' happens to work (it is falsy, so extractSeoFields reads the entry directly), which is an undocumented accident rather than an API. If per-call opt-out matters, group: null with an explicit check would say it on purpose.

  3. entry-schema.ts gained two new dependencies on the isomorphic entrypoint. import type { ComponentType } from 'react' (erased, but it means @types/react must resolve to typecheck canopycms's root entry — relevant given the PR already documents a hard skipLibCheck: true requirement) and a value import of DEFAULT_SEO_FIELD_NAMES from ./static/seo, which pulls the whole ~230-line SEO module into the client bundle graph for one constant. lint:bundle passes and seo.ts is pure, so neither is a break — but the constant would sit more naturally in its own module.

  4. createBuildCanopy is a new ACL-bypassing public export. canopycms/server's createBuildCanopy reads as STATIC_DEPLOY_USER and bypasses every branch and path check, with nothing preventing an adopter from calling it inside a server component. The JSDoc is as strong as prose can be (the security note names the misuse and points at getCanopy()), and getCanopyForBuild already had the same property — noting it because this is the first time that capability is a one-call factory an adopter is invited to use, so the footgun is now easier to reach than the guard. A runtime isBuildMode() warning on first use would cost nothing.

  5. parseTypedFilename's directory-component precondition is documented but unenforced. The JSDoc is unusually clear that 'foo/bar.slug.<id>.md' parses to type: 'foo/bar' and that callers must path.basename first. Now that it is a public export reachable by adopter code walking a filesystem — which is the stated motivation — a one-line if (filename.includes('/') || filename.includes('\\')) return null would make the contract self-enforcing at no cost. The returned slug carrying the Slug brand without running parseSlug is in the same category: documented, and a trap for the next caller who trusts the brand.

  6. Collection metadata is not path-ACL filtered, only entries. buildContentTree's extract(collectionData, …) runs for any collection with at least one visible descendant, and a maxDepth-capped collection returns its node without loading entries at all, so it is never pruned. The context doc says "entries the current user cannot read are omitted", which is accurate as written — flagging it so nobody reads the tree as fully ACL-scoped. Collection paths and .collection.json fields are disclosed structurally.

  7. dedupeSitemapItems warns to bare console.warn. Legal — the project-wide no-console allows warn, and canopycms-next is outside the worker-reachable override list. Consistent with the package; noting only because the repo's logging discipline is otherwise strict enough that a reader might expect canopyLogWarn.


What's good, specifically

  • The precedence ladder in checkBranchAccessWithDefault is documented as a ladder — numbered, highest-first, in the function's own doc — and every rung's scoping rationale is written at the site that could plausibly be "simplified" into a bug ("a short-circuit higher up would replace allowed_by_acl with base_branch and silently strip Withdraw rights"). That is the comment that stops the next refactor.
  • The two grants pulling in opposite directions on one flag (isProtectedBranch grants access, disables workflow actions) is genuinely subtle, and both sides say so, each pointing at the other.
  • ContentVisibilityOptions as a separate parameter rather than an options key is the right shape, for the reason stated: adopter code must not be able to supply, widen, or override the access check. Easy to get wrong by putting it on ListEntriesOptions where it would have read more naturally.
  • docs/adopter-migration.md's "Now deletable" column is the best idea in the epic. Naming the pattern of superseded code rather than a specific adopter's files is what makes it reusable, and the guard-narrowing entry's "you can move it back — relocating it may have silently broken the entryTransforms call that reads it" is the rare migration note that fixes damage the previous version caused.
  • Shipping the sitemap and SEO helpers as one change with the reason stated (noindex is one predicate feeding two surfaces) is the correct call, and mergeSeoFieldLocation makes the agreement structural rather than a matter of adopter discipline — including the explicit note on why a naïve spread would silently clobber the default.
  • Every-routable-type-by-default, with the mirror failure documented. Both directions are named: the type nobody remembered to add, and the type that has no route. The apps/example1 sitemap then demonstrates all three exclusions with a reason each, including "modelling home as a root index entry instead would remove the need for both lines" — showing the better design next to the workaround.
  • updatedAt's caveat is repeated at all four places it surfaces (ListEntriesItem, RoutableEntry, the sitemap option, the README) rather than stated once and assumed. A checkout-time mtime silently becoming a public <lastmod> is exactly the kind of thing one un-caveated call site produces.
  • The required: false inference is pinned in both directions, and the README table spells out the three-way distinction plus the runtime/type divergence it does not fix ("typed as present but validated as absent-tolerant") — stating the residual gap rather than implying the change closed it.
  • BlockComponentRegistry ships as a type with a documented loop instead of a renderBlocks(), and the README's loop carries the one assertion it needs with the reason it is safe, plus the if (!Component) return null guard and why compile-time exhaustiveness doesn't cover data at rest. That paragraph is the finding from review round three, written where a reader hits it.

Merge notes

  • Clean fast-forward: git rev-list --count origin/main ^HEAD is 0, so there is nothing to reconcile and no conflict surface. mergeStateStatus: BLOCKED is the review requirement.
  • #1 should land on this branch — a one-token regex change plus a bounded-runtime regression test, matching the one already written for stripTrailingSlashes.
  • #2 and #3 are small and thematically inseparable from #1; I'd fold them in while the context is fresh, since all three are the same class and #3 is a two-line change once stripTrailingSlashes is exported.
  • #4 and #5 are judgment calls that deserve an explicit decision rather than a silent inheritance — #5 in particular changes what every new adopter sees on their first npm run dev.
  • The PR body asks for a normal merge commit rather than a squash, with the reasoning (history verified clean, per-change commits worth keeping for bisect). I agree — the commit granularity here is genuinely useful, and this epic is the kind that gets read back.
  • One process observation, offered as such: the three review rounds were organized by dimension (authorization, types, static generation, docs), and the three findings above are all the same defect class crossing three of those dimensions — a regex in ai/, a URL predicate in static/, a copy in apps/. A dimension-partitioned review is structurally unlikely to notice a pattern that spans partitions, which may be the more durable lesson than "static analysis found what three careful readings did not" — CodeQL didn't find these either.

jpslav added 2 commits August 15, 2026 10:49
Five findings from a human review of PR #235:

1. HIGH: toPlainText's TAG_RE was a polynomial ReDoS (redundant `\s*` after
   ATTR_CONTENT already matched whitespace) -- measured 28.4s/128KB, fixed to
   0.74ms. Also found and fixed a second, more severe ReDoS in the same PR's
   fenced-code-block regex (`/^(```|~~~).*\n([\s\S]*?)\n\1\s*$/gm`, unbounded
   on unclosed fences) in both ai/to-plain-text.ts and ai/transform-components.ts
   -- measured 9s/530KB, fixed to well under 10ms via a linear line-scan with
   a precomputed nearest-closer lookup, verified against 13,000+ fuzz trials
   for exact output parity with the replaced regex.

2. MEDIUM: isAbsoluteUrl reintroduced the startsWith('//') off-site check that
   sanitizeHref removed two weeks ago for missing WHATWG backslash-equivalent
   spellings (/\evil.com, \\evil.com, \/evil.com). Extracted shared
   declaresScheme/isImplicitlyOffOrigin/neutralizeImplicitOffOrigin helpers to
   utils/sanitize-href.ts; resolveSeoUrl now neutralizes these spellings
   instead of passing them through as a trusted relative path.

3. MEDIUM: the ReDoS fix landed only where CodeQL pointed. Exported
   stripTrailingSlashes from canopycms/server and used it in
   apps/example1/app/lib/canopy.ts in place of the flagged `replace(/\/+$/, '')`.
   Repo-wide grep found no other live instances of that exact pattern; two
   structurally similar but currently-neutralized trim regexes (cli/migrate.ts,
   assets/keys.ts) captured as a future-task.

4. MEDIUM: looksLikeMalformedEntry's build guard missed the likeliest lost-page
   shape -- losing the ID segment entirely (`post.hello-world.md`, 3 segments)
   built clean with the page silently gone. Now also flags a 3-segment file
   whose first segment matches a known entry type, without touching README.md
   or a {contentId}.suffix.ext sibling artifact. Proved with four real
   `next build` runs against apps/example1.

5. MEDIUM: canopycms init's scaffolded config paired defaultBranchAccess: 'deny'
   with defaultPathAccess: 'deny', 403ing every route (including the dev-auth
   default user) on first `npm run dev`. Now scaffolds
   defaultPathAccess: { read: 'allow' }, matching apps/example1 and the
   template's own comment.

Also files a future-task: DEVELOPING.md claims `next build` reads the working
tree directly in dev mode; verified via a real build that it actually reads
the .canopy-dev branch clone (seeded from git-committed state) unconditionally,
same as `next dev` -- the exact trap this session's task briefing warned about.

Verification: full monorepo test suite (3787+87+26+42+143 tests, 0 failures),
typecheck, lint, lint:bundle, lint:tasks all clean; four next-build cases
proved against apps/example1's real content.
- toPlainText's TAG_RE was a polynomial ReDoS: 28.4s on a 128KB body,
  triggered by an unterminated tag followed by whitespace -- an ordinary MDX
  authoring accident, on a function aimed at code that walks every entry.
- A second, worse one the review did not catch: the fenced-code regex in
  to-plain-text.ts and transform-components.ts, 9s on unclosed fence openers.
  Replaced with a linear line scan, output verified byte-identical over
  13,000+ fuzz trials.
- isAbsoluteUrl reintroduced the startsWith('//') check sanitize-href removed
  two weeks ago, so WHATWG backslash spellings reached canonical and og:image
  as off-origin URLs. Both now share one answer to 'is this off-site' rather
  than maintaining two.
- The example app still shipped the /\/+$/ pattern CodeQL flagged elsewhere,
  in the code adopters copy from.
- The build guard missed the likeliest lost-page shape: a file that lost its
  ID segment has three segments and slipped through the >=4 test.
- A scaffolded project 403'd on every route on first run; the path default is
  separable from the branch default that was the actual divergence.
@jpslav

jpslav commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Thank you — this found more than three rounds of my own review did, and the process
observation is the most useful part.

All five actioned, on c8a74576. Full suite green (4126 tests), lint / lint:bundle /
lint:tasks / typecheck clean.

#1 — fixed. Reproduced your measurement independently: 28,443ms → 0.74ms on 128KB.
Redundant \s* removed, parity re-checked on your nine tag shapes, bounded-runtime
regression test added in the same shape as the stripTrailingSlashes one.

And a second, worse one your review didn't reach, found by sweeping the class rather
than the site: the fenced-code regex /^(```|~~~).*\n([\s\S]*?)\n\1\s*$/gm, present in
both to-plain-text.ts and transform-components.ts9,041ms on ~530KB of
unclosed fence openers. Replaced with a linear line scan using a precomputed
nearest-closer lookup, output verified byte-identical across 13,000+ fuzz trials.

#2 — fixed, by the route you preferred. isAbsoluteUrl and sanitizeHref now share
one answer: declaresScheme, isImplicitlyOffOrigin, neutralizeImplicitOffOrigin are
exported from utils/sanitize-href.ts and consumed by static/seo.ts. Confirmed via
Node's URL parser that the backslash spellings now stay same-origin with and without
siteUrl, while a literal //cdn… still passes through verbatim. Your note that
extraUrls can still land a protocol-relative <loc> is filed separately rather than
folded in silently.

#3 — fixed, and you were right about the shape of my mistake. I fixed the line CodeQL
named and swept only packages/**, which structurally could not see apps/. Repo-wide
grep now: no other live /\/+$/. Two structurally similar patterns in cli/migrate.ts
and assets/keys.ts are safe today only because upstream hyphen-collapsing bounds their
input — benchmarked to confirm, and filed, since that safety depends on step ordering
rather than on anything guaranteeing it.

#4 — fixed. Segment count replaced by an entry-type test, so a file that lost its ID
is caught. Four real next builds: post.hello-world.md FAILS (named in the error),
post.hello-world.BADID.md FAILS, README.md + a {contentId}.profile.json sibling
together BUILD CLEAN (17/17 pages).

#5 — applied your first-preference option. Template now scaffolds
defaultPathAccess: { read: 'allow' }, with a pinning test and the migration-guide note
about an adopter who later removes the key.


On the process observation — you're right, and it corrects a lesson I'd drawn too
narrowly. I'd concluded "static analysis found what three careful readings did not." The
sharper version is yours: my rounds were partitioned by dimension, your top three
findings are one defect class appearing once in each of three dimensions, and a
dimension-partitioned review is structurally unable to notice that — each reviewer sees
one instance and judges it local. CodeQL didn't find them either; it caught a fourth
instance in a fifth location.

The second ReDoS above is that lesson applied: it was found by asking "where else does
this class live", not by reading ai/ again.

Two incidental findings from the build verification, both filed:
next build reads .canopy-dev/content-branches/<branch>/ even at build time under
mode: 'dev', which contradicts DEVELOPING.md — three agents lost time to it before
this one did; and the extraUrls protocol-relative <loc> case from #2.

Follow-ups #6#13 are filed to .claude/future-tasks/ rather than actioned here.

@jpslav
jpslav merged commit a8c9c9c into main Aug 15, 2026
10 checks passed
@jpslav
jpslav deleted the integration-202608-b branch August 15, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants