RFC: Completing EmDash comments — from "it works" to best-in-class engagement #1497
Replies: 5 comments 1 reply
|
I like this. I'd be open to have the central parts in core. I do think reactions should be opt-in though. I would say upvote only for now. I think the cmponent registry ideas are scope creep and shouldn't be included here. I don't think real-time is worth the extra complexity. I'll approve reactions for PR. The dsecription in the tier 1 section looks good. Thanks! |
|
Follow-up now that Tier 1 (reactions + Best sort, #1540) has merged — proposing the next increment: optimistic comment display. Running reactions on a live site surfaced the next obvious friction point: when a visitor posts a comment, nothing appears until they manually reload the page. The This is especially visible once a collection runs Proposal (additive, backward-compatible, no framework island — same philosophy as reactions)Opt-in via a new
Why core (not a plugin)Same dividing line as the reactions work: the render slot and the POST response shape live in core's OfferHappy to prototype #1–2 on a branch the same way as |
|
Bumping this — it's been a bit over a week since the optimistic-display proposal above. No pressure on timing, just want to make sure it didn't get lost in the queue. Happy to go ahead and prototype #1–2 (optimistic insert + moderation-aware pending state) on a held branch the same way as |
|
Following up with Tier 2 — member accounts, public profiles, delete-your-own-comment from the original proposal (§5). Same offer as the optimistic-display thread above: happy to open the PR immediately if there's appetite, since it's already built and tested on a held branch ( What it is, in one line: reuse the existing Scope actually built (trimmed from the original Tier 2 sketch — see note below):
Deliberately not in this pass: the subscriptions/notifications half of the original Tier 2 sketch ( Same additive/backward-compatible discipline as Tier 1: new columns are nullable, new routes are opt-in, nothing existing changes. Built with tests (TDD per-task, red→green), typechecked, and passed a whole-branch review pass — one real bug that review caught (a public-route auth-resolution gap that would've made delete-your-own-comment silently 401 in production) is already fixed and covered by a regression test. Built with AI-assisted development (Claude, ultracode) — will fill out the PR template's AI disclosure box when it gets there, per CONTRIBUTING.md. |
|
Follow-up on the Tier 2 comment above — I went back through the discussion history and found something I should have addressed up front: you've already raised a specific, unresolved concern about exactly this design decision, twice. In #1315 (open signup + public subscriber profiles, a near-identical ask):
And in #1467 (paywalls):
Our Tier 2 build reuses the shared Why we still think it's defensible for comments specifically, as a narrower case than open signup / paywalls generally:
That said — you've now said this twice, and a community member (@rmakk, commenting on #1467) independently reached the same conclusion and built a separate plugin-based membership system for exactly this reason. That's real signal, not noise. So: we're happy to prototype the alternative — a genuinely separate, non-admin-shared account primitive for public commenters/subscribers, built to whatever security bar you'd want to see before it touches core. Rather than guess and burn effort on the wrong one, would rather ask directly: is the shared- |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
TL;DR
EmDash already ships a comment system in core — and it's a genuinely good "lightweight self-hosted" foundation (threaded, moderated, Turnstile-protected, hook-extensible, with an AI-moderation plugin already proving the extension model). The gap between what exists and "the best commenting system on the internet" is not a rewrite. It's a focused set of additive core primitives (reactions, commenter notifications, real-time, a public-component slot) plus a plugin ecosystem on the hooks that already exist.
This RFC proposes the dividing line and a tiered, backward-compatible roadmap:
Demo — working prototype (Tier 1: reactions + "Best" sort)
To de-risk the proposal rather than just describe it, I've already built and validated Tier 1 (reactions + a Reddit-style Wilson "Best" sort) end-to-end against
demos/simple. The branch is ready to PR the moment this isApproved for PR.Before — comments today (chronological, no reactions):
After — opt-in
<Comments reactions sort="best" />: one-tap likes with live counts, and the most-liked comment rises to the top (Wilson lower-bound, so a couple of likes can't outrank a heavily-liked comment); replies stay chronological:What landed in the prototype (all additive, behavior-preserving defaults):
_emdash_comment_reactionstable (migration042), deduped per voter by a salted IP hash — the same privacy primitive as commentip_hash.GET/POST /_emdash/api/comments/:collection/:contentId/reactions, registered in the Astro integration (verified live against the demo: toggle on/off and aggregate counts return the expected JSON).<Comments>gains two opt-in props —reactions(like button + live counts) andsort="best". Posting is progressively enhanced by a tiny inline script that is emitted only when reactions are enabled, so non-reaction pages ship zero extra JS. Counts and ordering are server-rendered (SEO + first paint); the client script only handles toggling.Validation: lint clean (0 diagnostics), typecheck clean, 118 tests passing — 15 new (Wilson math, toggle/dedupe, counts, rate-limit, best-sort ordering) plus the existing comment/migration/client/API suites. Includes a changeset (
emdash: minor).This is offered as a concrete first step (§5, Tier 1) — happy to adjust scope/shape based on maintainer feedback before opening the PR.
1. Motivation
One of EmDash's stated reasons to exist is to beat WordPress. WordPress and Disqus — the two defaults of the open web — share the same failure: a slow, low-engagement, spam-prone commenting experience. Comments are where social engagement and time-on-page are won or lost, and both incumbents leave that value on the table:
EmDash's architecture is purpose-built to beat both: first-party on the publisher's own Cloudflare zone, data in their own D1, SSR for SEO and speed, Workers/Turnstile/Workers-AI/Email at the edge. It can deliver Disqus's engagement loop with WordPress's data ownership and none of the tracking/perf/SEO baggage. The foundation already does part of this. This RFC is about finishing the job.
2. What EmDash already has (verified against source)
This framing matters: we are completing a system, not proposing a new one. Confirmed by reading the repo:
Data model —
_emdash_comments(packages/core/src/database/migrations/027_comments.ts):parent_id(ON DELETE CASCADE).author_name,author_email,author_url,author_user_id → users.id(ON DELETE SET NULL).status(defaultpending),ip_hash(privacy),user_agent,moderation_metadata(JSON).(collection, content_id, status),parent_id,(status, created_at),author_email,author_user_id.comments_enabled,comments_moderation(defaultfirst_time= auto-approve returning commenters — the Isso trust pattern),comments_auto_approve_users(default on),comments_closed_after_days(default 90).Rendering —
Comments.astro+CommentForm.astro(emdash/ui):<ol>/<article>withec--prefixed classes;threadedprop.Ingest + anti-spam —
POST /_emdash/api/comments/:collection/:contentId:X-EmDash-Request: 1CSRF header, honeypot field, IP-hash rate limiting, optional Cloudflare Turnstile (turnstileSiteKeyprop onCommentForm).Moderation pipeline —
packages/core/src/plugins/hooks.ts+comments/moderator.ts:comment:beforeCreate(mutate or reject), exclusivecomment:moderate(one provider returnsModerationDecision = { status: "approved" | "pending" | "spam", reason? }),comment:afterCreate(fire-and-forget, carriescontent+contentAuthor),comment:afterModerate.api/admin/comments/*).@emdash-cms/plugin-ai-moderationalready exists and claimscomment:moderateexclusively, running Cloudflare Workers AI (Llama Guard). The plugin extension model for comments is real and shipping.Notifications —
comments/notifications.ts: emails the content author when a comment is approved.This is roughly "good lightweight self-hosted tier (Isso/Remark42) + AI moderation." Solid. Not yet best-in-class.
3. The gap to best-in-class
From a survey of Disqus, Coral (Vox), Hyvor/Commento, Remark42/Cusdis/Isso, Giscus, WordPress, Substack/Ghost, and Reddit/HN/Discourse, "best" sorts into four layers. Legend: ✅ exists ·⚠️ partial · ❌ missing (all statuses verified against source).
Layer 1 — Friction-killers (table stakes)
escapeHtml+ auto-link URLs,white-space: pre-wrap)Layer 2 — The engagement flywheel (the actual goal: time-on-page + return visits)
Layer 3 — Trust & safety
plugin-ai-moderation)Layer 4 — Moonshots / differentiators
4. Core vs plugin — the architectural decision
The honest answer is hybrid, and the dividing line is precise and evidence-based.
Why some of this must be core (a plugin cannot build it today)
Verified plugin limits:
ctx.kv,_plugin_state) — not their own indexed D1 tables/migrations. Reactions/votes need an indexed relational table to sort-by-best at scale; that belongs in the core schema.afterCreatecan trigger mail, but the subscription table + unsubscribe tokens + identity are core concerns.page:fragments) or Portable-Text block components. It cannot ship a typed, prop-driven SSR-then-hydrate Astro island that auto-appears under an article. The "paste one line and a rich widget appears" delivery needs a small core extension point. (WidgetRenderer.astro'scomponentMapis also hardcoded to core widgets.)Why the rest should be a plugin (and already can be)
Verified plugin capabilities:
PluginRoute.public?: boolean— "Mark this route as publicly accessible (no authentication required). Public routes skip session/token auth and CSRF checks." So a reactions endpoint, import adapter, or webhook is plugin-doable.plugin-ai-moderation).page:fragments, Portable-Text block components,page:metadata(schema.orgComment/DiscussionJSON-LD).So: AI/spam moderation, analytics, badges, import-from-Disqus, Q&A-mode UI, membership gating, alternative themes → plugins.
5. Proposed roadmap (tiered, additive, independently shippable)
Each tier is a separate Discussion-able unit; ROI descends down the list. Data sketches follow EmDash conventions (text/ULID PKs, integer booleans,
idx_{table}_{col}, register migration inrunner.ts, never interpolate into SQL).Tier 1 — Reactions + Sort-by-Best (core; highest ROI, cleanest PR)
The single highest-ROI engagement primitive: one-tap participation for the silent majority, plus the signal needed to surface quality.
_emdash_comment_reactions(next migration042_*):id,comment_id → _emdash_comments.id (cascade),reaction(text; default setlike, extensible),voter_hash(IP+UA hash or visitor token, for dedupe),created_at; unique index on(comment_id, voter_hash, reaction); index oncomment_id.public: trueroutes, with the table the one piece that needs core.Bestsort; offerTop/Newalternates. Cheap to store/recompute per reaction.Comments.astro(reactions,sort) — default off → zero behavior change.Tier 2 — The retention loop: commenter notifications + lightweight visitor identity (core)
The mechanic every system in the survey ranks #1 for repeat visits — and the one EmDash is missing for commenters.
isRegisteredUser✓ badge)._emdash_comment_subscriptions(content_id,subscriber_emailor visitor token,scopethread|content, signedunsubscribe_token,created_at).comment:afterCreateevent (it already carriescontent+ thread context) to email subscribers via the existing email pipeline; honor per-subscriber unsubscribe. (Generalizesnotifications.tsbeyond author-only.)Tier 3 — Real-time + comment-aware caching (core)
Makes threads feel alive and fixes a correctness gap.
Astro.cache.set(cacheHint)) serves a stale comment list. Add a comment-aware cache boundary / hole-punch so new approved comments invalidate correctly.Tier 4 — Embed ergonomics: public-component registry / render-slots (core; cross-cutting)
A general extension point that unlocks one-line rich embeds for comments and benefits other plugins.
definePublicComponent/ named<EmDashSlot name="after-content" />mechanism, mirroring the existingvirtual:emdash/block-componentsgenerator with a newvirtual:emdash/public-componentsmodule collected from a new optional descriptor field (e.g.publicComponentsEntry).WidgetRenderer.astroto resolvecomponent-type widgets from plugin registrations instead of the hardcoded map.Tier 5 — Content quality: rich text + deeper threading (core)
Plugin lane — no core changes required
Buildable today on hooks +
public: trueroutes + KV: import-from-Disqus/WordPress, Q&A/AMA mode, membership-gated commenting, trust levels, advanced moderation (Banned-vs-Suspect word lists, shadowban, trust-weighted flagging), analytics widgets.6. Design constraints (the "do-not" list, drawn from the cautionary tales)
7. Backward compatibility & process
042+), new optional widget props, new optional descriptor fields, new opt-in routes. No change to existing migrations, the_emdash_commentsschema, the comment hook signatures, orComments.astro/CommentForm.astrodefaults.Approved for PRbefore any PR; each tier lands as its own PR with a changeset (minor), tests-first, nomessages.pochurn, AI disclosure on the PR.8. Open questions for maintainers
Prepared for the EmDash community. Repo claims verified against source at the time of writing; competitive analysis AI-assisted (
claude-opus-4-8).All reactions