Skip to content

fix: build sitemap.xml statically#18802

Open
myelinated-wackerow wants to merge 1 commit into
devfrom
fix/sitemap-static
Open

fix: build sitemap.xml statically#18802
myelinated-wackerow wants to merge 1 commit into
devfrom
fix/sitemap-static

Conversation

@myelinated-wackerow

Copy link
Copy Markdown
Collaborator

Summary

Part 1 of #18014. Makes /sitemap.xml build statically so it stops rendering in the Netlify serverless function, where public/content/ is excluded from the bundle.

Root cause

app/sitemap.ts -> getAllPagesWithTranslations() -> getDynamicIntlPagePaths() calls getAppsData() and getDeveloperToolsData(). Both are createCachedGetter wrappers with a finite revalidate (CACHE_REVALIDATE_DAY), which opts the route into ISR. Netlify then re-renders the sitemap in the serverless function, where public/content/ is stripped from the bundle via outputFileTracingExcludes (next.config.js). getPostSlugs("/") hits ENOENT, returns [], and the sitemap silently drops every markdown-driven page and truncates nondeterministically.

This is the same failure class already documented in docs/solutions/integration-issues/netlify-isr-404-async-server-components.md; the repo already ships getStaticAppsData (revalidate: false) as the sanctioned escape hatch for exactly this.

Changes

  • Add getStaticDeveloperToolsData (revalidate: false) mirroring the existing getStaticAppsData.
  • Point the sitemap's data reads at the static getters (getStaticAppsData / getStaticDeveloperToolsData), removing the ISR trigger.
  • export const dynamic = "force-static" on app/sitemap.ts, deliberately without revalidate (ISR would re-enter the broken lambda path; freshness rides the deploy cadence).
  • Guard getVideoSlugs() against ENOENT to match getPostSlugs, silencing the companion Grafana error.

Scope / what this does NOT fix

The sitemap is now complete but is still a single ~51 MB file (17.5k URLs x ~26 hreflang alternates), which is over Google's 50 MB per-file limit and rejected on size. Per-locale sharding via generateSitemaps() lands in a stacked follow-up PR and is required before the file is accepted. A CI regression guard (parse each shard, assert closing </urlset>, URL-count floor, sentinel late-alphabet URL) follows that.

Verification

  • pnpm type-check and pnpm lint pass.
  • The empty-slug failure is Netlify-lambda-only and does not reproduce locally (public/content/ is always present in dev), so the deploy preview is the real verification surface.

-- Opus 4.8

The sitemap route transitively calls getAppsData/getDeveloperToolsData, both wrapped with a finite revalidate, which opts the route into ISR. Netlify then re-renders it in the serverless function, where public/content is excluded from the bundle -- so getPostSlugs returns [] and the sitemap drops every markdown page and truncates nondeterministically.

Add a no-revalidate getStaticDeveloperToolsData (mirroring the existing getStaticAppsData), point the sitemap's data reads at the static getters, and set force-static on app/sitemap.ts so it is generated at build time where public/content is present. Also guard getVideoSlugs against ENOENT to match getPostSlugs.

Part 1 of #18014. The sitemap is now complete but still a single ~51MB file; per-locale sharding to get under Google's 50MB per-file limit follows in a stacked PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: wackerow <54227730+wackerow@users.noreply.github.com>
@netlify

netlify Bot commented Jul 14, 2026

Copy link
Copy Markdown

Deploy Preview for ethereumorg ready!

Name Link
🔨 Latest commit 3f9f090
🔍 Latest deploy log https://app.netlify.com/projects/ethereumorg/deploys/6a56b5912b9a8b000856b03a
😎 Deploy Preview https://deploy-preview-18802.ethereum.it
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
7 paths audited
Performance: 60 (🟢 up 4 from production)
Accessibility: 95 (no change from production)
Best Practices: 100 (no change from production)
SEO: 98 (no change from production)
PWA: 59 (🔴 down 1 from production)
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions github-actions Bot added the tooling 🔧 Changes related to tooling of the project label Jul 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔎 First-pass review — ✅ Looks mergeable

Surgical build fix: swaps the sitemap's transitive data reads to the revalidate: false static getters and pins app/sitemap.ts to force-static, so the route no longer opts into ISR and re-renders in the Netlify lambda where public/content/ is stripped. The new getStaticDeveloperToolsData mirrors the existing getStaticAppsData exactly (same createCachedGetter(..., false) shape, distinct *-static cache key), and the getVideoSlugs() ENOENT guard matches getPostSlugs line-for-line. This is the sanctioned escape hatch already documented for this failure class. Green on lint/type-check and unit; Chromatic + Netlify preview still running.

Analysis

Lane: code (build/data-caching infra) — tooling 🔧 label kept as the closest fit.

Checked:

  • getStaticDeveloperToolsData matches the getStaticAppsData pattern (src/lib/data/index.ts:64 vs :191): correct 3rd-arg false, unique cache key developer-tools-data-static. ✅
  • translationRegistry.ts dynamic-import destructure now pulls both static getters, both exported from @/lib/data. ✅
  • videos.ts ENOENT handler is faithful to getPostSlugs (src/lib/utils/md.ts:72) — warn + empty list on ENOENT, rethrow otherwise. ✅
  • No unused vars, no raw Intl.*, no chain names touched. Base dev correct.
  • force-static without revalidate is intentional and well-commented.

Non-blocking: As the description states, this only makes the sitemap complete — it's still a single ~51 MB file over Google's 50 MB limit until the generateSitemaps() sharding follow-up lands. In scope for this stacked PR-1; flag so the follow-up (and its CI regression guard) isn't forgotten.

Checks: Lint/type-check ✅, Unit ✅, Chromatic gate ✅; Visual regression + Netlify deploy in progress. The real verification surface (lambda empty-slug path) is the deploy preview, as noted.

Generated by PR Reviewer (team) for #18802 · 78.2 AIC · ⌖ 27.5 AIC · ⊞ 5.2K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tooling 🔧 Changes related to tooling of the project

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant