Skip to content

Message byte estimation and batching on @fedify/cfworkers - #959

Closed
SJang1 wants to merge 2 commits into
fedify-dev:mainfrom
SJang1:patch-2
Closed

SJang1 wants to merge 2 commits into
fedify-dev:mainfrom
SJang1:patch-2

Conversation

@SJang1

@SJang1 SJang1 commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixed WorkersMessageQueue.enqueueMany() failing when the given messages exceeded Cloudflare Queues' batch limits of 100 messages or 256 KB per batch.

Assisted-by: chatgpt codex-5.6-sol

Changes

  • estimates the serialized size of each message
  • splits the messages into multiple sendBatch() calls

dahlia and others added 2 commits July 15, 2026 12:42
[ci skip]
Added message byte estimation and batching logic to handle message limits.

Assisted-By: codex-5.6-sol
Copilot AI review requested due to automatic review settings July 19, 2026 06:34
@netlify

netlify Bot commented Jul 19, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for fedify-json-schema ready!

Name Link
🔨 Latest commit b7322dc
🔍 Latest deploy log https://app.netlify.com/projects/fedify-json-schema/deploys/6a5c70079af4a50008f03ab0
😎 Deploy Preview https://deploy-preview-959--fedify-json-schema.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request bumps the version of several packages to 2.0.23 and updates the WorkersMessageQueue implementation in @fedify/cfworkers to batch messages in enqueueMany based on message count and estimated byte size limits. Feedback was provided regarding a redundant check for undefined on the result of JSON.stringify(body), as it is called on an object literal and will never return undefined.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +352 to +359
const serialized = JSON.stringify(body);

if (serialized === undefined) {
throw new TypeError("Queue message must be JSON-serializable.");
}

return utf8Encoder.encode(serialized).byteLength +
ESTIMATED_METADATA_BYTES_PER_MESSAGE;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since body is constructed as an object literal { __fedify_ordering_key__, __fedify_payload__ }, JSON.stringify(body) will never return undefined (it would return "{}" even if both properties were undefined). Therefore, the check for serialized === undefined is redundant and can be safely removed to simplify the code.

      const serialized = JSON.stringify(body);
      return utf8Encoder.encode(serialized).byteLength +
        ESTIMATED_METADATA_BYTES_PER_MESSAGE;

@coderabbitai

coderabbitai Bot commented Jul 19, 2026 •

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 08a3c74a-1fc9-46e4-a22a-8a5315893a4b

📥 Commits

Reviewing files that changed from the base of the PR and between 2265642 and b7322dc.

📒 Files selected for processing (54)
  • CHANGES.md
  • packages/amqp/deno.json
  • packages/amqp/package.json
  • packages/cfworkers/deno.json
  • packages/cfworkers/package.json
  • packages/cfworkers/src/mod.ts
  • packages/cli/deno.json
  • packages/cli/package.json
  • packages/create/package.json
  • packages/debugger/deno.json
  • packages/debugger/package.json
  • packages/denokv/deno.json
  • packages/elysia/deno.json
  • packages/elysia/package.json
  • packages/express/deno.json
  • packages/express/package.json
  • packages/fastify/deno.json
  • packages/fastify/package.json
  • packages/fedify/deno.json
  • packages/fedify/package.json
  • packages/fixture/deno.json
  • packages/fresh/deno.json
  • packages/h3/deno.json
  • packages/h3/package.json
  • packages/hono/deno.json
  • packages/hono/package.json
  • packages/init/deno.json
  • packages/init/package.json
  • packages/koa/deno.json
  • packages/koa/package.json
  • packages/lint/deno.json
  • packages/lint/package.json
  • packages/nestjs/package.json
  • packages/next/package.json
  • packages/postgres/deno.json
  • packages/postgres/package.json
  • packages/redis/deno.json
  • packages/redis/package.json
  • packages/relay/deno.json
  • packages/relay/package.json
  • packages/sqlite/deno.json
  • packages/sqlite/package.json
  • packages/sveltekit/deno.json
  • packages/sveltekit/package.json
  • packages/testing/deno.json
  • packages/testing/package.json
  • packages/vocab-runtime/deno.json
  • packages/vocab-runtime/package.json
  • packages/vocab-tools/deno.json
  • packages/vocab-tools/package.json
  • packages/vocab/deno.json
  • packages/vocab/package.json
  • packages/webfinger/deno.json
  • packages/webfinger/package.json

📝 Walkthrough

Walkthrough

Cloudflare Workers queue fan-out now splits messages into size- and count-limited batches. The changelog and package manifests are updated for version 2.0.23.

Changes

Cloudflare Queue batching

Layer / File(s) Summary
Batch construction and validation
packages/cfworkers/src/mod.ts
enqueueMany estimates serialized message sizes, flushes batches at Cloudflare limits, and throws for non-serializable messages.

Release metadata

Layer / File(s) Summary
Version 2.0.23 metadata
CHANGES.md, packages/*/deno.json, packages/*/package.json
Adds the Version 2.0.23 changelog placeholder and updates package versions from 2.0.22 to 2.0.23.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: component/mq, type/bug

Suggested reviewers: 2chanhaeng, dahlia

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes Cloudflare Workers Queues sendBatch() limit violations in @fedify/cfworkers by estimating message payload sizes and splitting enqueueMany() into multiple batches, addressing issue #958. This PR also bumps the monorepo package versions to 2.0.23 and introduces a new CHANGES.md section for that version.

Changes:

  • Add byte-size estimation + batching logic to WorkersMessageQueue.enqueueMany() to stay under Cloudflare Queues batch constraints.
  • Bump multiple packages’ versions from 2.0.22 to 2.0.23.
  • Add a new CHANGES.md section for version 2.0.23.

Reviewed changes

Copilot reviewed 54 out of 54 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
CHANGES.md Adds a new 2.0.23 section stub.
packages/amqp/deno.json Bumps version to 2.0.23.
packages/amqp/package.json Bumps version to 2.0.23.
packages/cfworkers/deno.json Bumps version to 2.0.23.
packages/cfworkers/package.json Bumps version to 2.0.23.
packages/cfworkers/src/mod.ts Implements size estimation + batch splitting for enqueueMany().
packages/cli/deno.json Bumps version to 2.0.23.
packages/cli/package.json Bumps version to 2.0.23.
packages/create/package.json Bumps version to 2.0.23.
packages/debugger/deno.json Bumps version to 2.0.23.
packages/debugger/package.json Bumps version to 2.0.23.
packages/denokv/deno.json Bumps version to 2.0.23.
packages/elysia/deno.json Bumps version to 2.0.23.
packages/elysia/package.json Bumps version to 2.0.23.
packages/express/deno.json Bumps version to 2.0.23.
packages/express/package.json Bumps version to 2.0.23.
packages/fastify/deno.json Bumps version to 2.0.23.
packages/fastify/package.json Bumps version to 2.0.23.
packages/fedify/deno.json Bumps version to 2.0.23.
packages/fedify/package.json Bumps version to 2.0.23.
packages/fixture/deno.json Bumps version to 2.0.23.
packages/fresh/deno.json Bumps version to 2.0.23.
packages/h3/deno.json Bumps version to 2.0.23.
packages/h3/package.json Bumps version to 2.0.23.
packages/hono/deno.json Bumps version to 2.0.23.
packages/hono/package.json Bumps version to 2.0.23.
packages/init/deno.json Bumps version to 2.0.23.
packages/init/package.json Bumps version to 2.0.23.
packages/koa/deno.json Bumps version to 2.0.23.
packages/koa/package.json Bumps version to 2.0.23.
packages/lint/deno.json Bumps version to 2.0.23.
packages/lint/package.json Bumps version to 2.0.23.
packages/nestjs/package.json Bumps version to 2.0.23.
packages/next/package.json Bumps version to 2.0.23.
packages/postgres/deno.json Bumps version to 2.0.23.
packages/postgres/package.json Bumps version to 2.0.23.
packages/redis/deno.json Bumps version to 2.0.23.
packages/redis/package.json Bumps version to 2.0.23.
packages/relay/deno.json Bumps version to 2.0.23.
packages/relay/package.json Bumps version to 2.0.23.
packages/sqlite/deno.json Bumps version to 2.0.23.
packages/sqlite/package.json Bumps version to 2.0.23.
packages/sveltekit/deno.json Bumps version to 2.0.23.
packages/sveltekit/package.json Bumps version to 2.0.23.
packages/testing/deno.json Bumps version to 2.0.23.
packages/testing/package.json Bumps version to 2.0.23.
packages/vocab/deno.json Bumps version to 2.0.23.
packages/vocab/package.json Bumps version to 2.0.23.
packages/vocab-runtime/deno.json Bumps version to 2.0.23.
packages/vocab-runtime/package.json Bumps version to 2.0.23.
packages/vocab-tools/deno.json Bumps version to 2.0.23.
packages/vocab-tools/package.json Bumps version to 2.0.23.
packages/webfinger/deno.json Bumps version to 2.0.23.
packages/webfinger/package.json Bumps version to 2.0.23.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +387 to +391
const messageBytes = estimateMessageBytes(body);
const exceedsBatchLimit = batch.length >= MAX_BATCH_MESSAGES ||
estimatedBatchBytes + messageBytes > MAX_ESTIMATED_BATCH_BYTES;

if (batch.length > 0 && exceedsBatchLimit) {
Comment thread CHANGES.md
Comment on lines +6 to +11
Version 2.0.23
--------------

To be released.


Comment on lines +351 to +360
const estimateMessageBytes = (body: WrappedMessage): number => {
const serialized = JSON.stringify(body);

if (serialized === undefined) {
throw new TypeError("Queue message must be JSON-serializable.");
}

return utf8Encoder.encode(serialized).byteLength +
ESTIMATED_METADATA_BYTES_PER_MESSAGE;
};
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cfworkers: WorkersMessageQueue.enqueueMany() exceeds Cloudflare Queues batch size limit

3 participants