Skip to content

perf: offload sync webhook dispatch to bullmq#223

Merged
yash-pouranik merged 2 commits into
geturbackend:mainfrom
Ayush4958:move-synchronous-webhook-dispatching-to-bullmq
May 28, 2026
Merged

perf: offload sync webhook dispatch to bullmq#223
yash-pouranik merged 2 commits into
geturbackend:mainfrom
Ayush4958:move-synchronous-webhook-dispatching-to-bullmq

Conversation

@Ayush4958
Copy link
Copy Markdown
Contributor

@Ayush4958 Ayush4958 commented May 28, 2026

🚀 Pull Request Description

I had resolve a critical bottleneck in public API where synchronous webhook resolution was blocking the main Node.js event loop.

Previously, dispatchWebhooks was executed directly at the tail end of CRUD mutations, which tied API response times to database queries and payload construction logic.

Now It decouple the webhook resolution from the incoming API request by utilizing our BullMQ cluster, ensuring that data routes immediately return an HTTP 2XX response after database mutations are completed.

Fixes #217

🛠️ Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📝 Documentation update
  • 🎨 UI/UX improvement (Frontend only)
  • ⚙️ Refactor / Chore

🧪 Testing & Validation

Backend Verification:

  • I have run npm test in the backend/ directory and all tests passed.
  • I have verified the API endpoints using Postman/Thunder Client.
  • New unit tests have been added (if applicable).

Frontend Verification:

  • I have run npm run lint in the frontend/ directory.
  • Verified the UI changes on different screen sizes (Responsive).
  • Checked for any console errors in the browser dev tools.

✅ Checklist

  • My code follows the code style of this project.
  • I have performed a self-review of my code.
  • I have commented my code, particularly in hard-to-understand areas.
  • My changes generate no new warnings or errors.
  • I have updated the documentation (README/Docs) accordingly.

Summary by CodeRabbit

  • Bug Fixes

    • Improved webhook delivery reliability for document operations (create, update, delete, recover) by transitioning to asynchronous queue-based processing.
  • Tests

    • Updated test cases to reflect webhook delivery mechanism changes.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 28, 2026

Warning

Review limit reached

@Ayush4958, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 37 minutes and 50 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3ee4dc4a-5d19-4774-b253-06f35e0d0c5d

📥 Commits

Reviewing files that changed from the base of the PR and between 8ffe42c and 84ac7a4.

📒 Files selected for processing (3)
  • apps/dashboard-api/src/__tests__/project.controller.softDelete.test.js
  • apps/public-api/src/__tests__/softDelete.test.js
  • packages/common/src/queues/webhookQueue.js
📝 Walkthrough

Walkthrough

The PR migrates webhook dispatching from synchronous calls to asynchronous queue jobs. Controllers enqueue lightweight trigger-webhook jobs via webhookQueue.add(), and the webhook queue worker handles them by invoking the synchronous dispatchWebhooks internally, decoupling API response latency from webhook execution.

Changes

Webhook Dispatch Async Queue Migration

Layer / File(s) Summary
Webhook Queue Worker Handler
packages/common/src/queues/webhookQueue.js
Adds trigger-webhook job handler inside the BullMQ worker processor: reads projectId, event, collection, and payload from job data, derives action from the event string, calls dispatchWebhooks with the document and fields, then returns early.
Public API Data Controller Queue Migration
apps/public-api/src/controllers/data.controller.js
Replaces dispatchWebhooks imports with webhookQueue, then updates insertData, updateSingleData, deleteSingleDoc, and recoverSingleDoc to enqueue trigger-webhook jobs with event names (document.inserted, document.updated, document.deleted, document.recovered) and document payloads instead of direct calls.
Public API Soft-Delete Test Updates
apps/public-api/src/__tests__/softDelete.test.js
Mocks webhookQueue.add in @urbackend/common Jest setup and updates recoverSingleDoc test to assert trigger-webhook job enqueue with event: 'document.recovered' and removeOnComplete: true options.
Dashboard API Project Controller Queue Migration
apps/dashboard-api/src/controllers/project.controller.js
Replaces dispatchWebhooks imports with webhookQueue and updates recoverRow to enqueue trigger-webhook jobs with event: 'document.recovered' and the recovered document payload.
Dashboard API Project Controller Test Updates
apps/dashboard-api/src/__tests__/project.controller.softDelete.test.js
Mocks webhookQueue.add in @urbackend/common Jest setup and updates recoverRow test to assert trigger-webhook job enqueue with event: 'document.recovered' and removeOnComplete: true options.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

  • geturbackend/urBackend#203: Updates soft-delete recovery flows (recoverRow and recoverSingleDoc) to enqueue queued webhook jobs instead of synchronous calls, directly affecting the same recovery controller and test logic.

Suggested labels

feature, type:feature, level:critical

Suggested reviewers

  • yash-pouranik

Poem

🐰 Webhooks once blocked, now queue so free,
Jobs enqueued with Redis spree,
APIs respond without delay,
Workers dispatch throughout the day,
Async wheels spin, the throughput sings! 🚀

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: offloading synchronous webhook dispatch to BullMQ for performance improvement, which aligns with the core objective of the changeset.
Linked Issues check ✅ Passed The PR successfully implements all coding requirements from issue #217: replaces synchronous dispatchWebhooks calls with async webhookQueue.add('trigger-webhook', ...) in CRUD controllers, injects webhookQueue into execution context, and updates workers to handle queued jobs.
Out of Scope Changes check ✅ Passed All changes are directly aligned with issue #217 requirements—webhook dispatch refactoring in controllers and workers—with no extraneous modifications detected outside this scope.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/dashboard-api/src/__tests__/project.controller.softDelete.test.js`:
- Around line 164-172: The webhook job assertion in the soft-delete recovery
test is missing the collection field; update the expectation on
mockWebhookQueueAdd (the expect.objectContaining block used when asserting the
'trigger-webhook' call) to include collection: 'posts' alongside event:
'document.recovered', payload: restoredDoc, and projectId: 'proj_1' so the job
data fully matches the webhook payload shape.

In `@apps/public-api/src/__tests__/softDelete.test.js`:
- Around line 141-150: The test assertion for webhookQueue.add is missing
verification of the collection field; update the expect.objectContaining passed
to webhookQueue.add (the call with 'trigger-webhook' and restoredDoc) to include
collection: 'posts' so the job data assertion checks event, payload, projectId
and collection (i.e., add collection: 'posts' to the object passed to
expect.objectContaining in the test).

In `@packages/common/src/queues/webhookQueue.js`:
- Around line 85-102: Wrap the "trigger-webhook" branch (where job.name ===
"trigger-webhook") in a try/catch and call dispatchWebhooks inside the try; on
catch, log a detailed error containing projectId, collection, action, and
documentId using the existing logger in scope (e.g., processLogger or logger)
and then rethrow or fail the job so queue retry behavior is preserved; ensure
you reference dispatchWebhooks and the job.name check when applying the change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fdf2da9a-6ee4-404e-81cd-7d7dcdcd9808

📥 Commits

Reviewing files that changed from the base of the PR and between 75c1e5e and 8ffe42c.

📒 Files selected for processing (5)
  • apps/dashboard-api/src/__tests__/project.controller.softDelete.test.js
  • apps/dashboard-api/src/controllers/project.controller.js
  • apps/public-api/src/__tests__/softDelete.test.js
  • apps/public-api/src/controllers/data.controller.js
  • packages/common/src/queues/webhookQueue.js

Comment thread apps/public-api/src/__tests__/softDelete.test.js
Comment thread packages/common/src/queues/webhookQueue.js
@yash-pouranik
Copy link
Copy Markdown
Collaborator

@coderabbitai
Please check if its mergable?
I think it is.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 28, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
echo "=== PR Status ==="
gh pr view 223 --repo geturbackend/urBackend --json title,state,mergeable,mergeStateStatus,reviewDecision,statusCheckRollup,reviews,comments,isDraft

echo ""
echo "=== PR Checks ==="
gh pr checks 223 --repo geturbackend/urBackend

echo ""
echo "=== Open Review Comments ==="
gh api repos/geturbackend/urBackend/pulls/223/comments --jq '.[] | {id: .id, body: .body, resolved: .position, user: .user.login}' 2>/dev/null | head -100

echo ""
echo "=== PR Reviews ==="
gh api repos/geturbackend/urBackend/pulls/223/reviews --jq '.[] | {user: .user.login, state: .state, submitted_at: .submitted_at}' 2>/dev/null

Length of output: 36314



@yash-pouranik Yes, this PR looks good to merge! Here's a quick summary of its current state:

Signal Status
Merge conflicts ✅ None (branch is clean)
Review approval ✅ Approved
CI checks (8/8) ✅ All passing
CodeRabbit pre-merge checks (5/5) ✅ All passing
Open review comments ✅ All 3 addressed in commit 84ac7a4
Draft ✅ Not a draft

Everything is in order — no blockers. You're clear to merge. 🐇

@yash-pouranik yash-pouranik merged commit 76816a7 into geturbackend:main May 28, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance Optimization: Move Synchronous Webhook Dispatching to BullMQ

2 participants