persistent job queue with postgraphile worker #7383
Conversation
Summary of ChangesHello @n1ru4l, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request establishes a new workflows service, designed to centralize and manage asynchronous background tasks within the application. By integrating graphile-worker, it provides a scalable and reliable mechanism for handling operations such as sending various email notifications (e.g., for user verification, organization invitations, password resets, and rate limit alerts) and performing critical database maintenance like purging expired schema checks. This service aims to improve application responsiveness and reliability by offloading these processes from the main application flow. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
📚 Storybook DeploymentThe latest changes are available as preview in: https://pr-7383.hive-storybook.pages.dev |
💻 Website PreviewThe latest changes are available as preview in: https://pr-7383.hive-landing-page.pages.dev |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new @hive/workflows package, which sets up a graphile-worker to handle background tasks. The implementation includes environment variable parsing, a task definition kit, and several task implementations for features like email notifications and data purging. The overall structure is sound, but I've identified a critical type error in the worker's context initialization, a deviation from the style guide regarding fetch usage, and a placeholder implementation that needs to be completed.
| const response = await fetch('https://api.postmarkapp.com/email', { | ||
| method: 'POST', | ||
| headers: { | ||
| 'Content-Type': 'application/json', | ||
| Accept: 'application/json', | ||
| 'X-Postmark-Server-Token': config.token, | ||
| }, | ||
| body: JSON.stringify({ | ||
| From: emailFrom, | ||
| To: email.to, | ||
| Subject: email.subject, | ||
| HtmlBody: email.body, | ||
| MessageStream: config.messageStream, | ||
| }), | ||
| }); |
There was a problem hiding this comment.
The style guide discourages direct use of fetch because it lacks built-in retries. The postmark email provider uses fetch to send emails, which could lead to failed email deliveries on transient network issues. Please consider using a more robust HTTP client library that supports retries, like got which is used elsewhere in the repository, or implement a retry mechanism.
References
- Avoid using
fetchornode-fetchdirectly, as it does not have built-in retries. (link)
n1ru4l
force-pushed
the
experiment-postgraphile-worker
branch
from
b74e09e to
43524f0
Compare
|
🐋 This PR was built and pushed to the following Docker images: Targets: Platforms: Image Tag: |
| }), | ||
| }) as HivePubSub; | ||
|
|
||
| let dbPurgeTaskRunner: null | ReturnType<typeof createTaskRunner> = null; |
There was a problem hiding this comment.
This is now a cron in the workflow service
n1ru4l
changed the title
1 participant
Background
If Redis dies we loose pending jobs
Description
Deployment Strategy
Alternatively, we could also move tasks from bull to postgres, but this is additional work that can be avoided.
Checklist