-
Notifications
You must be signed in to change notification settings - Fork 283
Stratconn 6058/batch size #3106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
hi @nk1107 - please message me directly when you'd like me to review this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3106 +/- ##
==========================================
+ Coverage 79.81% 79.83% +0.01%
==========================================
Files 1165 1166 +1
Lines 21513 21533 +20
Branches 4154 4161 +7
==========================================
+ Hits 17171 17191 +20
Misses 3627 3627
Partials 715 715 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This reverts commit b4a2c43.
packages/destination-actions/src/destinations/customerio/customerio-properties.ts
Outdated
Show resolved
Hide resolved
Ah OK. that's the batch size for inbound API calls to Segment. You don't
need to worry about this.
The min and max batch sizes are you are setting relate to how many payloads
will be passed into the performBatch() function. This should be based off
of how many payloads the Customer.io API can handle.
So if Customer.io is able to handle up to 1000 events per batch, then we
would set the max to 1000, and maybe the default also. Can you find this
out please, and amend the PR?
Thanks,
Joe
…On Mon, Jul 28, 2025 at 10:10 AM nk1107 ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In
packages/destination-actions/src/destinations/customerio/customerio-properties.ts
<https://urldefense.com/v3/__https://github.com/segmentio/action-destinations/pull/3106*discussion_r2235478680__;Iw!!NCc8flgU!euG77E2jZKPFEHSfOjO0-Aql6zRRExuhYcaQ23f6jxFVGJgbIyaQc6pEL1r_lbqrNH_Oh7bgVIxkCadYk40YbRRvuQ$>
:
> + minimum: 2,
+ maximum: 15,
+ default: 10
I am referring to this documentation:
https://docs.customer.io/integrations/api/track/#operation/batch
<https://urldefense.com/v3/__https://docs.customer.io/integrations/api/track/*operation/batch__;Iw!!NCc8flgU!euG77E2jZKPFEHSfOjO0-Aql6zRRExuhYcaQ23f6jxFVGJgbIyaQc6pEL1r_lbqrNH_Oh7bgVIxkCadYk416xwAjsQ$>
It says : Your batch request must be smaller than 500kb. Each of the
requests within the batch must also be 32kb or smaller.
So for max_batch size i am considering the edge case where all the
requests in the batch are 32 kb which gives us 15 for the max_batch size.
I do am unsure if this works do let me know what more can i do to confirm
this number
—
Reply to this email directly, view it on GitHub
<https://urldefense.com/v3/__https://github.com/segmentio/action-destinations/pull/3106*discussion_r2235478680__;Iw!!NCc8flgU!euG77E2jZKPFEHSfOjO0-Aql6zRRExuhYcaQ23f6jxFVGJgbIyaQc6pEL1r_lbqrNH_Oh7bgVIxkCadYk40YbRRvuQ$>,
or unsubscribe
<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AK2F3MCD3HPLN6CPIVQU7Y33KXSHZAVCNFSM6AAAAACCI5CUI6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTANRRGQYTONZQGU__;!!NCc8flgU!euG77E2jZKPFEHSfOjO0-Aql6zRRExuhYcaQ23f6jxFVGJgbIyaQc6pEL1r_lbqrNH_Oh7bgVIxkCadYk40icb49hw$>
.
You are receiving this because you were assigned.Message ID:
***@***.***>
|
Hi @nk1107 please ping me on Slack or email when you'd like a re-review. |
packages/destination-actions/src/destinations/customerio/utils.ts
Outdated
Show resolved
Hide resolved
packages/destination-actions/src/destinations/customerio/utils.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please see comments.
hi @nk1107 PR deployed |
commit 349c8cb Author: Nick Aguilar <[email protected]> Date: Thu Jul 3 09:26:43 2025 -0700 Only append tags when isLiquidDirective is true commit b29a712 Author: Nick Aguilar <[email protected]> Date: Thu Jul 3 09:14:02 2025 -0700 Dry suggestion: uses finally block to call histogram commit c2d8704 Author: Nick Aguilar <[email protected]> Date: Mon Jun 30 16:55:09 2025 -0700 Reverts packages/browser-destinations/webpack.config.js commit 1964c15 Author: Nick Aguilar <[email protected]> Date: Mon Jun 30 16:45:22 2025 -0700 Removes perf_hooks usage since it proved to be a never-ending rabbithole of import errors commit 8850513 Author: Nick Aguilar <[email protected]> Date: Mon Jun 30 16:32:42 2025 -0700 Does not attempt to resolve the perf_hooks package as it's not browser supported commit 084b860 Author: Nick Aguilar <[email protected]> Date: Thu Jun 26 16:44:14 2025 -0700 Throws error only after logging out metrics commit 28b117c Author: Nick Aguilar <[email protected]> Date: Thu Jun 26 16:32:56 2025 -0700 Imports performance commit 5508da6 Author: Nick Aguilar <[email protected]> Date: Thu Jun 26 16:10:24 2025 -0700 Removes Subscription values which are undefined commit 856634a Author: Nick Aguilar <[email protected]> Date: Thu Jun 26 09:46:21 2025 -0700 Adds action and destination metadata values to tags commit 9ce22c5 Author: Nick Aguilar <[email protected]> Date: Thu Jun 26 11:14:39 2025 -0700 WIP - pipes through statsContext to evaluateLiquid directive function Tracks liquid metrics when batching as well WIP passes tags hopefully commit 6e6149c Author: Nick Aguilar <[email protected]> Date: Tue Jun 24 17:21:26 2025 -0700 WIP - pipes through statsContext to evaluateLiquid directive function commit b367ebd Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue Aug 12 17:13:24 2025 +0100 Publish (#3172) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit db16abc Author: Joe Ayoub <[email protected]> Date: Tue Aug 12 17:07:03 2025 +0100 register yonoma (#3170) commit 8a11fb1 Author: Vanessa Sun <[email protected]> Date: Tue Aug 12 23:54:11 2025 +0800 Stackadapt-Audience: add isPii field to upsertExternalAudienceMapping query (#3095) * add isPii field to upsertExternalAudienceMapping query * update snapshot for stackadapt-audiences * fix query input string format commit 7a17d8e Author: Joe Ayoub <[email protected]> Date: Tue Aug 12 16:53:39 2025 +0100 STRATCONN-6040 - [Yonomo] - New Destination (#3140) * First Action * Actions completed * testAuthentication * starting on tests * saving progress * Updated tests * all tests done * removing unnecesary tests * fixing timestamp in snapshots commit 0e0a155 Author: nk1107 <[email protected]> Date: Tue Aug 12 21:07:20 2025 +0530 Stratconn 6058/batch size (#3106) * add enable-batch and batch-size * change description * update test files * Revert "update test files" This reverts commit b4a2c43. * extract batch properties finally * update max limit * change label * Remove batching configuration fields from payload before sending to Customer.io * Remove batching configuration fields from BasePayload type commit b8d7e8c Author: Arijit Ray <[email protected]> Date: Tue Aug 12 21:06:19 2025 +0530 Dont throw 404 if profile is not found in intercom Ondelete (#3144) * Dont throw 404 if profile is not found in intercom Ondelete * fix tests commit 1924208 Author: Joe Ayoub <[email protected]> Date: Tue Aug 12 16:36:06 2025 +0100 [Attentive] - bugfix (#3151) * minor bugfix * fixing tests commit c6ed179 Author: Varadarajan V <[email protected]> Date: Tue Aug 12 21:05:48 2025 +0530 [STRATCONN-6023] - Add feature flag to facilitate GZIP Compression Rollout (#3152) * [STRATCONN-6023]- Add feature flag for rolling out compression * Revert batch size * add gzipcompress flag * Add tests
JIRA - https://twilio-engineering.atlassian.net/jira/software/c/projects/STRATCONN/boards/2317?assignee=712020%3Ae753d890-5047-4206-aaa4-9693b6ada1b0&assignee=712020%3A087e2fcd-dfdf-4343-a761-04f54a014b74&selectedIssue=STRATCONN-6058
Summary of the Ticket - Customers (Siemens) are running into issues with likely due to batch size being too large for Customer.io .
Change Requested - Add editable batch size input in the actions for Customer.io
This change adds
enable_batch
andbatch_size
properties in all the actions of actions-customerioTesting
With batch size greater than 100

With batch size less than 2
