-
Notifications
You must be signed in to change notification settings - Fork 174
[backend] feat: batching injects traces (#2830) #3990
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
7cf31ee to
3dbb8b2
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## release/current #3990 +/- ##
=====================================================
+ Coverage 50.12% 50.39% +0.26%
- Complexity 3683 3721 +38
=====================================================
Files 907 912 +5
Lines 27041 27301 +260
Branches 2027 2046 +19
=====================================================
+ Hits 13554 13758 +204
- Misses 12668 12707 +39
- Partials 819 836 +17 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
31f799e to
d8c8da3
Compare
7374d53 to
cb09f38
Compare
cb09f38 to
b51061d
Compare
20364ba to
eab9776
Compare
damgouj
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
openaev-api/src/main/java/io/openaev/migration/V4_53__Convert_expectations_to_jsonb.java
Show resolved
Hide resolved
openaev-api/src/main/java/io/openaev/rest/helper/queue/BatchQueueService.java
Outdated
Show resolved
Hide resolved
openaev-api/src/main/java/io/openaev/rest/helper/queue/BatchQueueService.java
Outdated
Show resolved
Hide resolved
openaev-api/src/main/java/io/openaev/rest/helper/queue/BatchQueueService.java
Outdated
Show resolved
Hide resolved
openaev-api/src/main/java/io/openaev/rest/inject/service/InjectStatusService.java
Show resolved
Hide resolved
openaev-model/src/main/java/io/openaev/database/model/Agent.java
Outdated
Show resolved
Hide resolved
9e70eb1 to
de769e3
Compare
969949b to
8cfdb14
Compare



Proposed changes
Testing Instructions
Related issues
Checklist
Further comments
Benchmark
For our benchmark, we used a use case Salt Typhhon like meaning 18 injects having 1000 endpoints throwing each 4 execution trace. All these injects have an output parser. Do note that the endpoints for our tests are fake endpoints. They are just threads on my machine doing post requests with their execution trace.
In total, we are supposed to have 72 000 execution traces ingested.
Current perfs with existing method :
Only 870 traces are pushed in 10mins. The backend can't follow to deal with more and the app is unusable during the benchmark. On average, each trace is taking around 20s to be ingested so around 3 trace ingested by minutes.
Perfs with this PR :

Since we're now using RabbitMQ, we can see the graph of ingestion in it :
We can see that all the 72 000 traces are pushed in a minute. They are then ingested in around 6 minutes. The ingestion varies quite a lot but it's around 200 ingestion per second or 12 000 per minute.