Traffic-based App Rewards: Compute aggregateActivityTotals, add ComputeAppRewardTrigger, expose new endpoints by adetokunbo · Pull Request #4420 · hyperledger-labs/splice

adetokunbo · 2026-03-16T09:08:54Z

@meiersi-da: I replaced NoOpActivityComputation with FakeAppActivityComputation in 41b32f1. The Fake will be replaced on this feature branch once its rebased on the nearly-done TCS changes

Pull Request Checklist

Cluster Testing

If a cluster test is required, comment /cluster_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.
If a hard-migration test is required (from the latest release), comment /hdm_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.

PR Guidelines

Include any change that might be observable by our partners or affect their deployment in the release notes.
Specify fixed issues with Fixes #n, and mention issues worked on using #n
Include a screenshot for frontend-related PRs - see README or use your favorite screenshot tool

Merge Guidelines

Make the git commit message look sensible when squash-merging on GitHub (most likely: just copy your PR description).

meiersi-da

Thanks a lot. Sorry for the delay in the review.

There seems to have been a misunderstanding wrt featured app parties lookup. See my comments. Let's make another pass once the current comments have been addressed.

apps/scan/src/main/openapi/scan.yaml

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/admin/http/HttpScanHandler.scala

...can/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanAppRewardsStore.scala

...rc/main/scala/org/lfdecentralizedtrust/splice/scan/automation/RewardComputationTrigger.scala

...can/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanAppRewardsStore.scala

adetokunbo · 2026-03-18T11:12:52Z

@meiersi-da PTAL, I've addressed the current set of comments

meiersi-da

Thanks a lot. We are getting closer. The code continues to require a high attention to detail. Not an easy one to write or review ;)

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/ScanAppRewardsStore.scala

...rc/main/scala/org/lfdecentralizedtrust/splice/scan/automation/RewardComputationTrigger.scala

meiersi-da · 2026-03-18T14:20:17Z

...rc/main/scala/org/lfdecentralizedtrust/splice/scan/automation/RewardComputationTrigger.scala

+      tasks <- (lastClosedO, earliestCompleteO) match {
+        case (Some((lastClosed, _)), Some(earliestComplete)) =>
+          appRewardsStore
+            .getNextRoundWithoutRootHash(updateHistory.historyId, lastClosed)


This feels off as a function. It seems to do too much.

Why not query the latest round for which the computation completed; and also query the latest round with complete data. Then the tasks available are all the round from max(earliestComplete, latestRoundWithRootHash + 1) to min(lastClosed, latestComplete) (with some adjustments for the optionals).

You can then just pick the first 4 round numbers from this interval.

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/admin/http/HttpScanHandler.scala

...alizedtrust/splice/integration/tests/ScanAppRewardsComputationTimeBasedIntegrationTest.scala

meiersi-da · 2026-03-18T14:48:17Z

...alizedtrust/splice/integration/tests/ScanAppRewardsComputationTimeBasedIntegrationTest.scala

+    "return activity totals after aggregation trigger runs" in { implicit _env =>
+      // Advance enough rounds so that at least one round is fully closed and aggregated.
+      // Round 0 requires multiple ticks to close.
+      for (_ <- 1 to 7) {


Why do you need 7 rounds?

I'd expect that closing rounds 0 and 1 should be sufficient; i.e, you need to advance rounds two times.

Changed the comment, that function increments by ticks not rounds.

...alizedtrust/splice/integration/tests/ScanAppRewardsComputationTimeBasedIntegrationTest.scala

adetokunbo · 2026-03-19T07:30:34Z

@meiersi-da: PTAL, most comments are addressed. I plan at least one more change; #4322 is now merged, so I will rebase on main and implement TODOs that can be now be resolved

meiersi-da

Thanks a lot. Please review the bugs and address them with tests that exhibit them before fixing them.

It's OK to not build a test for rewards being computed sequentially.

meiersi-da · 2026-03-19T07:54:17Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/AppActivityStore.scala

+    * A round is complete if the prior round also has activity records,
+    * proving ingestion was running continuously through it.


apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/AppActivityStore.scala

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/ScanAppRewardsStore.scala

meiersi-da · 2026-03-19T08:01:13Z

.../src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbAppActivityRecordStore.scala

+            from #${Tables.appActivityRecords} aar
+            where exists(
+              select 1 from #${Tables.appActivityRecords}
+              where round_number = aar.round_number - 1


Check the query plan for this. This might be expensive to do that way.

If you do want to check the existence, then first query for the max and do a join against that single value, instead of potentially forcing an expensive join.

something along the lines of

select max_round from (select max(round) - 1 from app_activity_records where history_id = $historyId ) where exists ( select 1 from app_activity_records where history_id = $history_id and round = max_round )

also: please add tests that surface the problem wrt missing check of history_id, and check all your queries wrt the missing comparison of history_id

Fixed; added, verifying with store tests that would have otherwise failed due to the missing historyId

meiersi-da · 2026-03-19T08:47:55Z

...rc/main/scala/org/lfdecentralizedtrust/splice/scan/automation/RewardComputationTrigger.scala

+  override protected def isStaleTask(
+      task: RewardComputationTrigger.Task
+  )(implicit tc: TraceContext): Future[Boolean] =
+    Future.successful(false)


It is stale if the latest completed round is larger than the task round. Should not happen, but seems like the better definition.

Implemented this.

meiersi-da · 2026-03-19T08:53:30Z

...rc/main/scala/org/lfdecentralizedtrust/splice/scan/automation/RewardComputationTrigger.scala

+      lastClosedO <- store.lookupRoundOfLatestData()
+      earliestCompleteO <- appActivityStore.earliestRoundWithCompleteAppActivity()
+      latestCompleteO <- appActivityStore.latestRoundWithCompleteAppActivity()
+      latestComputedO <- appRewardsStore.lookupLatestRoundWithRewardComputation()


This actually turns out to be wrong when the trigger executes tasks in parallel and the backend gets restarted: round 2 might complete before round 1, and thus round 1 will never be completed.

So what I'd suggest to do for now is:

return at most one task and thus ensure that they get solved one after the other

create a TODO for an issue to compute tasks in parallel, which will require introducing a watermark table to remember the round number up to which all rewards have been computed. We might not need that. It will depend on how fast catchup works for the round trigger.

FYI: @rautenrieth-da -- catchup speed of reward computation is something to add to the test plan

Done. Opened #4570

...rc/main/scala/org/lfdecentralizedtrust/splice/scan/automation/RewardComputationTrigger.scala

meiersi-da · 2026-03-19T09:16:06Z

...can/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanAppRewardsStore.scala

+  def computeRewards(
+      roundNumber: Long
+  )(implicit tc: TraceContext): Future[Unit] =
+    aggregateActivityTotals(roundNumber)


prudent engineering: add safety check that all activity records have been ingested for the round in question. Better safe than sorry.

Fixed: added an assertion; verified with a store test

meiersi-da · 2026-03-19T09:19:16Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/admin/http/HttpScanHandler.scala

+  ] = {
+    implicit val tc = extracted
+    withSpan(s"$workflowId.getRewardAccountingEarliestAvailableRound") { _ => _ =>
+      appRewardsStore.getEarliestRoundWithActivityTotals().map {


AFAIR, the goal of the HTTP function is to enable other SVs to determine whether the earliest round for which this SV will eventually have the activity totals available. So we should return the earliest round for which there is a complete set of activity records and not the earliest one for which activity totals have been computed.

I suspect we actually have no need to retrieve the earliest round with activity totals, so we can remove that function from the ScanAppRewardsStore.

adetokunbo · 2026-03-23T10:38:25Z

@meiersi-da PTAL.

FYI: In 69a15c3, ScanStore.lookupRoundOfLatestData is removed as it is not not necessary due to the way the synchronization logic currently works; it is not replaced with any equivalent from ScanRewardsReferenceStore as there is nothing it uses from that store at the moment.

In the next PR of #4384 it will need obtain data from the appropriate OpenMiningRound, and this w

…tory_for_hash Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Co-authored-by: Simon Meier <simon@digitalasset.com> Signed-off-by: Timothy Emiola <adetokunbo@users.noreply.github.com>

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Co-authored-by: Simon Meier <simon@digitalasset.com> Signed-off-by: Timothy Emiola <adetokunbo@users.noreply.github.com>

- change the condition so that two consecutive rounds have to have activity Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Co-authored-by: Simon Meier <simon@digitalasset.com> Signed-off-by: Timothy Emiola <adetokunbo@users.noreply.github.com>

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

- use the historyId and add unittests that verify historyId isolation - use a more efficient query, uses the index better Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

…ctivity records Signed-off-by: Tim Emiola <adetokunbo@emio.la>

meiersi-da

Thanks a lot. It looks like only some small polish is required.

meiersi-da · 2026-03-23T16:51:32Z

...alizedtrust/splice/integration/tests/ScanAppRewardsComputationTimeBasedIntegrationTest.scala

+    }
+
+    "return activity totals after aggregation trigger runs" in { implicit _env =>
+      // TODO(#4118): Update the time management here


This issue is closed: #4118

Seems like the wrong reference.

meiersi-da · 2026-03-23T16:52:25Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/admin/http/HttpScanHandler.scala

+        case None =>
+          ScanResource.GetRewardAccountingActivityTotalsResponse.NotFound(
+            ErrorResponse(
+              s"Activity totals not yet computed for round $roundNumber"


Suggested change

s"Activity totals not yet computed for round $roundNumber"

s"Activity totals not (yet) computed for round $roundNumber"

it might be never

meiersi-da · 2026-03-23T16:52:44Z

...rc/main/scala/org/lfdecentralizedtrust/splice/scan/automation/RewardComputationTrigger.scala

+
+/** Trigger that drives the CIP-0104 reward computation pipeline via
+  * ScanAppRewardsStore.computeRewards, which will eventually run three
+  * idempotent steps in one transaction:


Suggested change

* idempotent steps in one transaction:

* computation steps in one transaction:

meiersi-da · 2026-03-23T16:55:42Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/AppActivityStore.scala

+  ): Future[Option[Long]]
+
+  /** Find the latest round for which all app activity records have been ingested.
+    * Returns None if fewer than two consecutive rounds have been ingested.


Suggested change

* Returns None if fewer than two consecutive rounds have been ingested.

Let's not leak implementation details on the API. We don't want callers to rely on it, and break when the implementation changes.

meiersi-da · 2026-03-23T16:57:04Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/ScanAppRewardsStore.scala

+    * MUST only be called on rounds for which all app activity records have
+    * been ingested and for which the reward information has not yet been computed.
+    */
+  def computeRewards(roundNumber: Long)(implicit


Suggested change

def computeRewards(roundNumber: Long)(implicit

def computeAndStoreRewards(roundNumber: Long)(implicit

meiersi-da · 2026-03-23T17:01:08Z

apps/scan/src/main/openapi/scan.yaml

+      required:
+        - round_number
+        - total_app_activity_weight
+        - active_parties_count


@rautenrieth-da : wdyt about also exposing the number of activity records in the round? Seems like very useful debugging info.

meiersi-da · 2026-03-23T17:04:47Z

...can/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanAppRewardsStore.scala

+  def computeRewards(
+      roundNumber: Long
+  )(implicit tc: TraceContext): Future[Unit] =
+    aggregateActivityTotals(roundNumber)


@rautenrieth-da : wdyt about adding counters for:

number of parties with activity

number of activity records summarized

number of parties with rewards

number of batches created

tagged with synchronizer-id and migration-id.

It seems that having dashboards for these would seem valuable to observe the proper working of reward computation. However, I'm lacking the ops experience to judge their usefulness.

@adetokunbo : If we do so, then I'd still do that in a separate PR. Just create an issue from this comment.

meiersi-da · 2026-03-23T17:07:21Z

...can/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanAppRewardsStore.scala

+    } yield ()
+
+  /** Unnest per-verdict activity arrays and aggregate weights by party. */
+  private def unnestAndAggregate(historyId: Long, roundNumber: Long) =


@rautenrieth-da : could I ask you to review these DB queries from a maintainers perspective? They look fine to me.

adetokunbo added this to the Traffic-Based App Rewards milestone Mar 16, 2026

adetokunbo requested review from dfordivam, meiersi-da and rautenrieth-da March 16, 2026 09:08

adetokunbo self-assigned this Mar 16, 2026

adetokunbo linked an issue Mar 16, 2026 that may be closed by this pull request

Scan app: Traffic-based App Rewards: compute aggregateActivityTotals; add access via a new endpoint #4380

Open

meiersi-da requested changes Mar 17, 2026

View reviewed changes

adetokunbo changed the title ~~Compute aggregateActivityTotals, add ComputeAppRewardTrigger, expose new endpoints~~ Traffic-based App Rewards: Compute aggregateActivityTotals, add ComputeAppRewardTrigger, expose new endpoints Mar 18, 2026

adetokunbo force-pushed the adetokunbo/cip-104-add-app-reward-computation-trigger branch 2 times, most recently from cac78b8 to bc955b6 Compare March 18, 2026 11:09

adetokunbo requested a review from meiersi-da March 18, 2026 11:15

meiersi-da reviewed Mar 18, 2026

View reviewed changes

adetokunbo force-pushed the adetokunbo/cip-104-add-app-reward-computation-trigger branch from 066c3c1 to 6c01e33 Compare March 19, 2026 06:10

adetokunbo requested a review from meiersi-da March 19, 2026 07:28

meiersi-da requested changes Mar 19, 2026

View reviewed changes

adetokunbo force-pushed the adetokunbo/cip-104-add-app-reward-computation-trigger branch from 751ea69 to 8f12e5d Compare March 19, 2026 11:22

adetokunbo changed the base branch from feature-cip-104-scan-computation to feature-cip-104-scan-computation-2 March 19, 2026 11:23

adetokunbo changed the base branch from feature-cip-104-scan-computation-2 to adetokunbo/feature-cip-104-scan-computation March 23, 2026 03:05

adetokunbo mentioned this pull request Mar 23, 2026

RewardComputationTrigger: Parallei task computation #4570

Open

adetokunbo requested a review from meiersi-da March 23, 2026 10:31

adetokunbo force-pushed the adetokunbo/feature-cip-104-scan-computation branch from 1369090 to 0effc2c Compare March 23, 2026 10:48

adetokunbo force-pushed the adetokunbo/cip-104-add-app-reward-computation-trigger branch 2 times, most recently from cad918f to 2ee47df Compare March 23, 2026 12:24

adetokunbo force-pushed the adetokunbo/feature-cip-104-scan-computation branch from 0effc2c to 79c3627 Compare March 23, 2026 12:46

adetokunbo added 3 commits March 23, 2026 12:57

Rename V062 migration to V064 to avoid conflict with V062__update_his…

0c41b46

…tory_for_hash Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Add aggregation and query methods to DbScanAppRewardsStore

9f415db

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Add RewardComputationTrigger

fead9b0

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

adetokunbo and others added 28 commits March 23, 2026 12:58

Add a TODO comment where lookupRoundOfLastData is used

0096b50

Co-authored-by: Simon Meier <simon@digitalasset.com> Signed-off-by: Timothy Emiola <adetokunbo@users.noreply.github.com>

Remove historyId from the methods of ScanAppRewardsStore

8f4777a

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Use the updateHistoryId.historyId in DbScanAppRewardsStore methods

a50539f

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Use .map instead result/Future.xxx

9fc496e

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

[ci] Correct the trigger's comment

3f971c1

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Correct time advancement in the integration test and add a TODO

393ab64

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Make the totals test more precise

9459cb5

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Update description of getRewardAccountingEarliestAvailableRound

2499f9d

Co-authored-by: Simon Meier <simon@digitalasset.com> Signed-off-by: Timothy Emiola <adetokunbo@users.noreply.github.com>

Fix AppActivityStore.earliestRoundWithCompleteAppActivity

80c2dcf

- change the condition so that two consecutive rounds have to have activity Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Add earliestRound lower bound to getNextRoundWithoutRootHash

9206a20

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Correct the earliest available round check

9a0d9c6

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

s/getEarliestActivityRound/getEarliestRoundWithActivityTotals/

89b9b9f

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

[static] Refactor RewardComputationTrigger to range-based task retrieval

806af72

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

[ci] Insert with RETURNING

be8166f

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Remove scaladocs that leak implementation details

18e4e4c

Co-authored-by: Simon Meier <simon@digitalasset.com> Signed-off-by: Timothy Emiola <adetokunbo@users.noreply.github.com>

Improve 'with complete'

1148777

Co-authored-by: Simon Meier <simon@digitalasset.com> Signed-off-by: Timothy Emiola <adetokunbo@users.noreply.github.com>

Remove the TODO 4384 scaladoc

8a6e1c1

Co-authored-by: Simon Meier <simon@digitalasset.com> Signed-off-by: Timothy Emiola <adetokunbo@users.noreply.github.com>

Clarifies use computeRewards

5b28bd2

Co-authored-by: Simon Meier <simon@digitalasset.com> Signed-off-by: Timothy Emiola <adetokunbo@users.noreply.github.com>

Improve 'with complete'

6ecc2fb

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Fixes bug with missing history_id in a constraint; verifying unittest

5ff2327

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Remove scaladocs that leak implementation details

07866f8

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Replace incorrect parallelization; add a TODO for a fix

5b33cf3

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Fix and improve queries

e08548d

- use the historyId and add unittests that verify historyId isolation - use a more efficient query, uses the index better Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Stop using ScanStore.lookupRoundOfLatestData for synchronization

8277754

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Add a safety check that all activity records have been ingested

3558ee9

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Define isStaleTask

3be11f6

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

Update descriptions to indicate some apis are SV node internal

bf6b5fd

Signed-off-by: Tim Emiola <adetokunbo@emio.la>

[ci] Report the earliest round for which there is a complete set of a…

94e18ca

…ctivity records Signed-off-by: Tim Emiola <adetokunbo@emio.la>

adetokunbo force-pushed the adetokunbo/cip-104-add-app-reward-computation-trigger branch from 2ee47df to 94e18ca Compare March 23, 2026 13:02

meiersi-da reviewed Mar 23, 2026

View reviewed changes

		* A round is complete if the prior round also has activity records,
		* proving ingestion was running continuously through it.

	s"Activity totals not yet computed for round $roundNumber"
	s"Activity totals not (yet) computed for round $roundNumber"

	* idempotent steps in one transaction:
	* computation steps in one transaction:

	def computeRewards(roundNumber: Long)(implicit
	def computeAndStoreRewards(roundNumber: Long)(implicit

Conversation

adetokunbo commented Mar 16, 2026

Pull Request Checklist

Cluster Testing

PR Guidelines

Merge Guidelines

Uh oh!

meiersi-da left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adetokunbo commented Mar 18, 2026

Uh oh!

meiersi-da left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adetokunbo Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

adetokunbo commented Mar 19, 2026

Uh oh!

meiersi-da left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adetokunbo Mar 19, 2026 •

edited

Loading