feat: Concurrent executions with indexedDB #1235

Microchesst · 2025-05-01T20:56:16Z

Pull Request Template

Title

Implement Concurrent Digital Twin Execution with IndexedDB Integration

Type of Change

Description

This PR implements concurrent execution for Digital Twins with IndexedDB integration. Users can now start multiple executions of the same Digital Twin simultaneously, with execution history persisted in the browser's IndexedDB. The implementation includes a new execution history data model, IndexedDB service, and UI components to display and manage execution history.

##Working on Testing

Added unit tests for the IndexedDB service and Redux store
Manually tested concurrent execution functionality
Verified persistence of execution history across browser sessions
Fixed existing tests to accommodate the new functionality

Impact

Improves user experience by allowing multiple executions to run in parallel
Adds local persistence of execution history using IndexedDB
Enhances the UI with execution history list and status indicators

Checklist

My code adheres to the coding and style guidelines of the project.
I have added tests for all the new code and any changes made to existing code.
I have made corresponding changes to the documentation.

Microchesst · 2025-05-01T21:07:08Z

I know the test are not working currently. Its just to show the feature, i will work on the test. I am having trouble with getting test of redux to work 😕

prasadtalasila · 2025-05-02T05:49:30Z

@Microchesst thanks for the PR. I will get back to you by Monday.

prasadtalasila

@Microchesst Thanks for the PR. The feature looks really nice. Please see the comments.

client/package.json

prasadtalasila · 2025-05-02T09:41:53Z

client/src/preview/route/digitaltwins/execute/pipelineChecks.ts

please move all the *.ts files in route/digitaltwins/execute to model/backend/gitlab/execution directory. Do remember to

create an interface for each class so that testing becomes easier. The interfaces related to pipelines can be kept in model/backend/gitlab/execution/interfaces.ts

always have preview/ code depend on stable codebase rather than other way around

client/package.json

client/src/preview/route/digitaltwins/execute/LogDialog.tsx

client/src/preview/services/indexedDBService.ts

client/src/preview/store/digitalTwin.slice.ts

prasadtalasila · 2025-05-02T10:39:40Z

client/src/preview/store/executionHistory.slice.ts

Do think about the appropriate location for these slices. Is it src/store or src/model/store?

codecov · 2025-05-15T09:00:24Z

Codecov Report

Attention: Patch coverage is 76.21832% with 244 lines in your changes missing coverage. Please review.

Project coverage is 86.61%. Comparing base (664266b) to head (1c0f14c).
Report is 1 commits behind head on feature/distributed-demo.

Files with missing lines	Patch %	Lines
...c/model/backend/gitlab/execution/statusChecking.ts	16.07%	47 Missing ⚠️
.../backend/gitlab/services/ExecutionStatusService.ts	13.15%	33 Missing ⚠️
.../src/model/backend/gitlab/execution/logFetching.ts	58.66%	31 Missing ⚠️
...del/backend/gitlab/state/executionHistory.slice.ts	77.58%	26 Missing ⚠️
client/src/database/digitalTwins.ts	84.00%	20 Missing ⚠️
...e/digitaltwins/execution/executionStatusManager.ts	77.21%	18 Missing ⚠️
...oute/digitaltwins/execution/executionUIHandlers.ts	71.18%	17 Missing ⚠️
client/src/preview/util/digitalTwin.ts	82.25%	11 Missing ⚠️
client/src/preview/util/init.ts	16.66%	10 Missing ⚠️
.../src/components/execution/ExecutionHistoryList.tsx	93.69%	7 Missing ⚠️
... and 9 more

Additional details and impacted files

@@                     Coverage Diff                      @@
##           feature/distributed-demo    #1235      +/-   ##
============================================================
- Coverage                     92.08%   86.61%   -5.48%     
============================================================
  Files                            92      106      +14     
  Lines                          2526     3346     +820     
  Branches                        406      560     +154     
============================================================
+ Hits                           2326     2898     +572     
- Misses                          197      445     +248     
  Partials                          3        3

Files with missing lines	Coverage Δ
client/src/components/asset/HistoryButton.tsx	`100.00% <100.00%> (ø)`
client/src/database/types.ts	`100.00% <100.00%> (ø)`
client/src/model/backend/gitlab/constants.ts	`100.00% <100.00%> (ø)`
...rc/model/backend/gitlab/state/digitalTwin.slice.ts	`100.00% <ø> (ø)`
...src/model/backend/gitlab/types/executionHistory.ts	`100.00% <100.00%> (ø)`
client/src/preview/components/asset/AssetBoard.tsx	`97.82% <100.00%> (ø)`
client/src/preview/components/asset/AssetCard.tsx	`88.70% <100.00%> (ø)`
...lient/src/preview/components/asset/StartButton.tsx	`100.00% <100.00%> (ø)`
...eview/route/digitaltwins/create/CreateDTDialog.tsx	`100.00% <100.00%> (ø)`
...c/preview/route/digitaltwins/execute/LogDialog.tsx	`100.00% <100.00%> (ø)`
... and 22 more

... and 3 files with indirect coverage changes

Components	Coverage Δ
Website	`86.61% <76.21%> (-5.48%)`	⬇️
Lib Microservice	`∅ <ø> (∅)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

prasadtalasila · 2025-05-15T09:52:46Z

@Microchesst please move your pure typescript files (*.ts) to model/backend/gitlab/execution.

prasadtalasila · 2025-05-15T09:55:23Z

The redux slices can be placed on model/backend/gitlab/state and import the same into the aggregation modules in src/store.

client/package.json

…sting.

Copilot

Pull Request Overview

This PR adds concurrent execution support for Digital Twins by persisting and managing execution history in IndexedDB and enhancing the UI to display and control multiple runs.

Migrate digital-twin state and actions to model/backend/gitlab slices and introduce IndexedDB schema/service.
Introduce ExecutionHistoryLoader for initial load and polling, plus ExecutionHistoryList, updated dialogs/buttons to surface history.
Extend pipeline handlers (pipelineUtils, pipelineHandler, pipelineChecks) to create, track, and update concurrent executions.

Reviewed Changes

Copilot reviewed 65 out of 65 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
client/src/preview/route/digitaltwins/execute/LogDialog.tsx	Fetch execution history via thunk and render `ExecutionHistoryList` instead of inline logs
client/src/preview/route/digitaltwins/editor/Sidebar.tsx	Updated `selectDigitalTwinByName` import to new model/backend slice
client/src/preview/route/digitaltwins/create/CreateDTDialog.tsx	Switched `setDigitalTwin` import to new model/backend slice
client/src/preview/components/execution/ExecutionHistoryLoader.tsx	New component to load all history on startup and poll running executions
client/src/preview/components/execution/ExecutionHistoryList.tsx	New list component displaying sorted history with stop/delete controls and log details
client/src/preview/components/asset/StartStopButton.tsx	Show running count, integrate `handleStart`, and replace loading indicator
client/src/preview/components/asset/LogButton.tsx	Add badge count for history entries and disable logic based on execution count
client/src/preview/components/asset/DetailsButton.tsx	Updated `selectDigitalTwinByName` import path
client/src/preview/components/asset/AssetCard.tsx	Initialize `logButtonDisabled` to `false` and pass `assetName` to `LogButton`
client/src/preview/components/asset/AssetBoard.tsx	Updated `setShouldFetchDigitalTwins` import to new model/backend slice
client/src/model/backend/gitlab/types/executionHistory.ts	New `ExecutionStatus`, `JobLog`, `ExecutionHistoryEntry`, and IndexedDB schema/types
client/src/model/backend/gitlab/state/executionHistory.slice.ts	New slice with CRUD thunks, selectors, and polling logic for concurrent execution tracking
client/src/model/backend/gitlab/state/digitalTwin.slice.ts	Aligned `JobLog` type import and added `selectDigitalTwins` selector
client/src/model/backend/gitlab/execution/pipelineUtils.ts	Refactored pipeline state helpers to support execution history updates and concurrent flows
client/src/model/backend/gitlab/execution/pipelineHandler.ts	Updated `handleStart`/`handleStop` to dispatch history fetch and pass `executionId`
client/src/model/backend/gitlab/execution/pipelineChecks.ts	Enhanced status checks to update IndexedDB, handle timeouts, and use new interfaces
client/src/model/backend/gitlab/execution/interfaces.ts	Defined `PipelineStatusParams` and `PipelineHandlerDispatch` for typed thunk usage
client/src/database/digitalTwins.ts	IndexedDB service implementation conforming to new schema for persisting execution history
client/src/AppProvider.tsx	Registered `ExecutionHistoryLoader` at the app root
client/package.json	Added `uuid`, `@types/uuid`, `fake-indexeddb` and updated dependencies for IndexedDB and IDs

Comments suppressed due to low confidence (3)

client/src/preview/components/execution/ExecutionHistoryLoader.tsx:15

ExecutionHistoryLoader handles initial data fetch and polling but has no associated unit or integration tests. Consider adding tests to cover its effect logic and cleanup of the interval.

const ExecutionHistoryLoader: React.FC = () => {

client/src/preview/route/digitaltwins/execute/LogDialog.tsx:27

[nitpick] The component is named LogDialog but now presents execution history. Consider renaming it to ExecutionHistoryDialog for clarity and consistency with its content.

function LogDialog({ showLog, setShowLog, name }: LogDialogProps) {

client/src/model/backend/gitlab/execution/pipelineChecks.ts:76

Calling digitalTwin.getExecutionHistoryById assumes the DigitalTwin class exposes that method, but IndexedDB access should use indexedDBService. Replace this with indexedDBService.getExecutionHistoryById to avoid runtime errors.

const pipelineId = executionId

Copilot · 2025-05-24T19:31:23Z

client/src/preview/components/asset/StartStopButton.tsx

+          const setButtonText = () => {}; // Dummy function since we don't need to change button text
+          handleStart(
+            'Start',


The call to handleStart always passes 'Start' and you’ve removed any buttonText state, so the UI can never switch to 'Stop'. This breaks the stop functionality—consider managing a local buttonText state or passing the current executionId to toggle correctly.

Suggested change

const setButtonText = () => {}; // Dummy function since we don't need to change button text

handleStart(

'Start',

const newButtonText = buttonText === 'Start' ? 'Stop' : 'Start';

setButtonText(newButtonText);

handleStart(

newButtonText,

Copilot · 2025-05-24T19:31:23Z

client/src/preview/route/digitaltwins/execute/LogDialog.tsx

+    }
+  }, [dispatch, name, showLog]);
+
+  const handleViewLogs = () => {};


handleViewLogs is defined with no parameters but ExecutionHistoryList expects onViewLogs to accept an executionId: string. Either implement log fetching there or remove the unused callback.

Suggested change

const handleViewLogs = () => {};

const handleViewLogs = (executionId: string) => {

console.log(`View logs for execution ID: ${executionId}`);

};

Copilot · 2025-05-24T19:31:24Z

client/src/preview/components/execution/ExecutionHistoryList.tsx

+                    (jobLog: JobLog, index: number) => (
+                      <div key={index} style={{ marginBottom: '16px' }}>


[nitpick] Using the array index as a React key can lead to rendering issues if logs change order or new logs are inserted. Use a stable identifier (e.g., jobLog.jobName or a combined ID) instead.

Suggested change

(jobLog: JobLog, index: number) => (

<div key={index} style={{ marginBottom: '16px' }}>

(jobLog: JobLog) => (

<div key={jobLog.jobName} style={{ marginBottom: '16px' }}>

prasadtalasila

@Microchesst Thanks for the updates. The feature functionality is much improved and it works as discussed. However, I have a few comments on the existing implementation. Please check the comments. Thanks.

prasadtalasila · 2025-05-24T19:17:23Z

client/src/database/digitalTwins.ts

+ * Interface for IndexedDB operations
 */
-class IndexedDBService {
+export interface IIndexedDBService {


If the name is changed to IExecutionHistory, the ExecutionHistory suffix can be removed from the method names.

The rest of the code need not know about the use of IndexDB. The interface can be independent of IndexDB and the ExecutionHistory class can include IndexDB inside without letting other parts of the code knowing about it.

client/package.json

prasadtalasila · 2025-05-24T19:45:36Z

client/src/AppProvider.tsx

      <ThemeProvider theme={mdTheme}>
        <AuthProvider>
          <CssBaseline />
+          <ExecutionHistoryLoader />


Why is this react component here?

prasadtalasila · 2025-05-24T20:09:20Z

client/src/database/digitalTwins.ts

all of the methods are returning Promise<void>. There are other possible outcomes with the IndexDB operations. Perhaps they should be used to provide meaningful promises?

prasadtalasila · 2025-05-24T20:10:48Z

client/src/model/backend/gitlab/types/executionHistory.ts

+  pipelineId: number; // GitLab pipeline ID
+  timestamp: number; // Timestamp when the execution was started
+  status: ExecutionStatus; // Current status of the execution
+  jobLogs: JobLog[]; // Logs from the execution


Is the array collecting logs separately from each of the jobs in a pipeline?

prasadtalasila · 2025-05-25T09:39:14Z

client/src/preview/components/asset/StartStopButton.tsx

Previously one button was used for the start and stop actions but now they are separate. The stop is inside History button. Perhaps a HistoryButton.tsx component be placed in src/components/asset. The component responsibility could be:

StartButton.tsx --> manages only start
HistoryButton.tsx --> manages stop and logs

prasadtalasila · 2025-05-25T09:41:30Z

client/src/preview/components/execution/ExecutionHistoryList.tsx

please move this to src/components/execution

prasadtalasila · 2025-05-25T09:44:47Z

client/src/preview/components/execution/ExecutionHistoryLoader.tsx

+
+    const intervalId = setInterval(() => {
+      dispatch(checkRunningExecutions());
+    }, 10000); // Check every 10 seconds


please give a name to constant and place it in src/model/backend/gitlab/constants.ts

prasadtalasila · 2025-05-25T09:45:41Z

client/src/preview/route/digitaltwins/editor/Sidebar.tsx

 import { RootState } from 'store/store';
 import { addOrUpdateLibraryFile } from 'preview/store/libraryConfigFiles.slice';
 import { getFilteredFileNames } from 'preview/util/fileUtils';
 import { FileState } from '../../../store/file.slice';


paths from src directory

prasadtalasila · 2025-05-25T09:51:07Z

client/src/store/store.ts

+          'assets/setAsset',
+          'assets/deleteAsset',
+          // Execution history actions
+          'executionHistory/addExecutionHistoryEntry',


can this part be simplified but refactoring the executionHistory.slice and digitaltwin.slice?

…rchitecture.

sonarqubecloud · 2025-05-29T12:31:33Z

Quality Gate failed

Failed conditions
7.5% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

prasadtalasila

@Microchesst you are right about growing size of this PR. There are too many dependencies from src/route/digitaltwins/execution to src/preview. It makes sense to move the dependent files to corresponding locations in src/preview and then proceed with the PR. Hope that decision reduces the time to complete the PR.

prasadtalasila · 2025-05-29T19:58:32Z

client/src/route/digitaltwins/execution/executionUIHandlers.ts

please rename file to executionStatusHandlers.ts and variables to

setButtonText to setLogStatusText

LogButtonDisabled to LogDisabled

prasadtalasila · 2025-05-29T19:59:57Z

client/src/route/digitaltwins/execution/executionUIHandlers.ts

These files are importing from preview/. It might take too much of refactoring to remove dependence from preview/ code. If it is so, please place these files in the right location in preview/ itself.

prasadtalasila · 2025-05-29T20:03:29Z

client/src/store/selectors/executionHistory.selectors.ts

+    (entries) => entries.find((entry) => entry.id === id),
+  );
+
+export const selectSelectedExecutionId = (state: RootState) =>


prasadtalasila · 2025-05-29T20:07:56Z

client/src/model/backend/gitlab/services/ExecutionStatusService.ts

this file depends on IndexDB. Perhaps this should be moved to src/services?

prasadtalasila · 2025-05-29T20:11:44Z

client/src/route/digitaltwins/execution/digitalTwinAdapter.ts

perhaps moving this preview/util will reduce the time required to complete this PR

Microchesst added 2 commits May 1, 2025 20:50

feat: Concurrent executions with indexedDB

26e5724

yarn syntax

e168a9a

prasadtalasila reviewed May 2, 2025

View reviewed changes

Microchesst added 2 commits May 15, 2025 08:50

test: everything except e2e

6fc0304

yarn format

117405f

Microchesst added 2 commits May 15, 2025 17:46

e2e Test

a8a6614

test: e2e

047f00d

prasadtalasila reviewed May 22, 2025

View reviewed changes

client/package.json Show resolved Hide resolved

Microchesst added 4 commits May 24, 2025 12:27

feat: now using accordion and can load across reloads, still needs te…

b91218f

…sting.

Feat: interface for 'database/digitaltTwins.ts'

58f6d06

fix: now only updating a dt's specific state

c80a929

fix: moving some files out of preview code

9d3f7e3

prasadtalasila requested a review from Copilot May 24, 2025 19:28

Copilot AI reviewed May 24, 2025

View reviewed changes

prasadtalasila reviewed May 25, 2025

View reviewed changes

Microchesst added 2 commits May 25, 2025 19:52

test: unit int and e2e are now updated.

4e522fe

feat: refactoring to keep code structure, and to define a good code a…

1c0f14c

…rchitecture.

prasadtalasila reviewed May 29, 2025

View reviewed changes

Microchesst mentioned this pull request Jun 16, 2025

Split pipeline utilities into focused modules #1272

Closed

9 tasks

prasadtalasila mentioned this pull request Aug 22, 2025

[FEATURE]Migrate DT concurrent execution code from preview into stable codebase #1314

Open

4 tasks

atomicgamedeveloper mentioned this pull request Sep 19, 2025

feat: Concurrent executions with indexedDB Main Finalization #1328

Open

9 tasks

-  const handleViewLogs = () => {};
+  const handleViewLogs = (executionId: string) => {
+    console.log(`View logs for execution ID: ${executionId}`);
+  };

		(jobLog: JobLog, index: number) => (
		<div key={index} style={{ marginBottom: '16px' }}>

feat: Concurrent executions with indexedDB #1235

Are you sure you want to change the base?

feat: Concurrent executions with indexedDB #1235

Uh oh!

Conversation

Microchesst commented May 1, 2025

Pull Request Template

Title

Type of Change

Description

Impact

Checklist

Uh oh!

Microchesst commented May 1, 2025

Uh oh!

prasadtalasila commented May 2, 2025

Uh oh!

prasadtalasila left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

prasadtalasila commented May 15, 2025

Uh oh!

prasadtalasila commented May 15, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI May 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 24, 2025

Choose a reason for hiding this comment

Uh oh!

prasadtalasila left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented May 29, 2025

Quality Gate failed

Uh oh!

prasadtalasila left a comment

codecov bot commented May 15, 2025 •

edited

Loading