Skip to content

Commit 15304f2

Browse files
AnkitCLIAnkitCLI
authored
Merge pull request #92 from cloudsufi/dataValidationServiceNow
data validation for serviceNow
2 parents 20bff01 + 8997aab commit 15304f2

File tree

10 files changed

+312
-106
lines changed

10 files changed

+312
-106
lines changed

‎src/e2e-test/features/servicenowmultisource/RunTime.feature‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,13 +18,13 @@
1818
@Regression
1919
Feature: ServiceNow Multi Source - Run time scenarios
2020

21-
@TS-SN-MULTI-RNTM-01 @BQ_SINK
21+
@TS-SN-MULTI-RNTM-01 @BQ_SINK @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE
2222
Scenario: Verify user should be able to preview the pipeline
2323
When Open Datafusion Project to configure pipeline
2424
And Select plugin: "ServiceNow Multi Source" from the plugins list as: "Source"
2525
And Navigate to the properties page of plugin: "ServiceNow Multi Source"
2626
And configure ServiceNow Multi source plugin for below listed tables:
27-
| HARDWARE_CATALOG | RECEIVING_SLIP_LINE |
27+
| HARDWARE_CATALOG | DATE_TIME_TABLE |
2828
And fill Credentials section for pipeline user
2929
Then Validate "ServiceNow Multi Source" plugin properties
3030
And Close the Plugin Properties page
@@ -38,13 +38,13 @@ Feature: ServiceNow Multi Source - Run time scenarios
3838
And Wait till pipeline preview is in running state with a timeout of 500 seconds
3939
Then Verify the preview of pipeline is "success"
4040

41-
@TS-SN-MULTI-RNTM-02 @BQ_SINK
41+
@TS-SN-MULTI-RNTM-02 @BQ_SINK @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE
4242
Scenario: Verify user should be able to run the pipeline
4343
When Open Datafusion Project to configure pipeline
4444
And Select plugin: "ServiceNow Multi Source" from the plugins list as: "Source"
4545
And Navigate to the properties page of plugin: "ServiceNow Multi Source"
4646
And configure ServiceNow Multi source plugin for below listed tables:
47-
| RECEIVING_SLIP_LINE |
47+
| DATE_TIME_TABLE |
4848
And fill Credentials section for pipeline user
4949
Then Validate "ServiceNow Multi Source" plugin properties
5050
And Close the Plugin Properties page
@@ -62,7 +62,7 @@ Feature: ServiceNow Multi Source - Run time scenarios
6262
And Open and capture logs
6363
And Verify the pipeline status is "Succeeded"
6464

65-
@TS-SN-MULTI-RNTM-03 @CONNECTION @BQ_SINK
65+
@TS-SN-MULTI-RNTM-03 @CONNECTION @BQ_SINK @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE
6666
Scenario: Verify user should be able to deploy and run the pipeline using connection manager functionality
6767
When Open Datafusion Project to configure pipeline
6868
And Select plugin: "ServiceNow Multi Source" from the plugins list as: "Source"
@@ -78,7 +78,7 @@ Feature: ServiceNow Multi Source - Run time scenarios
7878
Then Click on the Create button
7979
And Use new connection
8080
And configure ServiceNow Multi source plugin for below listed tables:
81-
| RECEIVING_SLIP_LINE |
81+
| DATE_TIME_TABLE |
8282
Then Validate "ServiceNow Multi Source" plugin properties
8383
And Close the Plugin Properties page
8484
And Select Sink plugin: "BigQueryMultiTable" from the plugins list

‎src/e2e-test/features/servicenowmultisource/RunTimeWithMacros.feature‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
@Regression
1919
Feature: ServiceNow Multi Source - Run time scenarios (macro)
2020

21-
@TS-SN-MULTI-RNTM-MACRO-01 @BQ_SINK
21+
@TS-SN-MULTI-RNTM-MACRO-01 @BQ_SINK @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE
2222
Scenario: Verify user should be able to preview the pipeline when the source plugin is configured with macros
2323
When Open Datafusion Project to configure pipeline
2424
And Select plugin: "ServiceNow Multi Source" from the plugins list as: "Source"
@@ -39,7 +39,7 @@ Feature: ServiceNow Multi Source - Run time scenarios (macro)
3939
Then Validate "BigQuery Multi Table" plugin properties
4040
And Close the Plugin Properties page
4141
And Preview and run the pipeline
42-
And Enter runtime argument value "receiving_slip_line" for key "tableNames"
42+
And Enter runtime argument value "date_time_table" for key "tableNames"
4343
And Enter runtime argument value from environment variable "client.id" for key "clientId"
4444
And Enter runtime argument value from environment variable "client.secret" for key "clientSecret"
4545
And Enter runtime argument value from environment variable "rest.api.endpoint" for key "restApiEndpoint"
@@ -48,7 +48,7 @@ Feature: ServiceNow Multi Source - Run time scenarios (macro)
4848
Then Run the preview of pipeline with runtime arguments
4949
Then Verify the preview of pipeline is "success"
5050

51-
@TS-SN-MULTI-RNTM-MACRO-02 @BQ_SINK
51+
@TS-SN-MULTI-RNTM-MACRO-02 @BQ_SINK @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE
5252
Scenario: Verify user should be able to run the pipeline when the source plugin is configured with macros
5353
When Open Datafusion Project to configure pipeline
5454
And Select plugin: "ServiceNow Multi Source" from the plugins list as: "Source"
@@ -72,7 +72,7 @@ Feature: ServiceNow Multi Source - Run time scenarios (macro)
7272
And Close the Plugin Properties page
7373
And Save and Deploy Pipeline
7474
And Run the Pipeline in Runtime
75-
And Enter runtime argument value "receiving_slip_line" for key "tableNames"
75+
And Enter runtime argument value "date_time_table" for key "tableNames"
7676
And Enter runtime argument value from environment variable "client.id" for key "clientId"
7777
And Enter runtime argument value from environment variable "client.secret" for key "clientSecret"
7878
And Enter runtime argument value from environment variable "rest.api.endpoint" for key "restApiEndpoint"

‎src/e2e-test/features/servicenowsource/RunTime.feature‎

Lines changed: 55 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,12 +44,12 @@ Feature: ServiceNow Source - Run time scenarios
4444
And Click on the Preview Data link on the Sink plugin node: "BigQuery"
4545
Then Verify sink plugin's Preview Data for Input Records table and the Input Schema matches the Output Schema of Source plugin
4646

47-
@TS-SN-RNTM-2 @SN_SOURCE_CONFIG @SN_RECEIVING_SLIP_LINE @BQ_SINK
47+
@TS-SN-RNTM-2 @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE @BQ_SINK
4848
Scenario: Verify user should be able to deploy and run the pipeline where ServiceNow source is configured for Table mode
4949
When Open Datafusion Project to configure pipeline
5050
And Select plugin: "ServiceNow" from the plugins list as: "Source"
5151
And Navigate to the properties page of plugin: "ServiceNow"
52-
And configure ServiceNow source plugin for table: "RECEIVING_SLIP_LINE" in the Table mode
52+
And configure ServiceNow source plugin for table: "DATE_TIME_TABLE" in the Table mode
5353
And fill Credentials section for pipeline user
5454
And Enter input plugin property: "startDate" with value: "start.date"
5555
And Enter input plugin property: "endDate" with value: "end.date"
@@ -70,7 +70,7 @@ Feature: ServiceNow Source - Run time scenarios
7070
And Verify the pipeline status is "Succeeded"
7171
And Verify count of no of records transferred to the target BigQuery Table
7272

73-
@TS-SN-RNTM-03 @CONNECTION @SN_SOURCE_CONFIG @SN_RECEIVING_SLIP_LINE @BQ_SINK
73+
@TS-SN-RNTM-03 @CONNECTION @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE @BQ_SINK
7474
Scenario: Verify user should be able to deploy and run the pipeline using connection manager functionality
7575
When Open Datafusion Project to configure pipeline
7676
And Select plugin: "ServiceNow" from the plugins list as: "Source"
@@ -85,7 +85,7 @@ Feature: ServiceNow Source - Run time scenarios
8585
And Verify the test connection is successful
8686
Then Click on the Create button
8787
And Use new connection
88-
And configure ServiceNow source plugin for table: "RECEIVING_SLIP_LINE" in the Table mode
88+
And configure ServiceNow source plugin for table: "DATE_TIME_TABLE" in the Table mode
8989
Then Validate "ServiceNow" plugin properties
9090
And Close the Plugin Properties page
9191
And Select Sink plugin: "BigQueryTable" from the plugins list
@@ -102,3 +102,54 @@ Feature: ServiceNow Source - Run time scenarios
102102
And Open and capture logs
103103
And Verify the pipeline status is "Succeeded"
104104
Then Verify count of no of records transferred to the target BigQuery Table
105+
106+
@TS-SN-RNTM-4 @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE @BQ_SINK
107+
Scenario: Verify user should be able to deploy and run the pipeline with dateTime table with value type display
108+
When Open Datafusion Project to configure pipeline
109+
And Select plugin: "ServiceNow" from the plugins list as: "Source"
110+
And Navigate to the properties page of plugin: "ServiceNow"
111+
And configure ServiceNow source plugin for table: "DATE_TIME_TABLE" in the Table mode
112+
And fill Credentials section for pipeline user
113+
And Select dropdown plugin property: "valueType" with option value: "Display"
114+
Then Validate "ServiceNow" plugin properties
115+
And Close the Plugin Properties page
116+
And Select Sink plugin: "BigQueryTable" from the plugins list
117+
And Connect source as "ServiceNow" and sink as "BigQueryTable" to establish connection
118+
And Navigate to the properties page of plugin: "BigQuery"
119+
And Replace input plugin property: "project" with value: "projectId"
120+
And Enter input plugin property: "datasetProject" with value: "datasetprojectId"
121+
And Configure BigQuery sink plugin for Dataset and Table
122+
Then Validate "BigQuery" plugin properties
123+
And Close the Plugin Properties page
124+
And Save and Deploy Pipeline
125+
And Run the Pipeline in Runtime
126+
And Wait till pipeline is in running state
127+
And Open and capture logs
128+
And Verify the pipeline status is "Succeeded"
129+
And Verify count of no of records transferred to the target BigQuery Table
130+
Then Verify If new record created in ServiceNow application for table "date_time_table" is correct
131+
132+
@TS-SN-RNTM-5 @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE @BQ_SINK
133+
Scenario: Verify user should be able to deploy and run the pipeline with dateTime table with value type Actual
134+
When Open Datafusion Project to configure pipeline
135+
And Select plugin: "ServiceNow" from the plugins list as: "Source"
136+
And Navigate to the properties page of plugin: "ServiceNow"
137+
And configure ServiceNow source plugin for table: "DATE_TIME_TABLE" in the Table mode
138+
And fill Credentials section for pipeline user
139+
Then Validate "ServiceNow" plugin properties
140+
And Close the Plugin Properties page
141+
And Select Sink plugin: "BigQueryTable" from the plugins list
142+
And Connect source as "ServiceNow" and sink as "BigQueryTable" to establish connection
143+
And Navigate to the properties page of plugin: "BigQuery"
144+
And Replace input plugin property: "project" with value: "projectId"
145+
And Enter input plugin property: "datasetProject" with value: "datasetprojectId"
146+
And Configure BigQuery sink plugin for Dataset and Table
147+
Then Validate "BigQuery" plugin properties
148+
And Close the Plugin Properties page
149+
And Save and Deploy Pipeline
150+
And Run the Pipeline in Runtime
151+
And Wait till pipeline is in running state
152+
And Open and capture logs
153+
And Verify the pipeline status is "Succeeded"
154+
And Verify count of no of records transferred to the target BigQuery Table
155+
Then Verify If new record created in ServiceNow application for table "date_time_table" is correct

‎src/e2e-test/features/servicenowsource/RunTimeWithMacros.feature‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
@Regression
1919
Feature: ServiceNow Source - Run time scenarios (macro)
2020

21-
@TS-SN-RNTM-MACRO-01 @SN_SOURCE_CONFIG @SN_RECEIVING_SLIP_LINE @BQ_SINK
21+
@TS-SN-RNTM-MACRO-01 @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE @BQ_SINK
2222
Scenario: Verify user should be able to preview a pipeline when ServiceNow plugin is configured with macros
2323
When Open Datafusion Project to configure pipeline
2424
And Select plugin: "ServiceNow" from the plugins list as: "Source"
@@ -42,7 +42,7 @@ Feature: ServiceNow Source - Run time scenarios (macro)
4242
Then Validate "BigQuery" plugin properties
4343
And Close the Plugin Properties page
4444
And Preview and run the pipeline
45-
And Enter runtime argument value "receiving_slip_line" for key "tableName"
45+
And Enter runtime argument value "date_time_table" for key "tableName"
4646
And Enter runtime argument value from environment variable "client.id" for key "clientId"
4747
And Enter runtime argument value from environment variable "client.secret" for key "clientSecret"
4848
And Enter runtime argument value from environment variable "rest.api.endpoint" for key "restApiEndpoint"
@@ -51,7 +51,7 @@ Feature: ServiceNow Source - Run time scenarios (macro)
5151
And Run the preview of pipeline with runtime arguments
5252
Then Verify the preview of pipeline is "success"
5353

54-
@TS-SN-RNTM-MACRO-02 @SN_SOURCE_CONFIG @SN_RECEIVING_SLIP_LINE @BQ_SINK
54+
@TS-SN-RNTM-MACRO-02 @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE @BQ_SINK
5555
Scenario: Verify user should be able to run a pipeline when ServiceNow plugin is configured with macros
5656
When Open Datafusion Project to configure pipeline
5757
And Select plugin: "ServiceNow" from the plugins list as: "Source"
@@ -76,7 +76,7 @@ Feature: ServiceNow Source - Run time scenarios (macro)
7676
And Close the Plugin Properties page
7777
And Save and Deploy Pipeline
7878
And Run the Pipeline in Runtime
79-
And Enter runtime argument value "receiving_slip_line" for key "tableName"
79+
And Enter runtime argument value "date_time_table" for key "tableName"
8080
And Enter runtime argument value from environment variable "client.id" for key "clientId"
8181
And Enter runtime argument value from environment variable "client.secret" for key "clientSecret"
8282
And Enter runtime argument value from environment variable "rest.api.endpoint" for key "restApiEndpoint"
@@ -87,7 +87,7 @@ Feature: ServiceNow Source - Run time scenarios (macro)
8787
And Open and capture logs
8888
Then Verify the pipeline status is "Succeeded"
8989

90-
@TS-SN-RNTM-MACRO-03 @SN_SOURCE_CONFIG @SN_RECEIVING_SLIP_LINE @BQ_SINK
90+
@TS-SN-RNTM-MACRO-03 @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE @BQ_SINK
9191
Scenario: Verify pipeline failure message in logs when user provides an invalid Table Name with Macros
9292
When Open Datafusion Project to configure pipeline
9393
And Select plugin: "ServiceNow" from the plugins list as: "Source"
@@ -116,14 +116,14 @@ Feature: ServiceNow Source - Run time scenarios (macro)
116116
| Level | Message |
117117
| ERROR | invalid.tablename.logsmessage |
118118

119-
@TS-SN-RNTM-MACRO-04 @SN_SOURCE_CONFIG @SN_RECEIVING_SLIP_LINE @BQ_SINK
119+
@TS-SN-RNTM-MACRO-04 @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE @BQ_SINK
120120
Scenario: Verify pipeline failure message in logs when user provides invalid Credentials with Macros
121121
When Open Datafusion Project to configure pipeline
122122
And Select plugin: "ServiceNow" from the plugins list as: "Source"
123123
And Navigate to the properties page of plugin: "ServiceNow"
124124
And Fill Reference Name
125125
And Select mode as: "TABLE"
126-
And Enter input plugin property: "tableName" with value: "receiving_slip_line"
126+
And Enter input plugin property: "tableName" with value: "date_time_table"
127127
And Click on the Macro button of Property: "clientId" and set the value to: "clientId"
128128
And Click on the Macro button of Property: "clientSecret" and set the value to: "clientSecret"
129129
And Click on the Macro button of Property: "restApiEndpoint" and set the value to: "restApiEndpoint"
@@ -153,15 +153,15 @@ Feature: ServiceNow Source - Run time scenarios (macro)
153153
| Level | Message |
154154
| ERROR | invalid.credentials.logsmessage |
155155

156-
@TS-SN-RNTM-MACRO-05 @SN_SOURCE_CONFIG @SN_RECEIVING_SLIP_LINE @BQ_SINK
156+
@TS-SN-RNTM-MACRO-05 @SN_SOURCE_CONFIG @SN_DATE_TIME_TABLE @BQ_SINK
157157
Scenario: Verify pipeline failure message in logs when user provides invalid Advanced Properties with Macros
158158
When Open Datafusion Project to configure pipeline
159159
When Open Datafusion Project to configure pipeline
160160
And Select plugin: "ServiceNow" from the plugins list as: "Source"
161161
And Navigate to the properties page of plugin: "ServiceNow"
162162
And Fill Reference Name
163163
And Select mode as: "TABLE"
164-
And Enter input plugin property: "tableName" with value: "receiving_slip_line"
164+
And Enter input plugin property: "tableName" with value: "date_time_table"
165165
And fill Credentials section for pipeline user
166166
And Click on the Macro button of Property: "startDate" and set the value to: "startDate"
167167
And Click on the Macro button of Property: "endDate" and set the value to: "endDate"

0 commit comments

Comments
 (0)