Content - e2e tests in SDK Nightly (demisto#30189)

* added placeholder for future jobs

* added job for xsoar saas bucket

* test xsoar, ng e2e bash

* adding more commands to test

* test for nightlies

* adding support for cloud machines

* adding more commands to test

* more tests

* adding config for xsiam tests

* fix mpv2 e2e

* fix variables for xsoar ng, xsiam

* added prints

* env variables

* changes

* correct pipeline

* fix yml

* xsoar saas prepare bucket

* added test-content and modeling-rules test

* mpv2 changes

* fix issues with mpv2 and xsoar ng

* fix packs to install new line

* added workaround for filter_file

* fix env_results workaround

* test xsoar ng upload with -z

* test run-tests on machines

* xsoar ng auth id

* xsoar ng upload check

* fix upload without zip

* test-content change

* download IncidentField fix

* upload and test-content

* upload xsoar ng, xsiam

* adding 15 min sleep befor run_tests

* is_nightly: true

* remove is_nightly:true and remove install_content_and_test_integrations

* remove lock of HW and add install content step

* remove upload of integrations, is_nightly:true

* No unlock and fix variables

* test mpv2 ng parameter nightly:true

* install_content and remove explicit upload, add art folder instance create

* fix section end

* remove nightly:true for xsoar

* revert logging

* added after script to xsoar step, and art variable

* check ARTIFACTS_FOLDER value

* adding workaround for filter_file on xsoar

* added ALLOW_FAILURE var when run_tests

* fix new variable, allow failure on test model rules

* workaround for static env_results value in sdk

* fix workaround

* remove sleep for 15 min for cloud machines, added script to global

* delete sleep in ng

* remove create_entities_for_nightly_sdk

* fix merge

* fix merge again

* fix merge again if

* add more things to upload

* fix upload

* remove upload-download

* add e2e tests and check flow

* poetry lock, e2e tests to xsiam, ng

* support for e2e xsiam. ng

* poetry lock

* poetry

* cr + hello world

* xsiasm

* added general e2e tests step

* generalize steps

* step

* move copy env to the start

* test data

* uppercase INSTANCE_ROLE

* revert changes, and fix test data

* copy_env_results, prepare_for_test_content

* run_end_to_end_tests, cleanup_after_test_content, ng copy_env

* e2e test check xsoar

* revert xsoar e2e tests

* fix modeling rules

* xsiam re-ordering

* xif, test data

* modeling rules

* testdata

* fix modeling rules

* schema, check delete workaround

* run-commands-against-instance

* TUNNEL_PORT in xsoar

* unset DEMISTO_API_KEY # a temporary fix

* run-command-against-instance remove

* _time

* delete classifier

* OldValidateManager

* HelloWorld

* revert helloworld parsing rules

* override the filter file

* sanity tests

* cr

* assert test content

* ARTIFACTS_FOLDER_INSTANCE/succeeded_tests.tx

* num_of_tests_that_should_run

* EXIT_CODE

* =

* assert_test_content

* cr

* ciac-9091

* poetry lock

* poetry lock

* poetry lock and saas

* poetry lock revert

Author: MLainer1

.gitlab/ci/.gitlab-ci.global.yml

@@ -245,6 +245,32 @@
   - |
   - section_end "Checkout upload commit content nightly"
 
+.export_cloud_machine_constants: &export_cloud_machine_constants
+  # exporting the machine credentials
+  - CLOUD_SERVERS_PATH=$(cat $CLOUD_SERVERS_FILE)
+  - |
+    if [[ $TEST_XDR_ENV ]]; then
+      cat "${CLOUD_API_KEYS}" > "cloud_api_keys.json"
+    else
+      echo "${CLOUD_API_KEYS}" > "cloud_api_keys.json"
+    fi
+  - |
+    if [[ $TEST_XDR_ENV ]]; then
+      cat "${CLOUD_API_TOKENS}" > "cloud_api_tokens.json"
+    else
+      echo "${CLOUD_API_TOKENS}" > "cloud_api_tokens.json"
+    fi
+
+  - IFS=', ' read -r -a CLOUD_CHOSEN_MACHINE_ID_ARRAY <<< "${CLOUD_CHOSEN_MACHINE_IDS}"
+  - |
+    for CLOUD_CHOSEN_MACHINE_ID in "${CLOUD_CHOSEN_MACHINE_ID_ARRAY[@]}"; do
+      export XSIAM_SERVER_CONFIG=$(jq -r ".[\"${CLOUD_CHOSEN_MACHINE_ID}\"]" < "$CLOUD_SERVERS_PATH")
+      export DEMISTO_BASE_URL=$(echo "$XSIAM_SERVER_CONFIG" | jq -r ".[\"base_url\"]")
+      export XSIAM_AUTH_ID=$(echo "$XSIAM_SERVER_CONFIG" | jq -r ".[\"x-xdr-auth-id\"]")
+      export DEMISTO_API_KEY=$(jq -r ".[\"${CLOUD_CHOSEN_MACHINE_ID}\"]" < "cloud_api_keys.json")
+      export XSIAM_TOKEN=$(jq -r ".[\"${CLOUD_CHOSEN_MACHINE_ID}\"]" < "cloud_api_tokens.json")
+      break
+    done
 
 .default-before-script:
   before_script: