abnormal_security,aws_bedrock,crowdstrike,nvidia_gpu: add required variables in system tests

[moxarth-rathod]

Proposed commit message

Fix system tests for abnormal_security, aws_bedrock, crowdstrike, nvidia_gpu integrations

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• [s] I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

• Clone integrations repo.
• Install elastic package locally.
• Start elastic stack using elastic-package.
• Move to integrations/packages/abnormal_security or integrations/packages/aws_bedrock or
  integrations/packages/crowdstrike or integrations/packages/nvidia_gpu directory.
• Run the following command to run tests.

elastic-package test -v

Related issues

• Closes [Multiple Integrations]: System test failures due to missing required variables in data stream configs #15671

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[andrewkroh]

Why are there three different YAML approaches to defining the required variables? This change has

wait_interval: ""
file_selectors: |
queue_url:
ssl: |

which equates to either empty string or null

{
  "wait_interval": "",
  "file_selectors": "",
  "queue_url": null,
  "ssl": ""
}

Could we use the same approach in all cases?

(To be clear, I'm not thrilled that we can bypass the required variable check by passing empty strings or nulls, but that's a different problem.)

[moxarth-rathod]

Why are there three different YAML approaches to defining the required variables? This change has

wait_interval: ""
file_selectors: |
queue_url:
ssl: |

which equates to either empty string or null

{
  "wait_interval": "",
  "file_selectors": "",
  "queue_url": null,
  "ssl": ""
}

Could we use the same approach in all cases?

(To be clear, I'm not thrilled that we can bypass the required variable check by passing empty strings or nulls, but that's a different problem.)

I initially tried using null everywhere, but it didn’t resolve issue in system test. So, I went with a different approach. For consistency, I’ll change it to use an empty string everywhere.

[efd6]

Looking at the change that added this configuration value (#14235), I think that it would be entirely reasonable for the config to be made non-required. This is based on the agent config code that was added and the description in the commit.

[efd6]

The complaint from ep is that this should be an array. Should this not be null?

[moxarth-rathod]

It throws the same error when I use null. The other option was to use an empty array, like file_selectors: |
So, I’ve changed it to an empty string for consistency.

[efd6]

Why not use the empty array? file_selectors: [] (file_selectors: | is an empty string, depending on the lines that follow).

It seems to me that the validation is missing something if it allows a string in an array slot.

[moxarth-rathod]

@efd6 I've made the changes as per your suggestion.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 1e97e4f

History

• 💚 Build #33685 succeeded 43300a2
• 💚 Build #33682 succeeded 9974cb2
• 💔 Build #33506 failed a7289b3

cc @moxarth-rathod