feat: Initial cloud-bench implementation (#30)

* Initial cloud-bench implementation

* rename viewer

* Refactor to follow example structure

* use locals

* Simplify default regions

* Simplify default regions

* revert readme changes

* Update main.tf

* add GCP multiaccount example

* add GCP multiaccount example

* don't disable services on delete

* reorganize singe-project to match org

* support organizational cloud-bench

Co-authored-by: Néstor Salceda <[email protected]>
Co-authored-by: Alex <[email protected]>

Author: 3 people

README.md

@@ -16,10 +16,16 @@ The following GCP APIs must be enabled to deply resources correctly
 > ##### APIs Required by Cloud Scanning
 >* Cloud Run API
 >* Eventarc API
->* Secret Maanger API
+>* Secret Manger API
 >* Cloud Build API
 >* Identity and access management API
 
+> ##### APIs Required by Cloud Benchmarks
+>* Identity and access management API
+>* IAM Service Account Credentials API
+>* Cloud Resource Manager API
+>* Security Token Service API
+
 ### Module Usage
 
 ```hcl

examples/organization/README.md

@@ -10,6 +10,8 @@ This example deploys Cloud Connector into a GCP organizational GCP account.
 |------|---------|
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.14.0 |
 | <a name="requirement_google"></a> [google](#requirement\_google) | >= 3.67.0 |
+| <a name="requirement_google-beta"></a> [google-beta](#requirement\_google-beta) | >= 3.67.0 |
+| <a name="requirement_sysdig"></a> [sysdig](#requirement\_sysdig) | >= 0.5.21 |
 
 ## Providers
 
@@ -21,11 +23,12 @@ This example deploys Cloud Connector into a GCP organizational GCP account.
 
 | Name | Source | Version |
 |------|--------|---------|
-| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector |  |
-| <a name="module_cloud_scanning"></a> [cloud\_scanning](#module\_cloud\_scanning) | ../../modules/services/cloud-scanning |  |
-| <a name="module_connector_organization_sink"></a> [connector\_organization\_sink](#module\_connector\_organization\_sink) | ../../modules/infrastructure/organization_sink |  |
-| <a name="module_scanning_organization_sink"></a> [scanning\_organization\_sink](#module\_scanning\_organization\_sink) | ../../modules/infrastructure/organization_sink |  |
-| <a name="module_secure_secrets"></a> [secure\_secrets](#module\_secure\_secrets) | ../../modules/infrastructure/secrets |  |
+| <a name="module_cloud_bench"></a> [cloud\_bench](#module\_cloud\_bench) | ../../modules/services/cloud-bench | n/a |
+| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector | n/a |
+| <a name="module_cloud_scanning"></a> [cloud\_scanning](#module\_cloud\_scanning) | ../../modules/services/cloud-scanning | n/a |
+| <a name="module_connector_organization_sink"></a> [connector\_organization\_sink](#module\_connector\_organization\_sink) | ../../modules/infrastructure/organization_sink | n/a |
+| <a name="module_scanning_organization_sink"></a> [scanning\_organization\_sink](#module\_scanning\_organization\_sink) | ../../modules/infrastructure/organization_sink | n/a |
+| <a name="module_secure_secrets"></a> [secure\_secrets](#module\_secure\_secrets) | ../../modules/infrastructure/secrets | n/a |
 
 ## Resources
 
@@ -35,17 +38,20 @@ This example deploys Cloud Connector into a GCP organizational GCP account.
 | [google_organization_iam_member.organization_image_puller](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/organization_iam_member) | resource |
 | [google_service_account.connector_sa](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_account) | resource |
 | [google_service_account.scanning_sa](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_account) | resource |
-| [google_project.project](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/project) | data source |
+| [google_organization.org](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/organization) | data source |
+| [google_projects.all_projects](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/projects) | data source |
 
 ## Inputs
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
+| <a name="input_benchmark_project_ids"></a> [benchmark\_project\_ids](#input\_benchmark\_project\_ids) | Google cloud project IDs to run Benchmarks on | `list(string)` | `[]` | no |
 | <a name="input_create_gcr_topic"></a> [create\_gcr\_topic](#input\_create\_gcr\_topic) | Deploys a PubSub topic called `gcr` as part of this stack, which is needed for GCR scanning. Set to `true` only if it doesn't exist yet. If this is not deployed, and no existing `gcr` topic is found, the GCR scanning is ommited and won't be deployed. For more info see [GCR PubSub topic](https://cloud.google.com/container-registry/docs/configuring-notifications#create_a_topic). | `bool` | `true` | no |
 | <a name="input_location"></a> [location](#input\_location) | Zone where the stack will be deployed | `string` | `"us-central1"` | no |
 | <a name="input_max_instances"></a> [max\_instances](#input\_max\_instances) | Max number of instances for the workloads | `number` | `1` | no |
 | <a name="input_naming_prefix"></a> [naming\_prefix](#input\_naming\_prefix) | Naming prefix for all the resources created | `string` | `"secure-for-cloud"` | no |
-| <a name="input_project_id"></a> [project\_id](#input\_project\_id) | Project ID | `string` | n/a | yes |
+| <a name="input_organization_domain"></a> [organization\_domain](#input\_organization\_domain) | Organization domain. e.g. sysdig.com | `string` | n/a | yes |
+| <a name="input_project_id"></a> [project\_id](#input\_project\_id) | organizational member project ID where the secure-for-cloud workload is going to be deployed | `string` | n/a | yes |
 | <a name="input_sysdig_secure_api_token"></a> [sysdig\_secure\_api\_token](#input\_sysdig\_secure\_api\_token) | Sysdig's Secure API Token | `string` | n/a | yes |
 | <a name="input_sysdig_secure_endpoint"></a> [sysdig\_secure\_endpoint](#input\_sysdig\_secure\_endpoint) | Sysdig Secure API endpoint | `string` | `"https://secure.sysdig.com"` | no |
 

examples/organization/main.tf

@@ -9,27 +9,40 @@ EOT
 }
 
 provider "google" {
-  project = var.project_id
-  region  = var.location
+  region = var.location
 }
 
-data "google_project" "project" {
-  project_id = var.project_id
+provider "google-beta" {
+  region = var.location
+}
+
+provider "sysdig" {
+  sysdig_secure_url          = var.sysdig_secure_endpoint
+  sysdig_secure_api_token    = var.sysdig_secure_api_token
+  sysdig_secure_insecure_tls = !local.verify_ssl
+}
+
+data "google_organization" "org" {
+  domain = var.organization_domain
+}
+
+data "google_projects" "all_projects" {
+  filter = "parent.id:${data.google_organization.org.org_id} parent.type:organization"
 }
 
 #######################
 #      CONNECTOR      #
 #######################
 resource "google_service_account" "connector_sa" {
-  account_id   = "${var.naming_prefix}-cloud-connector"
+  account_id   = "${var.naming_prefix}-connector"
   display_name = "Service account for cloud-connector"
 }
 
 module "connector_organization_sink" {
   source = "../../modules/infrastructure/organization_sink"
 
-  organization_id = data.google_project.project.org_id
-  naming_prefix   = "${var.naming_prefix}-cloud-connector"
+  organization_id = data.google_organization.org.org_id
+  naming_prefix   = "${var.naming_prefix}-connector"
   filter          = local.connector_filter
 }
 
@@ -41,6 +54,7 @@ module "cloud_connector" {
   sysdig_secure_endpoint    = var.sysdig_secure_endpoint
   connector_pubsub_topic_id = module.connector_organization_sink.pubsub_topic_id
   max_instances             = var.max_instances
+  project_id                = var.project_id
 
   #defaults
   naming_prefix = var.naming_prefix
@@ -51,13 +65,13 @@ module "cloud_connector" {
 #       SCANNING      #
 #######################
 resource "google_service_account" "scanning_sa" {
-  account_id   = "${var.naming_prefix}-cloud-scanning"
+  account_id   = "${var.naming_prefix}-scanning"
   display_name = "Service account for cloud-scanning"
 }
 
 
 resource "google_organization_iam_custom_role" "org_gcr_image_puller" {
-  org_id = data.google_project.project.org_id
+  org_id = data.google_organization.org.org_id
 
   role_id     = "${var.naming_prefix}_gcr_image_puller"
   title       = "Sysdig GCR Image Puller"
@@ -66,7 +80,7 @@ resource "google_organization_iam_custom_role" "org_gcr_image_puller" {
 }
 
 resource "google_organization_iam_member" "organization_image_puller" {
-  org_id = data.google_project.project.org_id
+  org_id = data.google_organization.org.org_id
 
   role   = google_organization_iam_custom_role.org_gcr_image_puller.id
   member = "serviceAccount:${google_service_account.scanning_sa.email}"
@@ -75,8 +89,8 @@ resource "google_organization_iam_member" "organization_image_puller" {
 module "scanning_organization_sink" {
   source = "../../modules/infrastructure/organization_sink"
 
-  organization_id = data.google_project.project.org_id
-  naming_prefix   = "${var.naming_prefix}-cloud-scanning"
+  organization_id = data.google_organization.org.org_id
+  naming_prefix   = "${var.naming_prefix}-scanning"
   filter          = local.scanning_filter
 }
 
@@ -100,6 +114,21 @@ module "cloud_scanning" {
   cloud_scanning_sa_email  = google_service_account.scanning_sa.email
   create_gcr_topic         = var.create_gcr_topic
   scanning_pubsub_topic_id = module.connector_organization_sink.pubsub_topic_id
+  project_id               = var.project_id
 
   max_instances = var.max_instances
 }
+
+#######################
+#      BENCHMARKS     #
+#######################
+locals {
+  benchmark_projects_ids = length(var.benchmark_project_ids) == 0 ? [for p in data.google_projects.all_projects.projects : p.project_id] : var.benchmark_project_ids
+}
+
+module "cloud_bench" {
+  for_each = toset(local.benchmark_projects_ids)
+  source   = "../../modules/services/cloud-bench"
+
+  project_id = each.key
+}

examples/organization/variables.tf

@@ -4,9 +4,14 @@ variable "sysdig_secure_api_token" {
   description = "Sysdig's Secure API Token"
 }
 
+variable "organization_domain" {
+  type        = string
+  description = "Organization domain. e.g. sysdig.com"
+}
+
 variable "project_id" {
   type        = string
-  description = "Project ID"
+  description = "organizational member project ID where the secure-for-cloud workload is going to be deployed"
 }
 
 # Vars with defaults
@@ -34,6 +39,12 @@ variable "max_instances" {
   default     = 1
 }
 
+variable "benchmark_project_ids" {
+  default     = []
+  type        = list(string)
+  description = "Google cloud project IDs to run Benchmarks on"
+}
+
 variable "create_gcr_topic" {
   type        = bool
   description = "Deploys a PubSub topic called `gcr` as part of this stack, which is needed for GCR scanning. Set to `true` only if it doesn't exist yet. If this is not deployed, and no existing `gcr` topic is found, the GCR scanning is ommited and won't be deployed. For more info see [GCR PubSub topic](https://cloud.google.com/container-registry/docs/configuring-notifications#create_a_topic)."

examples/organization/versions.tf

@@ -6,5 +6,13 @@ terraform {
       source  = "hashicorp/google"
       version = ">= 3.67.0"
     }
+    google-beta = {
+      source  = "hashicorp/google-beta"
+      version = ">= 3.67.0"
+    }
+    sysdig = {
+      source  = "sysdiglabs/sysdig"
+      version = ">= 0.5.21"
+    }
   }
 }

examples/single-project/README.md

@@ -11,6 +11,8 @@ This example deploys Cloud Connector and Cloud Scanning into a GCP account.
 |------|---------|
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.14.0 |
 | <a name="requirement_google"></a> [google](#requirement\_google) | >= 3.67.0 |
+| <a name="requirement_google-beta"></a> [google-beta](#requirement\_google-beta) | >= 3.67.0 |
+| <a name="requirement_sysdig"></a> [sysdig](#requirement\_sysdig) | >= 0.5.21 |
 
 ## Providers
 
@@ -22,11 +24,12 @@ This example deploys Cloud Connector and Cloud Scanning into a GCP account.
 
 | Name | Source | Version |
 |------|--------|---------|
-| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector |  |
-| <a name="module_cloud_scanning"></a> [cloud\_scanning](#module\_cloud\_scanning) | ../../modules/services/cloud-scanning |  |
-| <a name="module_connector_project_sink"></a> [connector\_project\_sink](#module\_connector\_project\_sink) | ../../modules/infrastructure/project_sink |  |
-| <a name="module_scanning_project_sink"></a> [scanning\_project\_sink](#module\_scanning\_project\_sink) | ../../modules/infrastructure/project_sink |  |
-| <a name="module_secure_secrets"></a> [secure\_secrets](#module\_secure\_secrets) | ../../modules/infrastructure/secrets |  |
+| <a name="module_cloud_bench"></a> [cloud\_bench](#module\_cloud\_bench) | ../../modules/services/cloud-bench | n/a |
+| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector | n/a |
+| <a name="module_cloud_scanning"></a> [cloud\_scanning](#module\_cloud\_scanning) | ../../modules/services/cloud-scanning | n/a |
+| <a name="module_connector_project_sink"></a> [connector\_project\_sink](#module\_connector\_project\_sink) | ../../modules/infrastructure/project_sink | n/a |
+| <a name="module_scanning_project_sink"></a> [scanning\_project\_sink](#module\_scanning\_project\_sink) | ../../modules/infrastructure/project_sink | n/a |
+| <a name="module_secure_secrets"></a> [secure\_secrets](#module\_secure\_secrets) | ../../modules/infrastructure/secrets | n/a |
 
 ## Resources
 

examples/single-project/main.tf

@@ -8,15 +8,25 @@ EOT
 EOT
 }
 
-
-#######################
-#      CONNECTOR      #
-#######################
 provider "google" {
   project = var.project_id
   region  = var.location
 }
 
+provider "google-beta" {
+  project = var.project_id
+  region  = var.location
+}
+
+provider "sysdig" {
+  sysdig_secure_url          = var.sysdig_secure_endpoint
+  sysdig_secure_api_token    = var.sysdig_secure_api_token
+  sysdig_secure_insecure_tls = !local.verify_ssl
+}
+
+#######################
+#      CONNECTOR      #
+#######################
 resource "google_service_account" "connector_sa" {
   account_id   = "${var.naming_prefix}-cloud-connector"
   display_name = "Service account for cloud-connector"
@@ -37,6 +47,7 @@ module "cloud_connector" {
   sysdig_secure_api_token   = var.sysdig_secure_api_token
   sysdig_secure_endpoint    = var.sysdig_secure_endpoint
   connector_pubsub_topic_id = module.connector_project_sink.pubsub_topic_id
+  project_id                = var.project_id
 
   #defaults
   naming_prefix = var.naming_prefix
@@ -47,7 +58,6 @@ module "cloud_connector" {
 #######################
 #       SCANNING      #
 #######################
-
 resource "google_service_account" "scanning_sa" {
   account_id   = "${var.naming_prefix}-cloud-scanning"
   display_name = "Service account for cloud-scanning"
@@ -74,6 +84,7 @@ module "cloud_scanning" {
   cloud_scanning_sa_email  = google_service_account.scanning_sa.email
   scanning_pubsub_topic_id = module.scanning_project_sink.pubsub_topic_id
   create_gcr_topic         = var.create_gcr_topic
+  project_id               = var.project_id
 
   secure_api_token_secret_id = module.secure_secrets.secure_api_token_secret_name
   sysdig_secure_api_token    = var.sysdig_secure_api_token
@@ -83,3 +94,11 @@ module "cloud_scanning" {
   naming_prefix = var.naming_prefix
   verify_ssl    = local.verify_ssl
 }
+
+
+#######################
+#      BENCHMARKS     #
+#######################
+module "cloud_bench" {
+  source = "../../modules/services/cloud-bench"
+}

examples/single-project/variables.tf

@@ -15,6 +15,7 @@ variable "location" {
   default     = "us-central1"
   description = "Zone where the stack will be deployed"
 }
+
 variable "sysdig_secure_endpoint" {
   type        = string
   default     = "https://secure.sysdig.com"

examples/single-project/versions.tf

@@ -6,5 +6,13 @@ terraform {
       source  = "hashicorp/google"
       version = ">= 3.67.0"
     }
+    google-beta = {
+      source  = "hashicorp/google-beta"
+      version = ">= 3.67.0"
+    }
+    sysdig = {
+      source  = "sysdiglabs/sysdig"
+      version = ">= 0.5.21"
+    }
   }
 }