Skip to content

Commit d741c0a

Browse files
jimrightYazan Al-Yasin
jimright
and
Yazan Al-Yasin
committed
Add GCP hosts module
Signed-off-by: Jim Enright <[email protected]> Co-authored-by: Yazan Al-Yasin <[email protected]>
1 parent c9468d0 commit d741c0a

File tree

10 files changed

+633
-32
lines changed

10 files changed

+633
-32
lines changed

‎modules/hosts/README.md‎

Lines changed: 41 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,16 @@
11
<!-- BEGIN_TF_DOCS -->
2-
# Terraform Module for Hosts on ENTER\_INFRA\_PROVIDER
2+
# Terraform Module for Hosts on GCP
33

4-
> [!IMPORTANT]
5-
> This readme is automation generated using the [`terraform-docs`](https://terraform-docs.io/) command with the static contenta taken from [doc\_fragments/header.md](doc\_fragments/header.md)
6-
7-
The `hosts` module contains resource files to provision and manage <ENTER\_INFRA\_PROVIDER> instances with flexible configuration options for compute resources, storage volumes, and networking. This module is designed for Cloudera on premise infrastructure deployments on <ENTER\_INFRA\_PROVIDER>.
4+
The `hosts` module contains resource files to provision and manage GCP compute instances with flexible configuration options for compute resources, storage volumes, and networking. This module is designed for Cloudera on premise infrastructure deployments on GCP IaaS.
85

96
## Key Features
107

11-
<ENTER\_KEY\_FEATURES>
8+
* **Flexible Instance Deployment**: Create single instances or multiple numbered instances with customizable naming patterns
9+
* **Multi-Zone Distribution**: Automatically distribute instances across multiple GCP zones for high availability
10+
* **Public and Private IP Management**: Support for ephemeral public IPs, static external IP addresses, or private-only configurations
11+
* **Custom Storage Volumes**: Attach additional persistent disks with configurable size, type, and mount points
12+
* **SSH Key Management**: Automated SSH public key injection for secure instance access
13+
* **Network Tag Support**: Apply network tags for firewall rule targeting and security group management
1214

1315
## Usage
1416

@@ -24,21 +26,50 @@ No requirements.
2426

2527
## Providers
2628

27-
No providers.
29+
| Name | Version |
30+
|------|---------|
31+
| <a name="provider_google"></a> [google](#provider\_google) | n/a |
2832

2933
## Modules
3034

3135
No modules.
3236

3337
## Resources
3438

35-
No resources.
39+
| Name | Type |
40+
|------|------|
41+
| [google_compute_attached_disk.inventory](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_attached_disk) | resource |
42+
| [google_compute_disk.inventory](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_disk) | resource |
43+
| [google_compute_instance.pvc_base](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance) | resource |
3644

3745
## Inputs
3846

39-
No inputs.
47+
| Name | Description | Type | Default | Required |
48+
|------|-------------|------|---------|:--------:|
49+
| <a name="input_name"></a> [name](#input\_name) | Instance base name. If 'quantity' is 0, this is the exact name of the single instance. If 'quantity' > 0, name will be <name>-NN. | `string` | n/a | yes |
50+
| <a name="input_project_id"></a> [project\_id](#input\_project\_id) | GCP project ID where instances will be created. | `string` | n/a | yes |
51+
| <a name="input_region"></a> [region](#input\_region) | GCP region (e.g., us-central1) for the instances (used for regional resources if any, zone is primary for instances). | `string` | n/a | yes |
52+
| <a name="input_ssh_public_key"></a> [ssh\_public\_key](#input\_ssh\_public\_key) | The contents of the SSH public key to be added to the instance metadata for SSH access. | `string` | n/a | yes |
53+
| <a name="input_subnet_ids"></a> [subnet\_ids](#input\_subnet\_ids) | List of subnet self-links or names to assign instances to. Instances will be distributed across these subnets. Must not be empty if instances are created. | `list(string)` | n/a | yes |
54+
| <a name="input_zones"></a> [zones](#input\_zones) | GCP zone (e.g., us-central1-a) where instances will be created. | `list(string)` | n/a | yes |
55+
| <a name="input_external_ip_address"></a> [external\_ip\_address](#input\_external\_ip\_address) | Optional: A pre-allocated static external IP address to assign to the instance. If provided, 'public\_ip' should typically be false as this takes precedence. | `string` | `null` | no |
56+
| <a name="input_image"></a> [image](#input\_image) | Self-link the image (e.g., 'debian-cloud', 'centos-cloud', or your custom project). | `any` | `null` | no |
57+
| <a name="input_image_os_user"></a> [image\_os\_user](#input\_image\_os\_user) | GCP image default user. (e.g., 'cloud-user' for rhel-8) | `string` | `"cloud-user"` | no |
58+
| <a name="input_instance_labels"></a> [instance\_labels](#input\_instance\_labels) | Map of labels (key-value pairs) to apply to the compute instances. | `map(string)` | `{}` | no |
59+
| <a name="input_instance_type"></a> [instance\_type](#input\_instance\_type) | GCP machine type (e.g., e2-medium, n1-standard-1). | `string` | `"e2-micro"` | no |
60+
| <a name="input_network_tags"></a> [network\_tags](#input\_network\_tags) | List of network tags to attach to instances (for firewall rules). | `list(string)` | `[]` | no |
61+
| <a name="input_public_ip"></a> [public\_ip](#input\_public\_ip) | Assign an ephemeral public IP address to the instance's network interface. Set to false if 'external\_ip\_address' is provided. | `bool` | `false` | no |
62+
| <a name="input_quantity"></a> [quantity](#input\_quantity) | Number of instances. If 0 (default), one instance will be created with the exact 'name'. If > 0, 'quantity' instances will be created with numbered names. | `number` | `0` | no |
63+
| <a name="input_root_volume"></a> [root\_volume](#input\_root\_volume) | Root volume configuration for the boot disk. | <pre>object({<br/> delete_on_termination = optional(bool, true)<br/> volume_size = optional(number, 20)<br/> volume_type = optional(string, "pd-standard")<br/> })</pre> | `{}` | no |
64+
| <a name="input_startup_script"></a> [startup\_script](#input\_startup\_script) | Optional startup script (content, not path) to run on instance creation. | `string` | `"#!/bin/bash\nyum makecache >> /var/log/startup_script.log 2>&1\n"` | no |
65+
| <a name="input_subnetwork_project_id"></a> [subnetwork\_project\_id](#input\_subnetwork\_project\_id) | The project ID of the subnetwork. Defaults to var.project\_id if not set (for non-Shared VPC scenarios). | `string` | `""` | no |
66+
| <a name="input_volumes"></a> [volumes](#input\_volumes) | List of additional persistent volumes to create and attach to each instance. | <pre>list(object({<br/> name_suffix = string<br/> device_name = string # e.g. "sdb", "sdc" (will be /dev/sdb, /dev/sdc on instance)<br/> mount = string<br/> volume_size = optional(number, 100)<br/> volume_type = optional(string, "pd-standard")<br/> labels = optional(map(string), {})<br/> }))</pre> | `[]` | no |
4067

4168
## Outputs
4269

43-
No outputs.
70+
| Name | Description |
71+
|------|-------------|
72+
| <a name="output_instance_details"></a> [instance\_details](#output\_instance\_details) | A list of detailed information for each created instance. |
73+
| <a name="output_instances"></a> [instances](#output\_instances) | A list of the provisioned GCP compute instance objects. If quantity was 0, this list contains one instance. |
74+
| <a name="output_storage_volumes"></a> [storage\_volumes](#output\_storage\_volumes) | Map of additional storage volumes, keyed by instance ID. Each value is a list of volume details attached to that instance. |
4475
<!-- END_TF_DOCS -->

‎modules/hosts/doc_fragments/header.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,15 @@
1-
# Terraform Module for Hosts on ENTER_INFRA_PROVIDER
1+
# Terraform Module for Hosts on GCP
22

3-
> [!IMPORTANT]
4-
> This readme is automation generated using the [`terraform-docs`](https://terraform-docs.io/) command with the static contenta taken from [doc_fragments/header.md](doc_fragments/header.md)
5-
6-
The `hosts` module contains resource files to provision and manage <ENTER_INFRA_PROVIDER> instances with flexible configuration options for compute resources, storage volumes, and networking. This module is designed for Cloudera on premise infrastructure deployments on <ENTER_INFRA_PROVIDER>.
3+
The `hosts` module contains resource files to provision and manage GCP compute instances with flexible configuration options for compute resources, storage volumes, and networking. This module is designed for Cloudera on premise infrastructure deployments on GCP IaaS.
74

85
## Key Features
96

10-
<ENTER_KEY_FEATURES>
7+
* **Flexible Instance Deployment**: Create single instances or multiple numbered instances with customizable naming patterns
8+
* **Multi-Zone Distribution**: Automatically distribute instances across multiple GCP zones for high availability
9+
* **Public and Private IP Management**: Support for ephemeral public IPs, static external IP addresses, or private-only configurations
10+
* **Custom Storage Volumes**: Attach additional persistent disks with configurable size, type, and mount points
11+
* **SSH Key Management**: Automated SSH public key injection for secure instance access
12+
* **Network Tag Support**: Apply network tags for firewall rule targeting and security group management
1113

1214
## Usage
1315

‎modules/hosts/examples/ex01-minimal_inputs/main.tf‎

Lines changed: 161 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,169 @@
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
1414

15-
provider "aws" {
16-
region = var.aws_region
15+
provider "google" {
16+
project = var.project_id
17+
region = var.region
1718
}
1819

19-
module "ex01_hosts" {
20-
source = "../.."
20+
data "google_compute_zones" "available" {
21+
region = var.region
22+
}
23+
24+
locals {
25+
# Automatically fetches all zones in the selected region
26+
zones = data.google_compute_zones.available.names
27+
28+
asset_tags = {
29+
for k, v in var.asset_tags :
30+
# Sanitize both key and value
31+
lower(replace(replace(replace(replace(k,
32+
" ", "-"),
33+
"@", "_"),
34+
".", "_"),
35+
"[^a-z0-9_-]", "-")
36+
) =>
37+
lower(replace(replace(replace(replace(v,
38+
" ", "-"),
39+
"@", "_"),
40+
".", "_"),
41+
"[^a-z0-9_-]", "-")
42+
)
43+
}
44+
45+
tag_cluster_node = "${var.prefix}-cluster-node" # General tag for all cluster instances
46+
47+
}
48+
49+
module "ex01_network" {
50+
source = "../../../network"
51+
52+
project_id = var.project_id
53+
region = var.region
54+
zones = local.zones
55+
vpc_name = "${var.prefix}-pvc-base"
56+
vpc_exists = false
57+
prefix = var.prefix
58+
base_cidr_block = var.vpc_cidr
59+
60+
public_subnets = var.public_subnets_config
61+
private_subnets = var.private_subnets_config
62+
63+
asset_tags = local.asset_tags
64+
65+
}
66+
67+
module "ex01_nodes" {
68+
source = "../../"
69+
70+
project_id = var.project_id
71+
region = var.region
72+
zones = local.zones
73+
name = "${var.prefix}-hosts"
74+
quantity = 3
75+
instance_labels = merge(local.asset_tags, { "cloudera-role" = "host" })
76+
network_tags = [local.tag_cluster_node]
77+
image = google_compute_image.cluster_image
78+
image_os_user = var.os_user
79+
ssh_public_key = data.tls_public_key.selected.public_key_openssh
80+
subnet_ids = [module.ex01_network.created_private_subnets[0].self_link]
81+
public_ip = true
82+
instance_type = var.instance_type
83+
root_volume = { volume_size = var.root_volume_size }
84+
volumes = var.data_volumes
85+
}
86+
87+
# ------- SSH Keypair -------
88+
# Generate a new private key if needed
89+
resource "tls_private_key" "generated_private_key" {
90+
algorithm = "RSA"
91+
rsa_bits = 4096
92+
}
93+
94+
# Save the generated private key to a file if needed
95+
resource "local_sensitive_file" "pem_file" {
96+
filename = "${var.prefix}-ssh-key.pem"
97+
file_permission = "600"
98+
directory_permission = "700"
99+
content = tls_private_key.generated_private_key.private_key_pem
100+
}
101+
102+
# Load the public key from the correct private key file
103+
data "tls_public_key" "selected" {
104+
private_key_openssh = tls_private_key.generated_private_key.private_key_openssh
105+
}
106+
107+
# ------- Image with MULTI_IP_SUBNET -------
108+
109+
locals {
110+
111+
os_major_version = (
112+
var.os_version != null
113+
? regex("^([0-9]+)", var.os_version)[0]
114+
: "9" # default major is rhel9 if not set
115+
)
21116

22-
# <ENTER_REQUIRED_INPUTS>
117+
# os image mappings
118+
os_family_map = {
119+
rhel = {
120+
project = "rhel-cloud"
121+
family_prefix = "rhel"
122+
}
123+
rocky = {
124+
project = "rocky-linux-cloud"
125+
family_prefix = "rocky-linux"
126+
}
127+
}
23128

129+
gcp_image_project = lookup(
130+
local.os_family_map,
131+
var.os_type,
132+
local.os_family_map["rhel"]
133+
)["project"]
134+
135+
gcp_image_family = "${lookup(
136+
local.os_family_map,
137+
var.os_type,
138+
local.os_family_map["rhel"]
139+
)["family_prefix"]}-${local.os_major_version}"
140+
141+
}
142+
143+
# This data source gets the latest public RHEL/Rocky image to use as a source
144+
data "google_compute_image" "source_image" {
145+
family = local.gcp_image_family
146+
project = local.gcp_image_project
24147
}
148+
149+
# This resource creates your custom image with MULTI_IP_SUBNET to enable /24 mask when
150+
# Postgres reads "samenet" is set in pg_hba.conf
151+
resource "google_compute_image" "cluster_image" {
152+
name = "${var.prefix}-${local.gcp_image_family}"
153+
source_image = data.google_compute_image.source_image.self_link
154+
family = local.gcp_image_family
155+
156+
# Enable MULTI_IP_SUBNET feature
157+
dynamic "guest_os_features" {
158+
for_each = toset([
159+
"MULTI_IP_SUBNET",
160+
"GVNIC",
161+
"IDPF",
162+
"SEV_CAPABLE",
163+
"SEV_LIVE_MIGRATABLE",
164+
"SEV_LIVE_MIGRATABLE_V2",
165+
"SEV_SNP_CAPABLE",
166+
"TDX_CAPABLE",
167+
"UEFI_COMPATIBLE",
168+
"VIRTIO_SCSI_MULTIQUEUE"
169+
])
170+
content {
171+
type = guest_os_features.value
172+
}
173+
}
174+
175+
labels = merge(local.asset_tags, { "guest-os-features" = "multi-ip-subnet" })
176+
177+
lifecycle {
178+
ignore_changes = [source_image]
179+
}
180+
}

‎modules/hosts/examples/ex01-minimal_inputs/terraform.tfvars.sample‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,12 @@
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
1414

15-
# <ENTER_REQUIRED_INPUT_VALUES>
1615

1716
# ------- Global Settings -------
18-
name_prefix = "<ENTER_VALUE>"
17+
prefix = "<ENTER_VALUE>"
1918

2019
# ------- Cloud Settings -------
20+
project_id = "<ENTER_VALUE>"
21+
22+
region = "<ENTER_VALUE>"
23+

0 commit comments

Comments
 (0)