Skip to content

Commit 88b0ea6

Browse files
justin808justin808
committed
Docs
1 parent 24ef3c2 commit 88b0ea6

File tree

3 files changed

+350
-13
lines changed

3 files changed

+350
-13
lines changed

‎README.md‎

Lines changed: 245 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,256 @@
1-
# README
1+
# Terraform Google Cloud with Rails 8 App and Kamal v2
22

3-
This README would normally document whatever steps are necessary to get the
4-
application up and running.
3+
This repository contains a Rails 8 app with Terraform files for deploying the app on Google Cloud. The app uses Kamal v2 for deployment.
54

6-
Things you may want to cover:
5+
## Consulting Help Available
6+
* Would like assistance with using Kamal, Docker, Terraform, or Google Cloud?
7+
* Do you have concerns about the performance, security, and observability of your current deployments?
8+
* Are you interested in optimizing your deployment process to get Heroku features without the high costs?
79

8-
* Ruby version
10+
If so, please [email me](mailto:[email protected]) or [book a time](https://meetings.hubspot.com/justingordon/30-minute-consultation).
911

10-
* System dependencies
12+
Check out [ShakaCode's Infrastructure Optimization Services](https://www.shakacode.com/services/it-infrastructure-optimization/).
1113

12-
* Configuration
14+
### Other Resources
15+
* [Click to join **React + Rails Slack** to chat with Justin](https://reactrails.slack.com/join/shared_invite/enQtNjY3NTczMjczNzYxLTlmYjdiZmY3MTVlMzU2YWE0OWM0MzNiZDI0MzdkZGFiZTFkYTFkOGVjODBmOWEyYWQ3MzA2NGE1YWJjNmVlMGE).
1316

14-
* Database creation
17+
## Kamal Basics
1518

16-
* Database initialization
19+
In order to use Kamal, you should try to understand what's going on from first principles.
1720

18-
* How to run the test suite
21+
Kamal is a CLI that uses configuration files to orchestrate commands for Docker deployment on remote machines.
1922

20-
* Services (job queues, cache servers, search engines, etc.)
23+
Docs are nice. But lots are not in the docs. That's OK for 2 reasons:
24+
1. Kamal gives lots of output to the command line on what's running
25+
2. You have the source code, and you can ask AI for help.
2126

22-
* Deployment instructions
27+
## Why use Rails 8 with Terraform, Google Cloud, and Kamal v2?
2328

24-
* ...
29+
1. **Infrastructure as Code**: Terraform allows you to define your infrastructure in code, making it easier to manage and scale. By using Terraform, you don't have to configure anything in the Google Cloud Console manually.
30+
2. **Consistent Deployments**: With Terraform, you can ensure that your infrastructure is consistent across all environments. This helps in reducing errors and ensuring that your app runs smoothly.
31+
3. Scripts to stand-up and tear-down the infrastructure: The Terraform files in this repository include scripts to create and destroy the infrastructure. This makes it easy to spin up a new environment for testing and tear it down when you're done. Don't pay for resources you're not using!
32+
4. **Kamal v2**: Kamal v2 is a lightweight deployment tool that makes it easy to deploy Rails apps to Google Cloud. It handles the deployment process for you, so you don't have to worry about setting up Kubernetes clusters or managing containers.
33+
5. **Rails 8**: Rails 8 is the latest version of the popular Ruby on Rails framework. It comes with many new features and improvements that make it easier to build web applications.
34+
6. **Google Cloud**: Google Cloud is a powerful cloud platform that offers a wide range of services for building and deploying applications. By using Google Cloud, you can take advantage of its scalability, reliability, and security features.
35+
36+
## Requirements
37+
38+
1. [Google Cloud SDK](https://cloud.google.com/sdk/docs/install) with a gcloud account.
39+
2. [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli).
40+
3. Some domain name where you can add an A record to point to your server's IP address.
41+
3. Docker
42+
4. Ruby 3.3.4
43+
44+
## Setup
45+
1. Clone this repository.
46+
2. Create the Master Key: Run the following command to generate the development and test `master.key` and save it in the correct location:
47+
```bash
48+
echo "94cd5f24badf3102a4c6a09eb4a4a516" > config/master.key
49+
```
50+
Or just make a brand new one for development and test with `rails credentials:edit`.
51+
3. Install the required gems:
52+
```bash
53+
bundle install
54+
```
55+
4. Edit the `terraform-gcloud/variables.tf` file with your project details. You need to create a Google Cloud "project" for this demo.
56+
5. Edit the `config/deploy.yml` file:
57+
1. Set the `proxy.host` domain name (currently set to `gcp.kamaltutorial.com`)
58+
2. Change the `registry.username`
59+
3. Change the `ssh.user` to your username.
60+
61+
See [docs/prerequisites.md](docs/prerequisites.md) for more detailed setup instructions.
62+
63+
## Rails 8 Defaults
64+
This Rails 8 example app differs as little as possible from the default Rails 8 app. The main differences are:
65+
1. Terraform setup in the `terraform-gcloud` directory. Terraform is super nice because you can follow the example with minimal work to get your Rails app running on Google Cloud. For a non-tutorial application, you'd put the Terraform files in a separate git repo.
66+
2. The docker and Kamal setup has minimal changes.
67+
68+
## Secrets and Credentials
69+
Note:
70+
1. I originally created the example to work with 1Password. However, given that a GCP account is required, I decided to use the GCP Secret Manager.
71+
2. To demonstrate best practices for handling secrets, we will NOT use rails credentials for production secrets. Instead, we will use the Google Cloud Secret Manager. Rails credentials are great for non-production environments.
72+
73+
### Google Cloud Secret Manager
74+
75+
1. Open the [Google Cloud Secret Manager](https://console.cloud.google.com/security/secret-manager).
76+
2. Ensure that you have the correct project selected.
77+
3. Click on "Create Secret" to add a new secret.
78+
4. Add the following 3 secrets:
79+
- `KAMAL_REGISTRY_PASSWORD`: The password for the Docker registry.
80+
- `DB_PASSWORD`: The password for the database, as you like.
81+
- `SECRET_KEY_BASE`: Generate with `rails secret`.
82+
83+
### `deploy.yml`
84+
Note that the `deploy.yml` already has:
85+
```yaml
86+
env:
87+
secret:
88+
- DB_PASSWORD
89+
- SECRET_KEY_BASE
90+
```
91+
92+
### `.kamal/secrets`
93+
The `.kamal/secrets` already has this code. Conveniently, you don't need to duplicate your GCP PROJECT_ID.
94+
```
95+
PROJECT_ID=$(cd ./terraform-gcloud && terraform output -raw project_id)
96+
DB_PASSWORD=$(gcloud secrets versions access latest --secret=DB_PASSWORD --project="$PROJECT_ID")
97+
SECRET_KEY_BASE=$(gcloud secrets versions access latest --secret=SECRET_KEY_BASE --project="$PROJECT_ID")
98+
```
99+
100+
### config/database.yml
101+
Note that the `deploy.yml` file ensures that the DB_HOST and DB_PASSWORD are set in the environment. The DB_HOST is set to a seemingly magic value of
102+
```yaml
103+
DB_HOST: 172.18.0.1
104+
```
105+
106+
This is the value that Docker uses to refer to the host machine from within a Docker container. This is because the database is referenced as localhost on the host machine, not localhost in the Docker container. This way, we don't need an IP address for the database. The terraform script sets up the database to work like this by installing `cloud_sql_proxy` and running it.
107+
108+
109+
The `config/database.yml` file already has the following code, which needs to correspond to the secrets and the setup of the databases in `terraform-gcloud/main.tf`:
110+
```yaml
111+
default: &default
112+
host: <%= ENV.fetch("DB_HOST", "localhost") %>
113+
password: <%= ENV.fetch("DB_PASSWORD", "password") %>
114+
production:
115+
primary: &primary_production
116+
<<: *default
117+
database: rails_kamal_demo_production
118+
username: rails_user
119+
cache:
120+
<<: *primary_production
121+
database: rails_kamal_demo_production_cache
122+
migrations_paths: db/cache_migrate
123+
queue:
124+
<<: *primary_production
125+
database: rails_kamal_demo_production_queue
126+
migrations_paths: db/queue_migrate
127+
cable:
128+
<<: *primary_production
129+
database: rails_kamal_demo_production_cable
130+
migrations_paths: db/cable_migrate
131+
```
132+
133+
## Database Setup and Migrations
134+
The database is automatically created and migrated when the Rails app is deployed. This is done via the [bin/docker-entrypoint.sh](bin/docker-entrypoint.sh) script which calls `rails db:prepare`.
135+
136+
Note, the initial deployment will fail because the database schema needs to be created. This is expected. Just run `bundle exec kamal deploy` again after the initial setup.
137+
138+
## Automated Deployment
139+
Run `bin/terraform-gcloud/bin/stand-up` to create the infrastructure on Google Cloud and deploy the Rails app using Kamal v2.
140+
141+
This script has some useful features:
142+
143+
1. It creates the infrastructure on Google Cloud using Terraform.
144+
2. It updates the deploy.yml config file to reflect the IP address of the server and makes a longer deployment timeout.
145+
3. You get a chance to add an A record to your domain name pointing to the IP address.
146+
```
147+
Outputs:
148+
db_primary_name = "rails_kamal_demo_production"
149+
db_user = "rails_user"
150+
instance_ip = "34.59.165.111"
151+
project_id = "kamal-demo-444506"
152+
✅ New IP acquired: 34.59.165.111
153+
154+
=== Updating Configuration ===
155+
Old timeout was set to 30 seconds
156+
✅ Updated deploy.yml with new IP and timeout
157+
✅ Timeout is now set to 120 seconds
158+
159+
=== Verifying DNS Configuration ===
160+
Edit DNS for `kamaltutorial.com`: Update or add a DNS Type `A` record, Name: `gcp`, Value: `34.59.165.111`
161+
Press return to check DNS (or Ctrl-C to exit):
162+
Checking DNS...
163+
✅ DNS verification successful! gcp.kamaltutorial.com → 34.59.165.111
164+
```
165+
4. It deploys the Rails app using Kamal v2.
166+
5. Visit your domain name in the browser to see your Rails app running on Google Cloud!
167+
168+
## Tear Down
169+
170+
When you're done, run `bin/terraform-gcloud/bin/tear-down` to destroy the infrastructure on Google Cloud (and save any costs!)
171+
172+
The `tear-down` script has some useful features:
173+
1. Ensures calling `kamal app stop` or else terraform cannot destroy the database.
174+
2. Call `terraform destroy` to destroy the infrastructure on Google Cloud.
175+
176+
## Step by Step
177+
To get a sense of the basics of Terraform and Kamal v2, follow these steps.
178+
179+
### Terraform Setup
180+
First, ensure that you can run `terraform` commands to create the infrastructure on Google Cloud.
181+
182+
1. Install the Google Cloud SDK and Terraform.
183+
2. Run terraform commands from `cd terraform-gcloud`.
184+
2. Update the `terraform-gcloud/variables.tf` file with your project details as described above.
185+
3. Run `terraform init` in the `terraform-gcloud` directory to initialize the Terraform configuration.
186+
4. Run `terraform plan` to see the changes that Terraform will make to your infrastructure.
187+
5. Run `terraform apply` to create the infrastructure on Google Cloud. This takes about 10 minutes mainly due to provisioning the database. Note that the output will include the IP address of the server. You need this for 2 reasons:
188+
1. To add an A record to your domain name.
189+
2. To update the `config/deploy.yml` file with the IP address for your server.
190+
6. Run `terraform destroy` to tear down the infrastructure when you're done (after practicing the Kamal deployment)
191+
192+
### Kamal v2 Deployment
193+
Next, ensure that you can deploy the Rails app using Kamal v2.
194+
195+
1. Run `./bin/kamal setup` to set up the Kamal v2 configuration. Notice the error that the health checks failed. This is expected because the database needs to be created and migrated.
196+
2. Unless you change the `deploy.yml` file to have a longer `deploy_timeout`, you'll need to run `./bin/kamal deploy` a second time, because the database needs to be created and migrated. The `stand-up` script does this for you.
197+
3. Visit your domain name in the browser to see your Rails app running on Google Cloud!
198+
199+
## Troubleshooting
200+
If you encounter any issues during the deployment process, here are some common troubleshooting steps.
201+
202+
### Execution Flow
203+
First, it's important to understand the execution context of when running commands.
204+
205+
1. **Your Local Machine (or CI Machine):**
206+
* Runs commands like kamal deploy, which connects to the host machine via SSH.
207+
2. **Host Machine:**
208+
* Receives commands from Kamal and executes them, managing the Docker runtime environment.
209+
* Temporarily hosts deployment files (e.g., hook scripts) and runs them within Docker containers.
210+
* `ssh user@host` to get a shell on the host machine.
211+
3. **Docker Machine (Containers):**
212+
* The host machine runs Docker containers for the Rails app and the Kamal proxy.
213+
* The app runs here. Commands like bin/rails db:migrate execute inside these containers.
214+
* `docker ps` to see the running containers.
215+
* `docker logs CONTAINER_ID` to see the logs of a container.
216+
* `docker exec -it CONTAINER_ID bash` to get a shell in a container.
217+
218+
### Troubleshooting Steps
219+
1. First, read the console messages very carefully and look for the first error message. This is often the most important clue. If there's a health check timeout, it might be due to the failure to run migrations quickly enough, and then you simply need to run `./bin/kamal deploy` again.
220+
2. Check the logs for the Rails app and the Kamal proxy to see if there are any error messages. You can do this with the command `./bin/kamal logs`.
221+
3. If there is trouble with the database, then you won't get far because your default ENTRYPOINT `bin/docker-entrypoint` will fail. You need to first find the CONTAINER_ID of the failing container. You can do this by:
222+
1. ssh to the host machine, like `ssh <username>@<ip_address>`.
223+
2. Run `docker ps` and export a value for CONTAINER_ID. Then you can run `docker logs $CONTAINER_ID` to see the logs.
224+
3. You can run `docker run -it --entrypoint bash $CONTAINER_ID` to get a shell and then run `bin/docker-entrypoint` to see what's going on. This skips your default ENTRYPOINT.
225+
226+
### Debugging Tips with AI Tools
227+
228+
Next, use AI tools to help you debug. A prompt like this is very helpful. Substitute your host IP address and the Rails container ID.
229+
230+
```
231+
I'm using Kamal v2. Double check you are not giving me answers for Kamal v1
232+
233+
When you give me commands, tell me which execution context: Local machine, host machine, or docker container.
234+
235+
The remote host IP is 34.122.124.21.
236+
237+
The rails app docker container id is f41ea810b98f
238+
```
239+
240+
To get a good understanding of what's going on, you can run the following commands:
241+
242+
```
243+
Walk me through the output, one command at a time for the following output of kamal v2.
244+
Don't analyze everything. Go one command at a time.
245+
Only analyze the "Running" lines, one at a time.
246+
247+
<THEN PASTE COMMAND AND OUTPUT>
248+
```
249+
250+
## Unaddressed concerns
251+
1. Machine monitoring.
252+
* What happens when disk runs out of space?
253+
* What happens if memory maxes out?
254+
* What happens if CPU maxes out?
255+
* No Auto-Scaling
256+
2. Machine must have about twice as much memory as new app needs to run with old app during deployment. Why pay for all that extra memory when not needed?

‎docs/prerequisites.md‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Prerequisites
2+
3+
## GCP Account
4+
5+
6+
## SSH Access for GCP
7+
8+
To set up SSH with Google Cloud Platform (GCP) for use with Terraform, follow these steps:
9+
10+
1. **Generate SSH Keys**:
11+
If you don't already have an SSH key pair, generate one using the following command:
12+
```bash
13+
ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa
14+
```
15+
16+
2. **Add SSH Key to GCP**:
17+
Add your public SSH key to your GCP project metadata. This allows you to SSH into instances created by Terraform.
18+
```bash
19+
gcloud compute project-info add-metadata --metadata ssh-keys="$(whoami):$(cat ~/.ssh/id_rsa.pub)"
20+
```
21+
22+
3. **Configure Terraform**:
23+
Ensure your `variables.tf` file includes the `ssh_user` variable:
24+
```terraform
25+
variable "ssh_user" {
26+
description = "The SSH username to access the instance"
27+
type = string
28+
default = "your-username" # Replace with your SSH username
29+
}
30+
```
31+
32+
4. **Reference SSH Key in Terraform**:
33+
In your `main.tf` file, configure the `google_compute_instance` resource to use the SSH key (in source code):
34+
```terraform
35+
resource "google_compute_instance" "rails_app" {
36+
metadata = {
37+
ssh-keys = "${var.ssh_user}:${file("~/.ssh/id_rsa.pub")}"
38+
}
39+
}
40+
```
41+
42+
5. **SSH into the Instance**:
43+
Once the instance is created, you can SSH into it using:
44+
```bash
45+
gcloud compute ssh your-username@instance-name --zone=us-central1-a
46+
```
47+
48+
Replace `your-username` and `instance-name` with your actual SSH username and instance name.
49+
50+
Or you can use the `ssh` command directly:
51+
```bash
52+
ssh your-username@instance-ip
53+
```
54+
The instance-ip can be found in the `config/deploy.yml` which is updated after the Terraform setup done via `terraform-gcloud/bin/stand-up`.

‎terraform-gcloud/README.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Google Cloud Terraform with Rails App
2+
3+
## Requirements
4+
1. Update variables in `terraform-gcloud/variables.tf` with your project details.
5+
2. Ensure you have the [Google Cloud SDK](https://cloud.google.com/sdk/docs/install) installed with a gcloud account.
6+
3. Install [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli).
7+
8+
## Should Terraform files be in the same repository as a Rails app?
9+
10+
Yes, for a demo or small-scale example, you can mix Terraform files with your Rails app directory for simplicity. However, it’s important to keep a few best practices in mind to maintain clarity and scalability:
11+
12+
Suggested Setup for Simplicity:
13+
1. Create a terraform Directory:
14+
• Place all your Terraform files in a subdirectory within the Rails app, such as terraform/.
15+
• Example structure:
16+
17+
```
18+
my_rails_app/
19+
├── app/
20+
├── config/
21+
├── db/
22+
├── terraform/
23+
│ ├── main.tf
24+
│ ├── variables.tf
25+
│ ├── outputs.tf
26+
│ └── .terraform.lock.hcl
27+
├── Gemfile
28+
└── ...
29+
```
30+
31+
2. Isolate State Files:
32+
• Ensure that your terraform state files (e.g., terraform.tfstate) are either excluded from version control (via .gitignore) or stored remotely (e.g., in an S3 bucket for AWS setups).
33+
• Example .gitignore entry:
34+
35+
terraform/.terraform/
36+
terraform/terraform.tfstate*
37+
38+
39+
3. Use a Separate Namespace:
40+
• Use descriptive names for your Terraform resources to avoid conflicts if the Rails app scales or integrates with other infrastructure.
41+
4. Add Documentation:
42+
• Include a README.md in the terraform/ directory to explain how the Terraform setup interacts with the Rails app.
43+
44+
When to Separate Terraform Files:
45+
46+
For production-grade applications or long-term maintenance, it’s better to keep infrastructure as code (Terraform) in a separate repository or infrastructure-specific directory. This separation ensures:
47+
• Clear Boundaries: Between application code and infrastructure management.
48+
• Ease of Use: For teams managing either Rails development or infrastructure.
49+
• Scalability: Simplifies infrastructure scaling and reusability.
50+
51+
If this demo involves terraform working closely with Rails deployment (e.g., provisioning databases, load balancers, etc.), you can tightly couple them temporarily, but plan for eventual decoupling as the setup grows.

0 commit comments

Comments
 (0)