Skip to content

Commit 32374a9

Browse files
odgrimodgrim
authored
Merge branch 'main' into oidc_diagrams
2 parents b66ab26 + 030ea58 commit 32374a9

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+878
-518
lines changed

‎docs/2.0/docs/pipelines/architecture/change-types.md

Lines changed: 355 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,355 @@
1+
---
2+
id: change-types
3+
title: Pull Request Behaviors – Change Types
4+
sidebar_label: Pull Request Behaviors
5+
description: How the Pipelines change-detection engine maps repo changes to Terragrunt commands.
6+
---
7+
8+
# Pull Request Behaviors - Change Types
9+
Pipelines implements a sophisticated change detection system that categorizes different types of infrastructure changes. This system determines which Terragrunt commands to run based on the specific changes detected in a pull request.
10+
11+
## Overview
12+
13+
The change detection system analyzes file changes between two Git references (typically the base and head of a pull request) and categorizes them into specific change types. Each change type triggers different Terragrunt commands and workflows to ensure appropriate infrastructure updates.
14+
15+
## Change Type Categories
16+
17+
:::note
18+
19+
As of July 2025 pipelines emits events using the word Module. This is an outdated term, the updated term in the Terragrunt ecosystem is Unit. The next major release of pipelines will rename these events and this document will be updated at that time.
20+
21+
:::
22+
23+
### Unit Related Changes
24+
25+
#### `ModuleChanged`
26+
**Trigger**: Changes to existing Terragrunt units
27+
- **Files**: `terragrunt.hcl` files in existing directories
28+
- **Behavior**:
29+
- **PR**: `terragrunt plan`
30+
- **Merge**: `terragrunt apply`
31+
- **Use Case**: When you modify configuration values, add new resources, or change existing resources in a unit
32+
33+
#### `ModuleAdded`
34+
**Trigger**: Addition of new Terragrunt units
35+
- **Files**: New `terragrunt.hcl` files in new directories
36+
- **Behavior**:
37+
- **PR**: `terragrunt plan`
38+
- **Merge**: `terragrunt apply`
39+
- **Use Case**: When you create a new infrastructure component or environment
40+
41+
#### `ModuleDeleted`
42+
**Trigger**: Removal of existing Terragrunt units
43+
- **Files**: Deleted `terragrunt.hcl` files
44+
- **Behavior**:
45+
- **PR**: `terragrunt plan -destroy`
46+
- **Merge**: `terragrunt destroy`
47+
- **Use Case**: When you want to remove units
48+
49+
#### `ModuleDriftedAndChanged`
50+
**Trigger**: Units that have both drift and configuration changes
51+
- **Behavior**: Special handling for units that need both drift correction and configuration updates
52+
- **Use Case**: Complex scenarios where infrastructure has drifted from desired state and also has new changes
53+
54+
#### `ModulesAddedOrChanged`
55+
**Trigger**: Aggregated change type for multiple unit modifications
56+
- **Behavior**: Handles scenarios with multiple unit changes
57+
- **Use Case**: Large refactoring or bulk infrastructure updates
58+
59+
### "Envcommon" Changes
60+
61+
#### `EnvCommonChanged`
62+
**Trigger**: Changes to environment common configuration
63+
- **Files**: `.hcl` files in the `_envcommon/` directory
64+
- **Behavior**:
65+
- **PR**: `terragrunt run --all --units-that-include=<changed_file> plan`
66+
- **Merge**: `terragrunt run --all --units-that-include=<changed_file> apply`
67+
- **Use Case**: When you modify shared configuration that affects multiple units
68+
69+
### HCL Configuration Changes
70+
71+
#### `HCLChanged`
72+
**Trigger**: Changes to HCL configuration files
73+
- **Files**: Any `.hcl` file (excluding `terragrunt.hcl` and certain excluded files)
74+
- **Behavior**:
75+
- **PR**: `terragrunt run --all --queue-include-dir=<changed_directory> plan`
76+
- **Merge**: `terragrunt run --all --queue-include-dir=<changed_directory> apply`
77+
- **Use Case**: When you modify shared HCL configurations, variables, or other HCL-based settings and want to propagate those changes to units that read and include those files.
78+
79+
### Account Factory Changes
80+
81+
#### `AccountsRequested`
82+
**Trigger**: A new account is being requested
83+
- **Files**: New `.yml` or `.yaml` files in the `_new-account-requests/` directory
84+
- **Behavior**: Triggers account provisioning workflows
85+
- **Use Case**: When requesting new AWS accounts
86+
87+
#### `AccountsAdded`
88+
**Trigger**: New account directories
89+
- **Files**: New root-level directories (when Account Factory is enabled)
90+
- **Behavior**:
91+
- **PR**: `terragrunt run --all plan`
92+
- **Merge**: `terragrunt run --all apply`
93+
- **Use Case**: Baselining newly provisioned accounts as part of the Account Factory workflow
94+
95+
#### `AccountsChanged`
96+
**Trigger**: Modifications to existing account requests
97+
- **Files**: Modified `.yml` or `.yaml` files in the `_new-account-requests/` directory
98+
- **Behavior**: Triggers account update workflows
99+
- **Use Case**: This should be a very rare use-case where one of the fundamental configuration properties of an account is changed.
100+
101+
#### `AccountsDeleted`
102+
**Trigger**: Removal of account request files
103+
- **Files**: Deleted `.yml` or `.yaml` files in the `_new-account-requests/` directory
104+
- **Behavior**: Triggers account cleanup workflows
105+
- **Use Case**: Deletion of an account that was previously provisioned with Account Factory
106+
107+
### Drift Detection
108+
109+
#### `DriftDetected`
110+
**Trigger**: Changes to drift detection files
111+
- **Files**: `.drift-history.json` files in directories with `terragrunt.hcl`
112+
- **Behavior**:
113+
- **PR**: `terragrunt plan`
114+
- **Merge**: `terragrunt apply`
115+
- **Use Case**: When infrastructure has drifted from the desired state
116+
117+
### General File Changes
118+
119+
#### `FileChanged`
120+
**Trigger**: Changes to files that are read by HCL functions
121+
- **Files**: Any file that might be referenced by Terragrunt units (excluding certain directories)
122+
- **Behavior**:
123+
- **PR**: `terragrunt run --all --queue-include-units-reading=<changed_file> plan`
124+
- **Merge**: `terragrunt run --all --queue-include-units-reading=<changed_file> apply`
125+
- **Use Case**: When you modify files (e.g. shared configurations) that are referenced by Terragrunt units
126+
127+
## Change Detection Order
128+
129+
The change detection system processes changes in a specific order to ensure accurate categorization:
130+
131+
1. **Ignore Files**: Filters out files that should be ignored based on configuration
132+
2. **Account Added**: Detects new account directories
133+
3. **Configuration Files**: Filters configuration files
134+
4. **Environment Common**: Detects changes in `_envcommon/` directory
135+
5. **Account Requests**: Detects changes in `_new-account-requests/` directory
136+
6. **Drift Detection**: Detects drift detection files
137+
7. **Modules**: Detects Terragrunt unit changes
138+
8. **HCL Changes**: Detects general HCL file changes
139+
9. **File Changes**: Detects other file changes
140+
141+
## Special Considerations
142+
143+
### Account Factory Isolation
144+
When `AccountsAdded` changes are detected, they must be isolated in their own pull request. No other infrastructure changes can be included in the same PR.
145+
146+
### Module Change Detection Features
147+
The system supports a feature flag `moduleChangeOnUnitFile` that enables detecting module changes when any file within a Terragrunt unit (directory with `terragrunt.hcl`) is modified.
148+
149+
For more information about how Pipelines handles changes to non-IaC file dependencies, see the [File Dependencies guide](/2.0/docs/pipelines/guides/file-dependencies).
150+
151+
### Stack Values Files
152+
Changes to `terragrunt.values.hcl` files are treated as module changes and trigger the appropriate Terragrunt commands for the containing unit.
153+
154+
## Example Scenarios
155+
156+
### Scenario 1: Adding a New Unit
157+
- **Changes**: New directory with `terragrunt.hcl`
158+
- **Detected Type**: `ModuleAdded`
159+
- **Action**: `terragrunt plan` and `terragrunt apply`
160+
161+
### Scenario 2: Modifying Shared Configuration via the Envcommon pattern
162+
- **Changes**: File in `_envcommon/` directory
163+
- **Detected Type**: `EnvCommonChanged`
164+
- **Action**: Updates across all affected modules that use the changed envcommon file
165+
166+
### Scenario 3: Requesting New Account via Account Factory
167+
- **Changes**: New `.yml` file in `_new-account-requests/`
168+
- **Detected Type**: `AccountsRequested`
169+
- **Action**: Account provisioning workflow
170+
171+
### Scenario 4: Changing a Data File Used by Units
172+
- **Changes**: Modified `.json` file (e.g., `tags.json`, `config.json`) that is read by Terragrunt units
173+
- **Detected Type**: `FileChanged`
174+
- **Action**: `terragrunt run --all --queue-include-units-reading=<changed_file> plan` and `terragrunt run --all --queue-include-units-reading=<changed_file> apply` for all units that read the changed file
175+
176+
This change detection system ensures that pipelines runs the appropriate Terragrunt commands for each type of infrastructure change, maintaining consistency and reliability in your infrastructure deployments.
177+
178+
## User Perspective: What Commands Run for Your Changes
179+
180+
This section explains what happens from your perspective when you make different types of changes in your infrastructure repository.
181+
182+
### Common User Scenarios
183+
184+
#### Adding a New Environment or Module
185+
**What you do**: Create a new directory with a `terragrunt.hcl` file
186+
```bash
187+
mkdir prod/vpc
188+
# Create prod/vpc/terragrunt.hcl with your configuration
189+
```
190+
191+
**What pipelines does**:
192+
- Detects: `ModuleAdded`
193+
- Runs: `terragrunt plan` and `terragrunt apply`
194+
- Result: New infrastructure is created
195+
196+
#### Modifying Existing Infrastructure
197+
**What you do**: Edit an existing `terragrunt.hcl` file
198+
```bash
199+
# Edit prod/vpc/terragrunt.hcl to change instance type
200+
```
201+
202+
**What pipelines does**:
203+
- Detects: `ModuleChanged`
204+
- Runs: `terragrunt plan` and `terragrunt apply`
205+
- Result: Infrastructure is updated to match your changes
206+
207+
#### Removing Infrastructure
208+
**What you do**: Delete a `terragrunt.hcl` file
209+
```bash
210+
rm -rf staging/old-service/
211+
```
212+
213+
**What pipelines does**:
214+
- Detects: `ModuleDeleted`
215+
- Runs: `terragrunt destroy`
216+
- Result: Infrastructure is removed
217+
218+
#### Changing Shared Configuration
219+
**What you do**: Modify files in the `_envcommon/` directory
220+
```bash
221+
# Edit _envcommon/network.hcl to change default VPC settings
222+
```
223+
224+
**What pipelines does**:
225+
- Detects: `EnvCommonChanged`
226+
- Runs: `terragrunt plan` and `terragrunt apply` for all modules that reference the changed configuration
227+
- Result: All affected environments are updated with the new shared settings
228+
229+
#### Requesting a New Account
230+
**What you do**: Create a new YAML file in `_new-account-requests/`
231+
```bash
232+
# Create _new-account-requests/account-new-team.yml
233+
```
234+
235+
**What pipelines does**:
236+
- Detects: `AccountsRequested`
237+
- Runs: Account provisioning workflow
238+
- Result: New AWS account is created and configured
239+
240+
#### Adding a New Account Environment
241+
**What you do**: Create a new root-level directory (when account factory is enabled)
242+
```bash
243+
mkdir new-team-account/
244+
# Add terragrunt.hcl files for the new account
245+
```
246+
247+
**What pipelines does**:
248+
- Detects: `AccountsAdded`
249+
- Runs: Account setup and initialization workflows
250+
- Result: New account environment is created and configured
251+
252+
#### Modifying HCL Configuration Files
253+
**What you do**: Edit any `.hcl` file (not `terragrunt.hcl`)
254+
```bash
255+
# Edit common/variables.hcl to add new variables
256+
```
257+
258+
**What pipelines does**:
259+
- Detects: `HCLChanged`
260+
- Runs: Appropriate Terragrunt commands based on context
261+
- Result: Changes are applied to affected modules
262+
263+
### What Commands Are Actually Run
264+
265+
#### For Unit Changes (`ModuleChanged`, `ModuleAdded`)
266+
**Pull Request:**
267+
```bash
268+
terragrunt plan
269+
```
270+
**After Merge:**
271+
```bash
272+
terragrunt apply
273+
```
274+
275+
#### For Unit Deletion (`ModuleDeleted`)
276+
**Pull Request:**
277+
```bash
278+
terragrunt plan -destroy
279+
```
280+
**After Merge:**
281+
```bash
282+
terragrunt destroy
283+
```
284+
285+
#### For EnvCommon Changes (`EnvCommonChanged`)
286+
**Pull Request:**
287+
```bash
288+
terragrunt run --all --units-that-include=_envcommon/changed-file.hcl plan
289+
```
290+
**After Merge:**
291+
```bash
292+
terragrunt run --all --units-that-include=_envcommon/changed-file.hcl apply
293+
```
294+
295+
#### For HCL Configuration Changes (`HCLChanged`)
296+
**Pull Request:**
297+
```bash
298+
terragrunt run --all --queue-include-dir=path/to/changed/directory plan
299+
```
300+
**After Merge:**
301+
```bash
302+
terragrunt run --all --queue-include-dir=path/to/changed/directory apply
303+
```
304+
305+
#### For File Changes (`FileChanged`)
306+
**Pull Request:**
307+
```bash
308+
terragrunt run --all --queue-include-units-reading=path/to/changed/file plan
309+
```
310+
**After Merge:**
311+
```bash
312+
terragrunt run --all --queue-include-units-reading=path/to/changed/file apply
313+
```
314+
315+
### Understanding the Workflow
316+
317+
1. **You make changes** in your Git repository
318+
2. **You create a pull request** with your changes
319+
3. **Pipelines analyzes** the changes and determines the change types
320+
4. **Pipelines runs** the appropriate Terragrunt commands
321+
5. **You review** the plan output in the pull request
322+
6. **You approve** the changes (if required)
323+
7. **Pipelines applies** the changes to your infrastructure
324+
325+
### Command Types: `run` vs `run --all`
326+
327+
Pipelines uses two main types of Terragrunt commands:
328+
329+
#### Single Unit Commands (`plan`, `apply`, `destroy`)
330+
- **Used for**: Direct unit changes (`ModuleChanged`, `ModuleAdded`, `ModuleDeleted`, `DriftDetected`)
331+
- **Scope**: Operates on a single Terragrunt unit (directory with `terragrunt.hcl`)
332+
- **Example**: `terragrunt plan`
333+
334+
#### Multi-Unit Commands (`run --all`)
335+
- **Used for**: Changes that affect multiple units (`EnvCommonChanged`, `HCLChanged`, `FileChanged`, `AccountsAdded`)
336+
- **Scope**: Operates across multiple Terragrunt units that are affected by the change
337+
- **Key Flags**:
338+
- `--units-that-include=<file>`: Runs on all units that include the specified file
339+
- `--queue-include-dir=<directory>`: Runs on all units in the specified directory
340+
- `--queue-include-units-reading=<file>`: Runs on all units that read the specified file
341+
- **Example**: `terragrunt run --all --units-that-include=_envcommon/network.hcl plan`
342+
343+
#### Key Differences
344+
- **Single unit commands** are more targeted and efficient for direct unit changes
345+
- **Multi-unit commands** ensure that all affected units are updated when shared configurations change
346+
- **Multi-unit commands** use Terragrunt's dependency resolution to determine which units need updates
347+
348+
### Best Practices
349+
350+
- **Isolate account additions**: When adding new accounts, keep them in separate pull requests
351+
- **Review plans carefully**: Always review the `terragrunt plan` output before approving
352+
- **Use descriptive commit messages**: This helps with change tracking and debugging
353+
- **Test in staging first**: Make changes in staging environments before applying to production
354+
355+
This system ensures that your infrastructure changes are applied consistently and safely, with appropriate validation at each step.

‎docs/2.0/docs/pipelines/installation/addinggitlabrepo.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ First, you'll need to install [mise](https://mise.jdx.dev/), a powerful environm
118118

119119
5. `cd` to the root of <CustomizableValue id="REPOSITORY_NAME" /> where you wish to install Gruntwork Pipelines. Run the boilerplate tool to generate your repository structure:
120120
```bash
121-
boilerplate --template-url "[email protected]:gruntwork-io/terraform-aws-architecture-catalog.git//templates/gitlab-pipelines-infrastructure-live-root/?ref=v2.13.0" --output-folder . --var-file vars.yaml --non-interactive
121+
boilerplate --template-url "[email protected]:gruntwork-io/terraform-aws-architecture-catalog.git//templates/gitlab-pipelines-infrastructure-live-root/?ref=v3.1.0" --output-folder . --var-file vars.yaml --non-interactive
122122
```
123123

124124
If you encounter SSH issues, verify your SSH access to GitHub:

0 commit comments

Comments
 (0)