Allow unit values file to be written as json. #4595
Conversation
Co-authored-by: Ross Strickland <[email protected]>
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
📝 WalkthroughWalkthroughSupport for generating and consuming stack values files in JSON format has been added alongside the existing HCL format. This is controlled by a new Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CLI
participant StackLogic
participant FileSystem
User->>CLI: terragrunt stack generate [--json-values]
CLI->>StackLogic: Generate stack values (opts.StackValuesJSON)
alt opts.StackValuesJSON == true
StackLogic->>FileSystem: Write terragrunt.values.json
else opts.StackValuesJSON == false
StackLogic->>FileSystem: Write terragrunt.values.hcl
end
User->>CLI: terragrunt stack plan/apply
CLI->>StackLogic: Read stack values
alt terragrunt.values.json exists
StackLogic->>FileSystem: Read terragrunt.values.json
else
StackLogic->>FileSystem: Read terragrunt.values.hcl
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Note ⚡️ Unit Test Generation is now available in beta!Learn more here, or try it out under "Finishing Touches" below. ✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (9)
options/options.go (1)
308-310: Consider adding explicit struct-tags / comment for consistencyThe new field is fine functionally (zero-value false keeps existing behaviour), but every other boolean CLI flag exposed on
TerragruntOptionscarries at least ajson/mapstructuretag or an explanatory doc-comment describing the default.
Recommend adding something similar so downstream reflection (e.g. when serialising options or binding env vars) behaves consistently.- // StackValuesJSON writes stack values files in JSON format instead of HCL. - StackValuesJSON bool + // StackValuesJSON determines whether stack values files are written as JSON (`terragrunt.values.json`) + // instead of the default HCL (`terragrunt.values.hcl`). Defaults to false. + StackValuesJSON bool `mapstructure:"stack_values_json" json:"stack_values_json,omitempty"`docs/_docs/04_reference/02-cli-options.md (1)
214-221: Add precedence note & mirror change indocs-starlight.
- Users may wonder which file wins when both
terragrunt.values.hclandterragrunt.values.jsonare present. A short sentence clarifying the precedence (JSON vs HCL) will remove ambiguity.- Per the migration notice, every addition under
docs/**must be duplicated in the newdocs-starlight/**tree. Please verify that an equivalent--json-valuessection exists there.docs-starlight/src/content/docs/02-features/02-stacks.mdx (2)
181-185: Separate the two possible filenames for better readabilityCollapsing the two alternatives onto one line with Markdown underscores renders as italic text and makes the file tree harder to scan. Listing each variant on its own line—even if one is “optional”—keeps the tree visually aligned and avoids Markdown side-effects.
- - terragrunt.values.hcl _or terragrunt.values.json with --json-values_ + - terragrunt.values.hcl # default + - terragrunt.values.json # generated with `--json-values`Apply the same change to the
databaseentry below for consistency.
188-201: Minor wording & heading-style tweaks
- In headings we usually use title-case (“Values File Formats” → “Values file formats”) to match the surrounding docs.
- “contain the same values data” reads awkwardly—“contain the same data” is simpler.
-#### Values File Formats +#### Values file formats @@ -Both formats are functionally equivalent and contain the same values data. +Both formats are functionally equivalent and contain the same data.Optional: add an inline link to the CLI reference for
--json-valuesto aid discoverability.docs/_docs/04_reference/04-config-blocks-and-attributes.md (5)
1742-1744: Cross-link the new flag and clarify default behaviourNice addition. To make this self-contained, please:
- Add an inline link to the
--json-valuesoption in the CLI-options reference (02-cli-options.md) so that readers can jump straight to the flag description.- State explicitly that, when the flag is omitted, Terragrunt always falls back to
terragrunt.values.hcleven if a.jsonfile already exists (current wording leaves the precedence ambiguous).
1752-1759: Minor grammar / flowConsider “The generated values file contains…” (present tense) instead of “will contain”, which reads better in reference docs.
1762-1770: Usejsoncfence or drop the leading//commentThe leading
// .terragrunt-stack/...line is invalid JSON and causes red highlighting when rendered with ajsonfence. Either:-```json -// .terragrunt-stack/vpc/terragrunt.values.json +```jsonc +// .terragrunt-stack/vpc/terragrunt.values.jsonor move the path comment outside the fenced block.
1772-1781: Tiny wording fix“leverage those values via
valuesvariables regardless of format” →
“leverage those values via thevaluesvariables regardless of the file format”
1895-1896: Duplication with unit sectionThis bullet is nearly identical to the one in the unit section. Consider extracting to a reusable snippet or ensure any future edits touch both places to avoid drift.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
cli/commands/stack/cli.go(2 hunks)config/config_helpers.go(1 hunks)config/stack.go(6 hunks)config/stack_test.go(3 hunks)docs-starlight/src/content/docs/02-features/02-stacks.mdx(1 hunks)docs/_docs/02_features/02-stacks.md(1 hunks)docs/_docs/04_reference/01-configuration.md(1 hunks)docs/_docs/04_reference/02-cli-options.md(1 hunks)docs/_docs/04_reference/04-config-blocks-and-attributes.md(5 hunks)options/options.go(1 hunks)test/integration_stacks_test.go(1 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.go
⚙️ CodeRabbit Configuration File
Review the Go code for quality and correctness. Make sure that the Go code follows best practices, is performant, and is easy to understand and maintain.
Files:
options/options.goconfig/config_helpers.gocli/commands/stack/cli.gotest/integration_stacks_test.goconfig/stack.goconfig/stack_test.go
docs/**/*.md
⚙️ CodeRabbit Configuration File
Review the documentation for clarity, grammar, and spelling. Make sure that the documentation is easy to understand and follow. There is currently a migration underway from the Jekyll based documentation in
docsto the Starlight + Astro based documentation indocs-starlight. Whenever changes are made to thedocsdirectory, ensure that an equivalent change is made in thedocs-starlightdirectory to keep thedocs-starlightdocumentation accurate.
Files:
docs/_docs/04_reference/01-configuration.mddocs/_docs/02_features/02-stacks.mddocs/_docs/04_reference/04-config-blocks-and-attributes.mddocs/_docs/04_reference/02-cli-options.md
docs-starlight/**/*.md*
⚙️ CodeRabbit Configuration File
Review the documentation for clarity, grammar, and spelling. Make sure that the documentation is easy to understand and follow. There is currently a migration underway from the Jekyll based documentation in
docsto the Starlight + Astro based documentation indocs-starlight. Make sure that thedocs-starlightdocumentation is accurate and up-to-date with thedocsdocumentation, and that any difference between them results in an improvement in thedocs-starlightdocumentation.
Files:
docs-starlight/src/content/docs/02-features/02-stacks.mdx
🧠 Learnings (5)
📚 Learning: in terragrunt projects, all `.hcl` files can be assumed to be terragrunt configurations by default, ...
Learnt from: yhakbar
PR: gruntwork-io/terragrunt#3868
File: docs-starlight/patches/@astrojs%[email protected]:33-33
Timestamp: 2025-02-10T23:20:04.295Z
Learning: In Terragrunt projects, all `.hcl` files can be assumed to be Terragrunt configurations by default, with specific exceptions like `.terraform.lock.hcl` that need explicit handling.
Applied to files:
docs/_docs/04_reference/01-configuration.mdconfig/config_helpers.godocs/_docs/02_features/02-stacks.mddocs-starlight/src/content/docs/02-features/02-stacks.mdxdocs/_docs/04_reference/04-config-blocks-and-attributes.mddocs/_docs/04_reference/02-cli-options.mdconfig/stack.go
📚 Learning: the project uses a custom error package `github.com/gruntwork-io/terragrunt/internal/errors` which p...
Learnt from: levkohimins
PR: gruntwork-io/terragrunt#3723
File: cli/commands/stack/action.go:160-160
Timestamp: 2025-02-10T13:36:19.542Z
Learning: The project uses a custom error package `github.com/gruntwork-io/terragrunt/internal/errors` which provides similar functionality to `fmt.Errorf` but includes stack traces. Prefer using this package's error functions (e.g., `errors.Errorf`, `errors.New`) over the standard library's error handling.
Applied to files:
docs/_docs/04_reference/01-configuration.mddocs/_docs/02_features/02-stacks.mddocs-starlight/src/content/docs/02-features/02-stacks.mdxdocs/_docs/04_reference/04-config-blocks-and-attributes.mddocs/_docs/04_reference/02-cli-options.mdtest/integration_stacks_test.goconfig/stack.go
📚 Learning: avoid shadowing imported packages with local variables in go code, such as using a variable named `f...
Learnt from: yhakbar
PR: gruntwork-io/terragrunt#4169
File: cli/commands/hcl/validate/cli.go:29-60
Timestamp: 2025-04-17T13:02:28.098Z
Learning: Avoid shadowing imported packages with local variables in Go code, such as using a variable named `flags` when the `github.com/gruntwork-io/terragrunt/cli/flags` package is imported. Use more specific variable names like `flagSet` instead.
Applied to files:
config/config_helpers.gocli/commands/stack/cli.gotest/integration_stacks_test.goconfig/stack.go
📚 Learning: the terragruntconfig struct in config/config.go does contain an engine field that's used to store en...
Learnt from: partcyborg
PR: gruntwork-io/terragrunt#3974
File: config/config_partial.go:448-456
Timestamp: 2025-03-06T23:44:09.413Z
Learning: The TerragruntConfig struct in config/config.go does contain an Engine field that's used to store engine configuration data.
Applied to files:
config/config_helpers.go
📚 Learning: constants defined in one go file are accessible from other files in the same package without explici...
Learnt from: wakeful
PR: gruntwork-io/terragrunt#4491
File: test/integration_test.go:4265-4265
Timestamp: 2025-07-03T22:05:07.356Z
Learning: Constants defined in one Go file are accessible from other files in the same package without explicit imports. When reviewing Go code, consider the package-level scope rather than just the individual file scope.
Applied to files:
config/stack.go
🔇 Additional comments (25)
docs/_docs/04_reference/01-configuration.md (1)
182-186: Mirror the doc update indocs-starlightPer the repo-wide guideline, any change in the legacy
docs/tree must be duplicated indocs-starlight/.
Please replicate the “# or terragrunt.values.json with --json-values” comment in the equivalent MDX file so both doc sets stay in sync.config/config_helpers.go (1)
738-738: LGTM! Clean extension to support JSON values files.The addition of
|| strings.HasSuffix(targetConfig, valuesFileJSON)correctly extends the existing condition to recognize JSON values files alongside HCL values files. This change aligns perfectly with the PR objective of supporting both formats and maintains backward compatibility.cli/commands/stack/cli.go (2)
19-19: LGTM! Constant follows existing naming conventions.The
JSONValuesFlagNameconstant is properly defined and follows the existing naming pattern used for other flag constants in this file.
87-92: LGTM! Well-implemented CLI flag addition.The
--json-valuesflag implementation correctly follows the established patterns in this file:
- Uses proper flag creation with
flags.NewFlag- Includes environment variable support via
tgPrefix.EnvVars()- Correctly binds to
&opts.StackValuesJSON- Provides clear usage description
- Maintains consistency with other boolean flags
The integration is clean and maintains backward compatibility.
docs/_docs/02_features/02-stacks.md (2)
180-184: LGTM! Clear documentation of format options.The updates to the directory tree examples properly illustrate that values files can be generated in either HCL or JSON format using the
--json-valuesflag. The inline comments effectively communicate this option without cluttering the examples.
186-199: LGTM! Excellent documentation of the new feature.The "Values File Formats" section provides comprehensive and clear documentation:
- Clearly explains HCL as the default format
- Shows practical command examples for both formats
- Explains the functional equivalence of both formats
- Provides good guidance on when JSON might be preferred
- Mentions the deterministic alphabetical sorting behavior
The documentation is well-structured and will help users understand when and how to use the new JSON values feature.
docs/_docs/04_reference/04-config-blocks-and-attributes.md (3)
1748-1749: Update parallel docs-starlight tree snippetThe same tree comment (
# or terragrunt.values.json with --json-values) must appear in the matching file underdocs-starlight/...to keep both doc sets in sync, per the migration guideline.
1814-1818: Mirror change in docs-starlightReplicate the updated comment in the directory tree example inside the
docs-starlightreference to keep both doc paths aligned.
1828-1829: Bullet reads well – just ensure consistencyGood call-out about pre-created files. Confirm the same note is present in the stack/CLI docs so users don’t miss the nuance.
test/integration_stacks_test.go (4)
1318-1376: LGTM! Comprehensive test for JSON values generation.The test properly validates JSON values file generation with the
--json-valuesflag, including:
- File existence/non-existence checks
- Valid JSON structure validation
- Expected content verification including the auto-generated comment
- Both dev and prod environment coverage
1378-1409: LGTM! Good end-to-end test for JSON values usage.This test validates the complete workflow of running stack commands with JSON values, ensuring the values are properly consumed and JSON files are generated as expected.
1411-1432: LGTM! Essential test for JSON values readability.This focused test ensures that once JSON values files are generated, Terragrunt can properly read and use them in subsequent operations. The plan command validation confirms the feature works end-to-end.
1434-1468: LGTM! Critical test for backward compatibility and precedence.This test validates the most important aspect of the JSON values feature - that JSON files take precedence over HCL files when both exist. This ensures smooth migration paths and prevents breaking existing workflows.
config/stack_test.go (5)
173-235: LGTM! Excellent refactoring to support both HCL and JSON formats.The table-driven approach cleanly tests both formats while maintaining the deterministic sorting verification. The separation of test cases and verification logic improves maintainability.
238-292: LGTM! Clean helper function update.The addition of the
fileExtparameter is minimal and focused, enabling support for both HCL and JSON file extensions without breaking existing functionality.
354-400: LGTM! Comprehensive JSON values reading test.The test thoroughly validates JSON parsing with multiple data types (strings, numbers, booleans, objects) and includes proper comment field handling. Good coverage of the core JSON reading functionality.
402-442: LGTM! Essential precedence test.This test is crucial for validating that JSON values take precedence over HCL values when both files exist. The conflicting values approach clearly demonstrates the precedence behavior, and the assertions are explicit about expected behavior.
444-483: LGTM! Good comment handling test.This test ensures that JSON comment fields are properly handled and accessible while not interfering with other values. Important for validating the auto-generated comment feature.
config/stack.go (7)
5-5: LGTM! Clean constants and imports addition.The JSON import and
valuesFileJSONconstant follow existing patterns and are minimal additions that support the new functionality without affecting existing code.Also applies to: 37-37
776-823: LGTM! Well-structured writeValues function update.The function cleanly handles both HCL and JSON formats with proper conditional logic, error handling, and informative logging. The separation into helper functions maintains good code organization.
825-835: LGTM! Clean generic utility function.The
getSortedKeysfunction is well-implemented using Go generics, providing reusable key sorting functionality for deterministic output in both HCL and JSON formats.
837-865: LGTM! Good extraction with deterministic improvement.The HCL rendering function maintains existing functionality while adding deterministic key sorting. The separation of concerns improves maintainability.
867-893: LGTM! Well-implemented JSON rendering function.The function properly converts cty values to JSON with deterministic key sorting, auto-generated comments, and proper formatting. Good error handling and consistent behavior with the HCL equivalent.
895-939: LGTM! Solid precedence implementation in ReadValues.The function correctly implements JSON-first precedence while maintaining backward compatibility with HCL files. The existing parsing logic is properly reused, and the updated documentation clearly describes the behavior.
586-586: LGTM! Necessary parameter addition.The addition of the
optsparameter to thewriteValuescall enables the function to access theStackValuesJSONflag, completing the integration chain from CLI to file generation.
|
@pseudomorph could you explain why you need this feature? The Terragrunt CLI has a lot of flags, and adding another one should be justified with functionality that is valuable and hard to achieve otherwise. |
|
@yhakbar - Sure! We're currently working on a framework which leverages scaffold and stacks to replace our current homegrown set up to manage a large monorepo (3000+ deployments) which is deployed with Atlantis. At first I was looking at a soft adoption of stacks around our current setup, but the proliferation of repetitive files in Git feel less than ideal. Foregoing the soft adoption would be nice from a DRY perspective, but here is still some necessity of visibility (for the end-user) and change tracking in Git (for Atlantis). The idea I have is render out metadata files for each of the generated stack units, which information about the rendered units (file hashes), along with unit-specific values. That way users can understand what values a specific deployment is getting (and why), and Atlantis can target the appropriate point changes. We're planning to use stack operations for wholesale standup and teardown of infra, but do point operations on the day-to-day management... Lots of individual teams managing their specific slice of infra. Having the values file rendered out as json as part of stack generation allows for more efficient processing (no secondary operations or tooling). It is possible to use |
|
Actually, taking a step back, would it make more sense to place this functionality into a stack command? Something like: This information could be used within workflows to help with the orphan units/state matter raised in #4504 . Apologies, I should have started this off as an issue. |
|
@pseudomorph it's all good! Sometimes seeing a prototype helps shape better understanding of the problem and potential solutions. I would recommend creating an RFC, and then soliciting feedback from the community on the design. We want to think through all reasonable alternatives, and approaches (Discord is a good place to do that. Please tag me if you ask for feedback there). |
2 participants
Description
This allows for unit values files to be written as either
terragrunt.values.hclorterragrunt.values.json. Default is current behavior (writingterragrunt.values.hcl).TODOs
Read the Gruntwork contribution guidelines.
Release Notes (draft)
Added support for generating values files in JSON format using the
--json-valuesflag withterragrunt stack generate.Migration Guide
N/A
Summary by CodeRabbit
New Features
--json-valuesflag with stack commands. Both HCL and JSON formats are now supported and functionally equivalent.Documentation
--json-valuesflag, explain values file formats, and provide usage examples for both HCL and JSON output options.Tests