How to manage Terragrunt Monorepos efficiently?

When working with large monorepos, managing Terragrunt plans can become overwhelming. Each project generating a comment in the pull request can result in a flood of information, making it challenging to track changes effectively. This is where Digger's feature to group plans by the source module comes in handy, streamlining the process and reducing noise.

What is Grouping by Module?

Grouping by module is a method to consolidate the number of comments generated during a pull request by categorizing them according to the source module that triggered the change. By doing so, it reduces the number of comments to one per group if the plans are similar, making it easier to review changes.

How to Enable Grouping by Module

To enable this feature, you need to modify the digger.yml configuration file. Add the following flag:

comment_render_mode: "group_by_module"

This setting instructs Digger to render only one comment per group. Each group corresponds to a source module that triggered a change. If a project is affected by changes from multiple modules, it can belong to multiple groups. Within each group, if the plans are similar, only the first plan will be displayed.

Determining Plan Similarity

Digger determines the similarity of plans by comparing the set of resource keys in the plan JSON. If the resource keys for all projects in a group are the same, the plans are considered similar. Consequently, only one plan from the group will be shown, while the rest will be collapsed.

Benefits of Grouping Mode

The grouping mode brings about several significant changes in how Digger handles plans and comments:

1. Backend Comment Rendering: In this mode, comment rendering is shifted from the CLI to the backend. This transition helps in managing and organizing comments more efficiently.
2. Backend Storage of Plan Summaries: Plan summaries (not the JSON itself) are now stored on the backend. This centralized storage ensures easy access and better management of plan summaries.

Practical Example

Imagine a scenario where multiple Terragrunt modules in your monorepo trigger changes. Without grouping by module, each change would generate a separate comment, cluttering the pull request. With the group_by_module feature enabled, Digger consolidates these comments. If the plans are similar (same set of resource keys), you will see only one comment for the group, significantly reducing noise and making it easier to review the changes.

Here's how a digger.yml configuration might look:

comment_render_mode: "group_by_module"
projects:
- name: my-first-app
  dir: app-one
- name: my-second-app
  dir: app-two
  auto_merge: true

With this setup, Digger efficiently groups comments by source module, ensuring that your pull requests remain clean and manageable.

While the example above shows the basic structure, you can add more configuration options to your digger.yml file based on your specific needs. Here are some additional options you might consider:

1. Workflows: You can define custom workflows for your projects.
2. Generate Projects: If you have a large number of projects, you can use the generate_projects directive to dynamically generate the list of projects.
3. Include/Exclude Patterns: You can specify wildcard and glob patterns to include or exclude certain directories in your projects.

Conclusion

The group_by_module feature in Digger is a powerful tool for teams managing large monorepos with Terragrunt. By reducing the noise in pull requests and organizing comments more effectively, it enhances the review process, making it easier for teams to collaborate and maintain high-quality codebases. Enabling this feature is straightforward and can significantly improve the workflow for handling infrastructure changes.