Skip to content

Platform Engineering: Template Creation & Publishing

Overview

Platform engineers can create reusable infrastructure templates from existing projects (blueprints, POCs, or reference architectures) and publish them for consumption through Internal Developer Platforms like Backstage.

Template Format: ThothCTL uses #{ }# placeholder expressions for parameterization, not Jinja2 syntax.

Example Blueprint: terragrunt_ecs_blueprint - Production-ready ECS Fargate architecture with ALB, RDS, and multi-AZ deployment.

Use Case: From Project to Template

Scenario

A platform engineering team has developed a production-ready reference architecture for a microservices platform. They want to:

  1. Convert the working project into a reusable template
  2. Parameterize environment-specific values using #{ }# expressions
  3. Publish to a Git repository
  4. Make it available in Backstage for self-service

Workflow

%%{init: {'theme':'base', 'themeVariables': {
  'primaryColor':'#3b82f6',
  'primaryTextColor':'#ffffff',
  'primaryBorderColor':'#2563eb',
  'lineColor':'#94a3b8',
  'secondaryColor':'#10b981',
  'tertiaryColor':'#8b5cf6',
  'background':'transparent',
  'mainBkg':'#3b82f6',
  'secondBkg':'#10b981',
  'tertiaryBkg':'#8b5cf6',
  'clusterBkg':'rgba(241, 245, 249, 0.05)',
  'clusterBorder':'#475569',
  'titleColor':'currentColor',
  'edgeLabelBackground':'transparent',
  'nodeTextColor':'#ffffff',
  'textColor':'currentColor',
  'nodeBorder':'#1e293b',
  'fontSize':'14px'
}}}%%
graph LR
    A["Working<br/>Project"] --> B["Convert to<br/>Template"]
    B --> C["Parameterize<br/>Values"]
    C --> D["Test<br/>Template"]
    D --> E["Publish to<br/>Git"]
    E --> F["Register in<br/>Backstage"]
    F --> G["Self-Service<br/>Consumption"]

    classDef projectStyle fill:#3b82f6,stroke:#60a5fa,stroke-width:2px,color:#fff
    classDef processStyle fill:#10b981,stroke:#34d399,stroke-width:2px,color:#fff
    classDef publishStyle fill:#8b5cf6,stroke:#a78bfa,stroke-width:2px,color:#fff

    class A projectStyle
    class B,C,D processStyle
    class E,F,G publishStyle

Step-by-Step Guide

Step 1: Prepare Your Reference Architecture

Ensure your project is production-ready:

# Navigate to your reference architecture
cd /path/to/microservices-platform

# Verify structure
tree -L 2

Example Project Structure: 

microservices-platform/
├── common/
│   ├── backend.tf
│   └── variables.tf
├── environments/
│   ├── dev/
│   ├── staging/
│   └── prod/
├── stacks/
│   ├── networking/
│   ├── compute/
│   ├── database/
│   └── monitoring/
├── docs/
│   └── README.md
└── root.hcl

Step 2: Convert Project to Template

Use ThothCTL to convert your project into a reusable template:

# Convert project to template
thothctl project convert --make-template --template-project-type terraform

# Or for Terraform with Terragrunt
thothctl project convert --make-template --template-project-type terraform-terragrunt

# Or for standalone Terragrunt
thothctl project convert --make-template --template-project-type terragrunt

# Or for OpenTofu
thothctl project convert --make-template --template-project-type tofu

# Or for CDK v2
thothctl project convert --make-template --template-project-type cdkv2

# Or for Terraform modules
thothctl project convert --make-template --template-project-type terraform_module

# Or for custom project types
thothctl project convert --make-template --template-project-type custom

What This Does: - Identifies hardcoded values from [project_properties] in .thothcf.toml - Creates #{placeholder}# expressions in project files - Saves the template to ~/.thothcf/<project_name>/ - Preserves project structure - Creates parameterized files

Step 3: Review Generated Template

The conversion creates template files with #{ }# placeholder expressions:

Before (Project): 

# environments/prod/main.tf
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"

  tags = {
    Name        = "microservices-vpc"
    Environment = "production"
    Team        = "platform-engineering"
  }
}

After (Template): 

# environments/#{environment}#/main.tf
resource "aws_vpc" "main" {
  cidr_block = "#{vpc_cidr}#"

  tags = {
    Name        = "#{project_name}#-vpc"
    Environment = "#{environment}#"
    Team        = "#{team_name}#"
  }
}

Example from Blueprint: 

# common/common.tfvars
project_name = "#{project}#"
environment  = "#{environment}#"
region       = "#{backend_region}#"
dynamodb_table = "#{dynamodb_backend}#"
backend_bucket = "#{backend_bucket}#"

Step 3.1: Manual Replacement (Alternative)

You can also manually replace values using your IDE or shell script:

# Example replacements from terragrunt_ecs_blueprint
project_name = "#{project}#"
environment  = "#{environment}#"
region       = "#{backend_region}#"
dynamodb_table = "#{dynamodb_backend}#"
backend_bucket = "#{backend_bucket}#"

Using find and replace in your IDE: 1. Find: "my-project-name" → Replace: "#{project}#" 2. Find: "production" → Replace: "#{environment}#" 3. Find: "us-east-1" → Replace: "#{backend_region}#"

Step 3.5: Configure .thothcf.toml

The .thothcf.toml file is critical for template functionality. It maps #{ }# placeholders to parameters with validation rules and defines project structure requirements.

Create .thothcf.toml in your template root:

[thothcf]
project_id = "microservices-platform"
project_type = "terraform"

# Define template parameters with validation
[template_input_parameters.project_name]
template_value = "#{project}#"
condition = "^[a-zA-Z0-9_-]+$"
description = "Project Name"

[template_input_parameters.environment]
template_value = "#{environment}#"
condition = "(dev|qa|stg|staging|test|prod)"
description = "Environment name (dev|qa|stg|staging|test|prod)"

[template_input_parameters.deployment_region]
template_value = "#{backend_region}#"
condition = "^[a-z]{2}-[a-z]{4,10}-\\d$"
description = "AWS Region for deployment"

[template_input_parameters.backend_bucket]
template_value = "#{backend_bucket}#"
condition = "^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$"
description = "S3 bucket for Terraform state"

[template_input_parameters.backend_region]
template_value = "#{backend_region}#"
condition = "^[a-z]{2}-[a-z]{4,10}-\\d$"
description = "Backend AWS Region"

[template_input_parameters.backend_dynamodb]
template_value = "#{dynamodb_backend}#"
condition = "^[a-zA-Z0-9_.-]{3,255}$"
description = "DynamoDB table for state locking"

[template_input_parameters.owner]
template_value = "#{team_name}#"
condition = "^[a-zA-Z0-9_-]+$"
description = "Team or role owner for this deployment"

[template_input_parameters.vpc_cidr]
template_value = "#{vpc_cidr}#"
condition = "^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}/\\d{1,2}$"
description = "VPC CIDR block"

[template_input_parameters.cloud_provider]
template_value = "aws"
condition = "(aws|azure|oci|gcp)"
description = "Cloud provider (aws|azure|oci|gcp)"

[template_input_parameters.deployment_profile]
template_value = "default"
condition = "^[a-zA-Z0-9_.-]{3,255}$"
description = "AWS CLI deployment profile"

# Define project structure requirements
[project_structure]
root_files = [
   ".gitignore",
   "README.md",
   ".thothcf.toml"
]
ignore_folders = [
    ".git",
    ".terraform",
    "Reports"
]

[[project_structure.folders]]
name = "common"
mandatory = true
type = "root"
content = ["backend.tf", "variables.tf"]

[[project_structure.folders]]
name = "environments"
mandatory = true
type = "root"

[[project_structure.folders]]
name = "stacks"
mandatory = true
type = "root"

Key Components:

  1. Project Metadata: Identifies template type and ID
  2. Template Parameters: Maps each #{ }# placeholder to a parameter with:
  3. template_value: The placeholder used in files
  4. condition: Regex validation pattern
  5. description: Human-readable description
  6. Project Structure: Defines required files and folders for validation

Optional: Backstage template.yaml

If you plan to integrate with Backstage for self-service, you can create a separate template.yaml in Backstage scaffolder format (see Step 6 below). The .thothcf.toml is the primary configuration for ThothCTL's template engine.

Step 4: Test Template Locally

Before publishing, test the template by converting it back to a project:

# Create a test project from the template
thothctl project convert --make-project --template-project-type terraform

# Verify generated project
cd test-microservices
terraform init
terraform plan

Step 5: Publish Template to Git

Push your template to a Git repository:

# Initialize Git repository
cd microservices-platform
git init
git add .
git commit -m "feat: create microservices platform template"

# Push to remote (GitHub, GitLab, Azure DevOps)
git remote add origin https://github.com/your-org/microservices-platform-template.git
git push -u origin main

# Tag the release
git tag -a v1.0.0 -m "Release v1.0.0"
git push origin v1.0.0

Step 6: Register in Backstage

Create a Backstage template definition:

template.yaml (Backstage format): 

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: microservices-platform
  title: Microservices Platform on AWS
  description: Production-ready microservices infrastructure with networking, compute, database, and monitoring
  tags:
    - terraform
    - aws
    - microservices
    - platform-engineering
spec:
  owner: platform-team
  type: infrastructure

  parameters:
    - title: Project Information
      required:
        - project_name
        - team_name
      properties:
        project_name:
          title: Project Name
          type: string
          description: Name for your microservices platform
        team_name:
          title: Team Name
          type: string
          description: Team responsible for this infrastructure
        environment:
          title: Environment
          type: string
          description: Target environment
          enum:
            - dev
            - staging
            - prod
          default: dev

    - title: Network Configuration
      properties:
        vpc_cidr:
          title: VPC CIDR Block
          type: string
          description: CIDR block for the VPC
          default: "10.0.0.0/16"
        aws_region:
          title: AWS Region
          type: string
          description: AWS region for deployment
          default: us-east-1
          enum:
            - us-east-1
            - us-west-2
            - eu-west-1

  steps:
    - id: fetch-template
      name: Fetch Template
      action: fetch:template
      input:
        url: https://github.com/your-org/microservices-platform-template
        values:
          project_name: ${{ parameters.project_name }}
          team_name: ${{ parameters.team_name }}
          environment: ${{ parameters.environment }}
          vpc_cidr: ${{ parameters.vpc_cidr }}
          aws_region: ${{ parameters.aws_region }}

    - id: publish
      name: Publish to GitHub
      action: publish:github
      input:
        allowedHosts: ['github.com']
        description: ${{ parameters.project_name }} - Microservices Platform
        repoUrl: github.com?owner=your-org&repo=${{ parameters.project_name }}

    - id: register
      name: Register Component
      action: catalog:register
      input:
        repoContentsUrl: ${{ steps.publish.output.repoContentsUrl }}
        catalogInfoPath: '/catalog-info.yaml'

  output:
    links:
      - title: Repository
        url: ${{ steps.publish.output.remoteUrl }}
      - title: Open in Backstage
        icon: catalog
        entityRef: ${{ steps.register.output.entityRef }}

Step 7: Self-Service Consumption

Developers can now use the template through Backstage:

  1. Navigate to Backstage Software Catalog
  2. Click "Create Component"
  3. Select "Microservices Platform on AWS"
  4. Fill in parameters
  5. Click "Create"

Result: A new repository with fully configured infrastructure ready for deployment.

Advanced: Converting Between Frameworks

Terragrunt to Terramate

For advanced deployment patterns:

# Convert to Terramate stacks
thothctl project convert \
  --make-terramate-stacks \
  --branch-name main

# This creates Terramate stack structure
# with proper dependencies and ordering

Template to Project

Convert a template back to a working project:

# Create project from template
thothctl project convert \
  --make-project \
  --template-project-type terraform

Use Cases

1. Blueprint Templates

Scenario: Create standardized blueprints for common architectures

Examples: - Three-tier web application - Serverless API platform - Data lake infrastructure - ML/AI platform

Command: 

thothctl project convert --make-template --template-project-type terraform

2. POC to Production Template

Scenario: Convert successful POC into production-ready template

Workflow: 1. Develop and validate POC 2. Harden security and compliance 3. Convert to template 4. Publish for team use

Command: 

thothctl project convert --make-template --template-project-type terraform

3. Reference Architecture Library

Scenario: Build a library of reference architectures

Structure: 

reference-architectures/
├── microservices-platform/
├── data-analytics-platform/
├── ml-training-platform/
└── serverless-api-platform/

Each template includes: - Infrastructure code - Documentation - Best practices - Cost estimates - Security configurations

4. Multi-Cloud Templates

Scenario: Create templates for different cloud providers

Examples: - AWS microservices template - Azure microservices template - GCP microservices template

Benefit: Consistent architecture across clouds

Integration with CI/CD

GitHub Actions

name: Validate Template

on:
  push:
    branches: [main]
  pull_request:

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install ThothCTL
        run: pip install thothctl

      - name: Test Template
        run: |
          cd templates/source-project
          thothctl project convert --make-project --template-project-type terraform
          terraform init
          terraform validate

GitLab CI

validate-template:
  stage: test
  script:
    - pip install thothctl
    - thothctl project convert --make-project --template-project-type terraform
    - terraform init && terraform validate

Best Practices

1. Template Design

✅ Do: - Use meaningful parameter names - Provide sensible defaults - Include comprehensive documentation - Add validation rules - Include examples

❌ Don't: - Hardcode sensitive values - Over-parameterize (keep it simple) - Skip documentation - Ignore security best practices

2. Version Control

✅ Do: - Use semantic versioning (v1.0.0) - Tag releases - Maintain CHANGELOG.md - Document breaking changes

3. Documentation

Include in your template: - README.md - Overview and quick start - USAGE.md - Detailed usage guide - ARCHITECTURE.md - Architecture decisions - PARAMETERS.md - Parameter reference - EXAMPLES.md - Usage examples

4. Testing

Test your templates: - Unit tests (Terraform validate) - Integration tests (actual deployment) - Security scans (Checkov, Trivy) - Cost analysis (ThothCTL cost analysis)

Template Repository Structure

microservices-platform-template/
├── .github/
│   └── workflows/
│       └── validate.yml
├── common/
│   ├── backend.tf
│   └── variables.tf
├── environments/
│   └── #{environment}#/
├── stacks/
│   ├── networking/
│   ├── compute/
│   ├── database/
│   └── monitoring/
├── docs/
│   ├── README.md
│   ├── ARCHITECTURE.md
│   └── PARAMETERS.md
├── examples/
│   ├── dev-example.yaml
│   ├── staging-example.yaml
│   └── prod-example.yaml
├── .thothcf.toml           # ThothCTL template config (parameters + structure)
├── backstage-template.yaml  # Backstage template config
├── catalog-info.yaml        # Backstage catalog entry
├── CHANGELOG.md
└── LICENSE

Commands Reference

Command Purpose
thothctl project convert --make-template Convert project to template
thothctl project convert --make-project Convert template to project
thothctl project convert --make-terramate-stacks Create Terramate stacks
thothctl init project --reuse --space <name> Create project from VCS template
thothctl project bootstrap Add ThothCTL support to existing project
thothctl project upgrade Upgrade project from template

Benefits

For Platform Engineers

✅ Standardization - Consistent infrastructure across teams
✅ Reusability - Write once, use many times
✅ Governance - Enforce best practices and compliance
✅ Efficiency - Reduce time from idea to deployment
✅ Quality - Battle-tested, production-ready templates

For Development Teams

✅ Self-Service - Deploy infrastructure without platform team
✅ Speed - Minutes instead of weeks
✅ Confidence - Pre-validated, secure configurations
✅ Focus - Spend time on features, not infrastructure
✅ Consistency - Same patterns across projects

Next Steps

  1. Identify your most common infrastructure patterns
  2. Create reference implementations
  3. Convert to templates using ThothCTL
  4. Publish to Git repositories
  5. Register in Backstage or your IDP
  6. Enable self-service for development teams

Platform Engineering Team
Enabling developer velocity through self-service infrastructure