Automating OpenTofu on Ubuntu 24.04 with Ansible and Azure Service Principals

OpenTofu has rapidly matured into a production-grade, community-driven alternative to Terraform. However, moving from manual installation to automated, secure infrastructure-as-code (IaC) workflows requires careful attention to detail, especially when dealing with package repositories, GPG keys, and cloud authentication.

In this guide, we will walk through how to install OpenTofu on Ubuntu 24.04 using Ansible, troubleshoot common shell errors, and configure secure, non-interactive Azure authentication using Service Principals.

Why OpenTofu?

Since its fork from Terraform in 2023, OpenTofu has maintained compatibility with Terraform’s HCL syntax while providing a truly open-source alternative under the Linux Foundation. For organizations concerned about licensing changes or seeking community-driven governance, OpenTofu represents a compelling choice for infrastructure automation.

Prerequisites

Before you begin, ensure you have:

Control Node: Ansible 2.9+ installed
Target Node: Ubuntu 24.04 Server (fresh or existing installation)
Cloud Provider: Microsoft Azure with active subscription
Access Level: Sufficient permissions to create Service Principals in Azure AD (now Microsoft Entra ID)
SSH Access: Key-based authentication configured between control and target nodes

Step 1: Installing OpenTofu via Ansible

While the official documentation provides manual steps, automating this via Ansible introduces specific challenges. Two common pitfalls include GPG key corruption via tee and shell compatibility issues with pipefail.

The GPG Key Pitfall

Using tee to write GPG keys can sometimes append a newline character, corrupting the keyring. Instead, use get_url or curl -o to write files directly. This ensures binary-exact copies of the GPG keys without shell interpretation.

The `pipefail` Error

Ansible’s shell module uses /bin/sh (often dash on Ubuntu) by default, which does not support set -o pipefail. You must explicitly specify executable: /bin/bash to avoid cryptic errors like:

/bin/sh: 1: set: Illegal option -o pipefail

The Ansible Playbook

Here is a robust task set to install OpenTofu using the official APT repository:

---
- name: Install OpenTofu on Ubuntu 24.04 (APT repo)
  hosts: all
  become: true
  tasks:
    - name: Install dependencies for repo setup
      ansible.builtin.apt:
        name:
          - apt-transport-https
          - ca-certificates
          - curl
          - gnupg
        state: present
        update_cache: true

    - name: Ensure APT keyrings directory exists
      ansible.builtin.file:
        path: /etc/apt/keyrings
        state: directory
        mode: "0755"

    - name: Download OpenTofu GPG key (opentofu.gpg)
      ansible.builtin.get_url:
        url: https://get.opentofu.org/opentofu.gpg
        dest: /etc/apt/keyrings/opentofu.gpg
        mode: "0644"

    - name: Download repository GPG key and dearmor to keyring
      ansible.builtin.shell: |
        set -euo pipefail
        curl -fsSL https://packages.opentofu.org/opentofu/tofu/gpgkey \
        | gpg --no-tty --batch --dearmor -o /etc/apt/keyrings/opentofu-repo.gpg
        chmod a+r /etc/apt/keyrings/opentofu-repo.gpg
      args:
        executable: /bin/bash
        creates: /etc/apt/keyrings/opentofu-repo.gpg

    - name: Add OpenTofu APT repository
      ansible.builtin.copy:
        dest: /etc/apt/sources.list.d/opentofu.list
        mode: "0644"
        content: |
          deb [signed-by=/etc/apt/keyrings/opentofu.gpg,/etc/apt/keyrings/opentofu-repo.gpg] https://packages.opentofu.org/opentofu/tofu/any/ any main
          deb-src [signed-by=/etc/apt/keyrings/opentofu.gpg,/etc/apt/keyrings/opentofu-repo.gpg] https://packages.opentofu.org/opentofu/tofu/any/ any main

    - name: Install OpenTofu package
      ansible.builtin.apt:
        name: tofu
        state: present
        update_cache: true

    - name: Verify OpenTofu installation
      ansible.builtin.command: tofu version
      register: tofu_version
      changed_when: false

    - name: Display OpenTofu version
      ansible.builtin.debug:
        msg: "{{ tofu_version.stdout }}"

# ============== Azure CLI installation tasks ============
    - name: Add Microsoft GPG key
  ansible.builtin.get_url:
        url: https://packages.microsoft.com/keys/microsoft.asc
        dest: /etc/apt/keyrings/microsoft.asc
        mode: "0644"

    - name: Add Azure CLI repository
      ansible.builtin.copy:
        dest: /etc/apt/sources.list.d/azure-cli.list
        content: |
          deb [signed-by=/etc/apt/keyrings/microsoft.asc] https://packages.microsoft.com/repos/azure-cli/ noble main
        mode: "0644"

    - name: Install Azure CLI
      ansible.builtin.apt:
        name: azure-cli
        state: present
        update_cache: true

Key Improvements in This Playbook

Idempotency: The creates parameter ensures GPG key operations only run once
Error Handling: Using set -euo pipefail catches errors in piped commands
Verification: The playbook confirms successful installation by checking the version
Proper Permissions: All files have explicit, secure permission modes

Step 2: Azure Authentication (CLI vs. Service Principal)

A common error when running OpenTofu on automation servers is:

Error: unable to build authorizer for Resource Manager API: 
could not configure AzureCli Authorizer: launching Azure CLI: exec: "az": executable file not found in $PATH

This occurs because the azurerm provider defaults to Azure CLI authentication, which is interactive and requires the az binary. For automation (Ansible, CI/CD), this is not suitable.

The Solution: Service Principals

Use a Service Principal (SP) for non-interactive authentication. This method is secure, scriptable, and does not require browser logins or installed CLI tools.

Understanding Service Principals

A Service Principal is essentially an identity created for use with applications, hosted services, and automated tools. Think of it as a “service account” for Azure resources. Unlike user accounts, Service Principals:

Don’t require MFA or interactive login
Can be scoped to specific subscriptions or resource groups
Support secret rotation without service disruption
Provide audit trails separate from user activity

Creating the Service Principal

Option 1: Azure Portal

Navigate to Microsoft Entra ID > App registrations
Click New registration
Name: opentofu-automation-sp (use a descriptive name)
Click Register
Note the Application (client) ID and Directory (tenant) ID
Go to Certificates & secrets > New client secret
Add description: OpenTofu Ansible automation
Set expiration (recommended: 12-24 months)
Copy the secret value immediately (it won’t be shown again)

Option 2: Azure CLI

# Create the Service Principal
az ad sp create-for-rbac \
  --name "opentofu-automation-sp" \
  --role Contributor \
  --scopes /subscriptions/{SUBSCRIPTION_ID}

# Output will include:
# {
#   "appId": "YOUR_CLIENT_ID",
#   "password": "YOUR_CLIENT_SECRET",
#   "tenant": "YOUR_TENANT_ID"
# }

Assigning Permissions

The Service Principal needs appropriate RBAC roles:

Navigate to Subscriptions > Select your subscription
Click Access Control (IAM)
Click Add > Add role assignment
Select role: Contributor (or custom role with required permissions)
Assign access to: User, group, or service principal
Select members: Search for your Service Principal name
Click Review + assign

Best Practice: Instead of Subscription-level Contributor, consider:

Creating a custom role with minimum required permissions
Scoping to specific resource groups
Using separate Service Principals for dev/staging/production

Required Environment Variables

OpenTofu recognizes the following environment variables automatically when using the azurerm provider:

ARM_CLIENT_ID – The Application (client) ID
ARM_CLIENT_SECRET – The client secret value
ARM_TENANT_ID – The Directory (tenant) ID
ARM_SUBSCRIPTION_ID – Your Azure subscription ID

You can find your subscription ID with:

az account show --query id --output tsv

Step 3: Injecting Credentials Securely with Ansible

Do not hardcode secrets in your .tf files or persist them to /etc/environment unless absolutely necessary. The best practice is to inject them only for the duration of the task using Ansible’s environment keyword.

Why This Matters

Hardcoding secrets leads to:

Accidental commits to version control
Secrets exposed in CI/CD logs
Difficulty in credential rotation
Compliance violations (SOC 2, ISO 27001, etc.)

Recommended Ansible Task

---
- name: Run OpenTofu plan (non-interactive Azure auth)
  hosts: opentofu_servers
  become: false
  environment:
    ARM_CLIENT_ID: "{{ azure_client_id }}"
    ARM_CLIENT_SECRET: "{{ azure_client_secret }}"
    ARM_TENANT_ID: "{{ azure_tenant_id }}"
    ARM_SUBSCRIPTION_ID: "{{ azure_subscription_id }}"
  tasks:
    - name: Initialize OpenTofu
      ansible.builtin.command:
        cmd: tofu init
        chdir: /path/to/your/terraform/code
      register: tofu_init

    - name: Run tofu plan
      ansible.builtin.command:
        cmd: tofu plan -out=tfplan
        chdir: /path/to/your/terraform/code
      register: tofu_plan

    - name: Display plan output
      ansible.builtin.debug:
        var: tofu_plan.stdout_lines

Storing Secrets Securely in Ansible

Option 1: Ansible Vault

# Create encrypted vars file
ansible-vault create group_vars/all/vault.yml

# Add your secrets:
# azure_client_id: "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx"
# azure_client_secret: "your-secret-here"
# azure_tenant_id: "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx"
# azure_subscription_id: "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx"

# Run playbook with vault password
ansible-playbook -i inventory playbook.yml --ask-vault-pass

Option 2: External Secret Management

For enterprise environments, integrate with:

HashiCorp Vault: Use community.hashi_vault collection
Azure Key Vault: Use azure.azcollection.azure_rm_keyvaultsecret_info
AWS Secrets Manager: Use community.aws modules
CyberArk, 1Password, Bitwarden etc.

Example with HashiCorp Vault:

- name: Retrieve Azure credentials from Vault
  community.hashi_vault.vault_kv2_get:
    url: https://vault.example.com:8200
    path: secret/azure/opentofu
    auth_method: token
    token: "{{ lookup('env', 'VAULT_TOKEN') }}"
  register: vault_secrets

- name: Set facts from Vault
  ansible.builtin.set_fact:
    azure_client_id: "{{ vault_secrets.secret.client_id }}"
    azure_client_secret: "{{ vault_secrets.secret.client_secret }}"

Step 4: Handling Multiple Subscriptions

You do not need a separate Service Principal for each subscription if they reside in the same tenant.

Single Service Principal, Multiple Subscriptions

Create one Service Principal in your Azure AD tenant
Assign the Contributor role (or appropriate role) to that SP on each subscription
Switch subscriptions in OpenTofu by changing the ARM_SUBSCRIPTION_ID variable

Practical Implementation

inventory.yml:

all:
  children:
    dev:
      hosts:
        dev-server:
          ansible_host: 10.0.1.10
          azure_subscription_id: "xxxxx-dev-subscription-id"
    prod:
      hosts:
        prod-server:
          ansible_host: 10.0.2.10
          azure_subscription_id: "xxxxx-prod-subscription-id"
  vars:
    azure_client_id: "{{ vault_azure_client_id }}"
    azure_client_secret: "{{ vault_azure_client_secret }}"
    azure_tenant_id: "{{ vault_azure_tenant_id }}"

playbook.yml:

---
- name: Deploy infrastructure to multiple subscriptions
  hosts: all
  environment:
    ARM_CLIENT_ID: "{{ azure_client_id }}"
    ARM_CLIENT_SECRET: "{{ azure_client_secret }}"
    ARM_TENANT_ID: "{{ azure_tenant_id }}"
    ARM_SUBSCRIPTION_ID: "{{ azure_subscription_id }}"
  tasks:
    - name: Deploy to {{ inventory_hostname }}
      ansible.builtin.command:
        cmd: tofu apply -auto-approve
        chdir: /infrastructure/{{ inventory_hostname }}

Best Practices and Production Considerations

1. State File Management

OpenTofu state files contain sensitive information. For production:

terraform {
  backend "azurerm" {
    resource_group_name  = "tfstate-rg"
    storage_account_name = "tfstatestorage"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

Configure the backend to use your Service Principal credentials:

environment:
  ARM_CLIENT_ID: "{{ azure_client_id }}"
  ARM_CLIENT_SECRET: "{{ azure_client_secret }}"
  ARM_TENANT_ID: "{{ azure_tenant_id }}"
  ARM_SUBSCRIPTION_ID: "{{ azure_subscription_id }}"
  ARM_ACCESS_KEY: "{{ azure_storage_access_key }}"  # For state storage

2. Credential Rotation

Implement regular secret rotation:

- name: Rotate Service Principal credentials
  hosts: localhost
  tasks:
    - name: Create new client secret
      azure.azcollection.azure_rm_adserviceprincipal_info:
        app_id: "{{ azure_client_id }}"
      # Follow with secret rotation logic
      
    - name: Update Ansible Vault
      # Update vault.yml with new credentials
      
    - name: Verify new credentials work
      # Test with tofu plan

3. Least Privilege Access

Instead of Contributor at subscription level:

# Create custom role with minimal permissions
az role definition create --role-definition '{
  "Name": "OpenTofu Deployer",
  "Description": "Can manage infrastructure via OpenTofu",
  "Actions": [
    "Microsoft.Resources/subscriptions/resourceGroups/*",
    "Microsoft.Compute/*",
    "Microsoft.Network/*",
    "Microsoft.Storage/*"
  ],
  "NotActions": [],
  "AssignableScopes": ["/subscriptions/{SUBSCRIPTION_ID}"]
}'

4. Logging and Auditing

Enable detailed logging:

environment:
  TF_LOG: "INFO"  # Or DEBUG, TRACE for troubleshooting
  TF_LOG_PATH: "/var/log/opentofu/{{ ansible_date_time.iso8601 }}.log"

Monitor Service Principal usage:

Review Azure AD Sign-in logs
Set up alerts for failed authentication attempts
Track resource modifications via Azure Activity Log

Troubleshooting Common Issues

Issue 1: GPG Key Verification Failed

Error:

Err:6 https://packages.opentofu.org/opentofu/tofu/any any InRelease
  The following signatures couldn't be verified because the public key is not available

Solution:

- name: Re-import GPG keys
  ansible.builtin.shell: |
    rm -f /etc/apt/keyrings/opentofu*.gpg
  become: true

# Then re-run the GPG key tasks from Step 1

Issue 2: Authentication Timeout

Error:

Error: building account: getting authenticated object ID: parsing json result from the Azure CLI

Solution: Ensure Service Principal has propagated (can take 5-10 minutes after creation):

- name: Wait for Service Principal propagation
  ansible.builtin.pause:
    minutes: 2
    prompt: "Waiting for Azure AD replication..."

Issue 3: Subscription Access Denied

Error:

Error: Error ensuring Resource Providers are registered
Status=403 Code="AuthorizationFailed"

Solution: Verify RBAC assignment:

# Check role assignments
az role assignment list \
  --assignee {CLIENT_ID} \
  --subscription {SUBSCRIPTION_ID} \
  --output table

Issue 4: State Lock Timeout

When using Azure Storage for state:

Error: Error acquiring the state lock
Lock Info:
  ID:        xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx
  Operation: OperationTypeApply

Solution:

# Force unlock (use with caution)
tofu force-unlock xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx

Conclusion

By combining OpenTofu’s open-source licensing with Ansible’s automation capabilities and Azure Service Principals, you can build a secure, reproducible Infrastructure as Code pipeline.

Key Takeaways:

✅ Use Ansible’s get_url and proper shell executables to avoid GPG key corruption
✅ Avoid interactive CLI authentication in production—use Service Principals
✅ Inject secrets via environment variables, never hardcode them
✅ One Service Principal can manage multiple subscriptions in the same tenant
✅ Store secrets in Ansible Vault or external secret management systems
✅ Implement least privilege access and regular credential rotation
✅ Use remote state backends for team collaboration

This approach provides a foundation for GitOps workflows, CI/CD integration, and enterprise-grade infrastructure automation. Whether you’re managing a handful of resources or a complex multi-subscription architecture, these patterns will scale with your needs while maintaining security and auditability.

Why OpenTofu?

Prerequisites

Step 1: Installing OpenTofu via Ansible

The GPG Key Pitfall

The pipefail Error

The Ansible Playbook

Key Improvements in This Playbook

Step 2: Azure Authentication (CLI vs. Service Principal)

The Solution: Service Principals

Understanding Service Principals

Creating the Service Principal

Assigning Permissions

Required Environment Variables

Step 3: Injecting Credentials Securely with Ansible

Why This Matters

Recommended Ansible Task

Storing Secrets Securely in Ansible

Step 4: Handling Multiple Subscriptions

Single Service Principal, Multiple Subscriptions

Practical Implementation

Best Practices and Production Considerations

1. State File Management

2. Credential Rotation

3. Least Privilege Access

4. Logging and Auditing

Troubleshooting Common Issues

Issue 1: GPG Key Verification Failed

Issue 2: Authentication Timeout

Issue 3: Subscription Access Denied

Issue 4: State Lock Timeout

Conclusion

Additional Resources

The `pipefail` Error