codeYaan Logo Blog
Home Blog Services Courses
Login
Pulumi ESC: Secrets and Config Management for Teams
AI TOOLS March 15, 2026, 11:30 a.m.

Pulumi ESC: Secrets and Config Management for Teams

When teams start building infrastructure as code (IaC) with Pulumi, managing secrets and configuration quickly becomes a source of friction. Hard‑coding API keys, scattering environment variables across CI pipelines, or using ad‑hoc encryption scripts leads to accidental leaks and painful onboarding. Pulumi’s Elastic Secret Store (ESC) solves these problems by providing a unified, cloud‑native vault that integrates seamlessly with Pulumi’s configuration model. In this post we’ll explore how ESC works, walk through practical code samples, and share pro tips for keeping your team’s secrets safe and your deployments smooth.

Why Traditional Pulumi Config Falls Short

Pulumi’s pulumi config command is great for simple values like region names or feature flags, but it stores data in plaintext within the .pulumi folder by default. Even when you enable the --secret flag, the encrypted blobs are tied to the Pulumi service account that created them. This creates two pain points for teams:

  • Limited access control. Anyone with read access to the stack can decrypt every secret, regardless of their role.
  • Cross‑environment drift. Moving a stack between clouds or CI systems often requires manual re‑encryption.

ESC (Elastic Secret Store) decouples secret storage from the Pulumi service, letting you use cloud‑native KMS, IAM policies, and audit logs while keeping the same Pulumi experience.

Getting Started with ESC

The first step is to enable ESC for a stack. This is a one‑time operation that creates a dedicated secret store in your chosen cloud provider. The following CLI snippet shows how to enable ESC on AWS, but the same commands work for Azure and GCP with minor flag changes.

pulumi stack init dev
pulumi config set esc true --secret
pulumi esc enable --provider aws

Behind the scenes Pulumi provisions an AWS KMS key, an SSM Parameter Store namespace, and the necessary IAM roles. From that point on, any secret you set with pulumi config set --secret will be stored in ESC instead of the legacy store.

Reading Secrets in Your Code

Accessing ESC‑backed secrets is identical to the classic approach: use pulumi.Config and call requireSecret. The difference is that the secret is fetched from the remote ESC at runtime, not from a local encrypted file.

import pulumi
import pulumi_aws as aws

config = pulumi.Config()
db_password = config.require_secret("dbPassword")

# Example: create an RDS instance using the secret
db = aws.rds.Instance("mydb",
    engine="postgres",
    instance_class="db.t3.micro",
    allocated_storage=20,
    username="admin",
    password=db_password,
    skip_final_snapshot=True)

Because db_password is a pulumi.Output that carries secret metadata, Pulumi automatically redacts it from logs and state files. This means you can safely pass the secret to resources without ever exposing it in plaintext.

Sharing Secrets Across Stacks

Large organizations often have multiple stacks—dev, staging, prod—that need to share a common set of credentials (e.g., a service‑to‑service API key). ESC makes this straightforward with stack references. By exposing a secret from a “source” stack, other stacks can import it without duplicating the value.

# In the source stack (e.g., infra-common)
import pulumi

config = pulumi.Config()
config.set_secret("serviceApiKey", "s3cr3t-key-123")

pulumi.export("serviceApiKey", config.require_secret("serviceApiKey"))

Now any downstream stack can reference the secret:

# In the consumer stack (e.g., app-prod)
import pulumi

common = pulumi.StackReference("org/infra-common/dev")
api_key = common.get_output("serviceApiKey").apply(lambda k: k)  # still a secret

# Use the API key in a Lambda environment variable (auto‑redacted)
import pulumi_aws as aws
lambda_fn = aws.lambda_.Function("myFunc",
    runtime="python3.9",
    handler="handler.main",
    role=aws_iam_role.lambda_role.arn,
    environment=aws.lambda_.FunctionEnvironmentArgs(
        variables={"API_KEY": api_key}
    ),
    code=aws.s3.BucketObject("code.zip", bucket=bucket.id, source=pulumi.FileAsset("code.zip")).bucket)

Because the secret travels through Pulumi’s output system, it never lands in plain text on your CI runner or in the state file.

Real‑World Use Case: Multi‑Region Deployments

Imagine a fintech startup that runs workloads in both AWS us‑east‑1 and us‑west‑2. Regulatory compliance requires that database credentials be encrypted at rest with a KMS key that lives in each region. With ESC, you can define a per‑region secret store and let Pulumi automatically pick the right one based on the stack’s region config.

import pulumi
import pulumi_aws as aws

region = pulumi.Config("aws").require("region")
# Enable ESC with region‑specific KMS key
pulumi.esc.enable(provider="aws", region=region)

config = pulumi.Config()
db_password = config.require_secret("dbPassword")

# Provision a regional RDS instance
db = aws.rds.Instance(f"db-{region}",
    engine="postgres",
    instance_class="db.t3.micro",
    allocated_storage=50,
    username="admin",
    password=db_password,
    skip_final_snapshot=True,
    availability_zone=f"{region}a")

This pattern eliminates the need for separate CI pipelines per region; a single Pulumi program can target any region while ESC ensures the correct KMS key is used under the hood.

Pro Tip: Rotate Secrets Without Downtime

Enable automatic secret rotation by pairing ESC with a Lambda function that updates the secret in place. Store the Lambda ARN as a secret metadata tag, and configure a CloudWatch Event to trigger rotation every 30 days. Because Pulumi reads the secret at deployment time, you can safely roll out a new version of your infrastructure without touching the code.

Here’s a minimal example that creates a rotation Lambda and ties it to an SSM parameter managed by ESC:

import pulumi
import pulumi_aws as aws

# Create a Lambda that generates a new password
rotator = aws.lambda_.Function("secretRotator",
    runtime="python3.9",
    handler="rotate.handler",
    role=aws_iam_role.lambda_role.arn,
    code=aws.s3.BucketObject("rotator.zip",
        bucket=bucket.id,
        source=pulumi.FileAsset("rotate.zip")).bucket)

# Create the secret in ESC (backed by SSM Parameter Store)
secret = aws.ssm.Parameter("dbPassword",
    type="SecureString",
    value="initial-placeholder",
    key_id=aws_kms_key.key.arn,
    tags={"rotationLambdaArn": rotator.arn})

# Grant the Lambda permission to update the parameter
aws.lambda_.Permission("allowRotate",
    action="lambda:InvokeFunction",
    function=rotator.name,
    principal="ssm.amazonaws.com",
    source_arn=secret.arn)

# Schedule rotation every 30 days
aws.cloudwatch.EventRule("rotateRule",
    schedule_expression="rate(30 days)")

aws.cloudwatch.EventTarget("rotateTarget",
    rule="rotateRule",
    arn=rotator.arn,
    input=secret.id.apply(lambda id: f'{{"parameterId":"{id}"}}'))

When the rule fires, the Lambda receives the parameter ID, generates a fresh password, and updates the SSM entry. Because ESC reads directly from SSM, any subsequent Pulumi deployments automatically pick up the new secret without code changes.

Managing Access with IAM Policies

ESC integrates with the underlying cloud provider’s IAM system, allowing fine‑grained permissions such as “read‑only access to dev secrets” or “full rotation rights in prod”. Below is an example of an AWS IAM policy that grants a CI role permission to read secrets from the dev namespace while denying writes.

policy = aws.iam.Policy("EscReadOnlyDev",
    description="Allows read‑only access to ESC secrets in dev",
    policy=pulumi.Output.all().apply(lambda _: {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "ssm:GetParameter",
                    "ssm:GetParameters",
                    "kms:Decrypt"
                ],
                "Resource": [
                    "arn:aws:ssm:*:*:parameter/esc/dev/*",
                    "arn:aws:kms:*:*:key/*"
                ]
            },
            {
                "Effect": "Deny",
                "Action": [
                    "ssm:PutParameter",
                    "ssm:DeleteParameter",
                    "kms:Encrypt"
                ],
                "Resource": "*"
            }
        ]
    }))

Attach this policy to your CI role, and any Pulumi job that runs against the dev stack will be able to fetch secrets but never modify them. This separation of duties is a core security best practice for larger teams.

Escaping Common Pitfalls

Even with ESC, teams can stumble over a few subtle issues. Here are the most frequent traps and how to avoid them:

  1. Forgot to enable ESC on a new stack. Pulumi will silently fall back to the legacy secret store, leading to accidental plaintext writes. Always run pulumi esc enable as part of your stack bootstrap script.
  2. Mixing secret and non‑secret config keys. If you store a password in a non‑secret config entry, Pulumi will log it in clear text. Use config.set_secret or the --secret flag consistently.
  3. Hard‑coding resource ARNs. ESC resources (KMS keys, SSM parameters) have region‑specific ARNs. Prefer using Pulumi outputs to propagate ARNs between stacks.

Pro tip: Add a pre‑commit hook that runs pulumi config get --show-secrets on a dry‑run and fails if any secret appears in the output. This catches accidental leakage early.

Integrating ESC with CI/CD Pipelines

Most teams run Pulumi from GitHub Actions, GitLab CI, or Azure Pipelines. The key to a smooth integration is to grant the pipeline’s service principal just enough permissions to read (and optionally rotate) secrets. Below is a minimal GitHub Actions workflow that logs into AWS, enables ESC, and runs a Pulumi preview.

name: Pulumi Deploy

on:
  push:
    branches: [ main ]

jobs:
  preview:
    runs-on: ubuntu-latest
    permissions:
      id-token: write   # needed for OIDC
    steps:
      - uses: actions/checkout@v3

      - name: Configure AWS OIDC
        uses: aws-actions/configure-aws-credentials@v2
        with:
          role-to-assume: arn:aws:iam::123456789012:role/github-actions-pulumi
          aws-region: us-east-1

      - name: Install Pulumi
        run: |
          curl -fsSL https://get.pulumi.com | sh
          export PATH=$PATH:$HOME/.pulumi/bin

      - name: Enable ESC (once per stack)
        run: |
          pulumi login --cloud-url s3://my-pulumi-state
          pulumi stack select dev
          pulumi esc enable --provider aws

      - name: Pulumi Preview
        run: pulumi preview --stack dev
        env:
          PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

The workflow uses OIDC to assume an AWS role that already has the read‑only ESC policy attached. No static credentials are stored in the repository, and the preview step will fail if any secret is inadvertently printed.

Advanced Pattern: Dynamic Secrets for Short‑Lived Jobs

Some workloads, like CI runners or temporary batch jobs, benefit from secrets that expire after a few hours. ESC can generate these dynamic secrets on demand using AWS STS or GCP IAM short‑lived tokens. The pattern is:

  1. Store a long‑lived master credential in ESC.
  2. At runtime, request a short‑lived token from the cloud provider’s STS service.
  3. Pass the token to the workload; it automatically expires.

Below is a Python helper that fetches a temporary AWS session token using a master secret stored in ESC:

import pulumi
import pulumi_aws as aws
import boto3

config = pulumi.Config()
master_access_key = config.require_secret("awsMasterAccessKey")
master_secret_key = config.require_secret("awsMasterSecretKey")

def get_temp_credentials():
    # Pulumi secret outputs are resolved at runtime
    return pulumi.Output.all(master_access_key, master_secret_key).apply(
        lambda creds: boto3.client(
            "sts",
            aws_access_key_id=creds[0],
            aws_secret_access_key=creds[1]
        ).get_session_token(DurationSeconds=3600)["Credentials"]
    )

temp_creds = get_temp_credentials()

# Example: launch an EC2 instance using the temporary credentials
def launch_instance(creds):
    ec2 = boto3.client(
        "ec2",
        aws_access_key_id=creds["AccessKeyId"],
        aws_secret_access_key=creds["SecretAccessKey"],
        aws_session_token=creds["SessionToken"]
    )
    # ... launch logic ...

temp_creds.apply(launch_instance)

This approach ensures that even if a temporary token is compromised, its usefulness is limited to a short window, aligning with the principle of least privilege.

Monitoring and Auditing ESC Activity

Visibility into who accessed which secret and when is essential for compliance. ESC automatically emits CloudWatch (AWS), Azure Monitor (Azure), or Cloud Logging (GCP) events for every read, write, or rotation operation. By creating a log‑based metric and attaching an alert, you can be notified of anomalous access patterns.

# AWS CloudWatch metric filter for ESC reads
read_filter = aws.cloudwatch.LogMetricFilter("EscReadFilter",
    log_group_name="/aws/ssm/parameters",
    pattern="{ $.eventName = GetParameter && $.requestParameters.name = \"esc/dev/*\" }",
    metric_transformation=aws.cloudwatch.LogMetricFilterMetricTransformationArgs(
        name="EscReadCount",
        namespace="PulumiESC",
        value="1"
    ))

# Alarm if more than 100 reads in 5 minutes (possible credential abuse)
alarm = aws.cloudwatch.MetricAlarm("EscReadAlarm",
    alarm_name="HighEscReadVolume",
    comparison_operator="GreaterThanThreshold",
    evaluation_periods=1,
    metric_name="EscReadCount",
    namespace="PulumiESC",
    period=300,
    statistic="Sum",
    threshold=100,
    alarm_actions=[sns_topic.arn])

Similar configurations exist for Azure and GCP, leveraging their native monitoring solutions. The key takeaway is that ESC doesn’t hide activity; it makes it observable.

Best Practices Checklist

  • Enable ESC on every stack during the initial bootstrap.
  • Never store production secrets in plain‑text Pulumi config.
  • Use stack references to share common secrets instead of duplicating them.
  • Apply least‑privilege IAM policies to CI roles and developers.
  • Implement automatic rotation for high‑risk credentials.
  • Monitor read/write events and set alerts for unusual spikes.
  • Prefer short‑lived dynamic tokens for transient workloads.

Pro tip: Store the ESC enablement command in a shared Makefile or scripts/bootstrap.sh. This guarantees every new developer runs the exact same steps, eliminating “my stack works but yours doesn’t”

Share this article
Back to Blog