Secret Rotation

Overview

SanMarcSoft uses the pass password store as the source of truth for all secrets. Secrets are then distributed to their respective services via Pulumi config, Cloudflare Worker secrets API, or environment variables.

Pass Store Structure

pass/
  cloudflare/
    api-token              -- Cloudflare API token (all zones)
    account-id             -- Cloudflare account ID
    zones/
      sanmarcsoft-com      -- Zone ID for sanmarcsoft.com
      verifieddit-com      -- Zone ID for verifieddit.com
      matthewstevens-org   -- Zone ID for matthewstevens.org
  sanmarcsoft/
    scaleway/
      access-key           -- Scaleway API access key
      api-secret           -- Scaleway API secret key (also used for registry auth)
  verifieddit/
    clerk/
      secret-key           -- Clerk production secret key
      test-secret-key      -- Clerk testing secret key
      publishable-key      -- Clerk production publishable key (pk_live_*)
    stripe/
      secret-key           -- Stripe API secret key
      webhook-secret       -- Stripe webhook signing secret
  sightengine/
    api-user               -- Sightengine API user ID
    api-secret             -- Sightengine API secret key
  aws/
    account-id             -- AWS account ID
    phenom-drop/
      ecr-repo             -- ECR repository URL
      apprunner-arn        -- App Runner service ARN

Rotation Procedures

1. Rotate Cloudflare API Token

1. Generate new token in Cloudflare Dashboard > My Profile > API Tokens
2. Update pass store:

   1	pass edit cloudflare/api-token

3. Test the new token:

   1 2	curl -s -H "Authorization: Bearer $(pass cloudflare/api-token)" \ "https://api.cloudflare.com/client/v4/user/tokens/verify" | jq '.success'

4. No service restart needed (token is read from pass at invocation time)

2. Rotate Scaleway API Secret

1. Generate new API key in Scaleway Console > IAM > API Keys
2. Update pass store:

   1 2	pass edit sanmarcsoft/scaleway/api-secret pass edit sanmarcsoft/scaleway/access-key # if access key also changed

3. Update Pulumi config for all stacks:

   1 2	cd infra pulumi config set --secret scaleway-secret-key "$(pass sanmarcsoft/scaleway/api-secret)"

4. Test registry access:

   1 2 3

   skopeo list-tags \ "docker://rg.fr-par.scw.cloud/sanmarcsoft/verifieddit-www" \ --creds "nologin:$(pass sanmarcsoft/scaleway/api-secret)"

3. Rotate Clerk Secret Key

1. Rotate in Clerk Dashboard > API Keys
2. Update pass store:

   1	pass edit verifieddit/clerk/secret-key

3. Update Cloudflare Worker secret:

   1 2 3 4 5 6 7 8 9 10 11 12

   CF_TOKEN=$(pass cloudflare/api-token) ACCOUNT_ID=$(pass cloudflare/account-id) curl -s -X PUT \ "https://api.cloudflare.com/client/v4/accounts/${ACCOUNT_ID}/workers/scripts/verifieddit-badges/secrets" \ -H "Authorization: Bearer ${CF_TOKEN}" \ -H "Content-Type: application/json" \ --data '{ "name": "CLERK_SECRET_KEY", "text": "'"$(pass verifieddit/clerk/secret-key)"'", "type": "secret_text" }'

4. Update Pulumi config for stripe-backend:

   1 2 3

   cd services/stripe-backend/infra pulumi config set --secret clerk-secret-key "$(pass verifieddit/clerk/secret-key)" pulumi up

4. Rotate Stripe Keys

1. Roll keys in Stripe Dashboard > Developers > API Keys
2. Update pass store:

   1 2	pass edit verifieddit/stripe/secret-key pass edit verifieddit/stripe/webhook-secret

3. Update Pulumi config:

   1 2 3 4

   cd services/stripe-backend/infra pulumi config set --secret stripe-secret-key "$(pass verifieddit/stripe/secret-key)" pulumi config set --secret stripe-webhook-secret "$(pass verifieddit/stripe/webhook-secret)" pulumi up

5. Rotate Sightengine API Keys

1. Regenerate in Sightengine Dashboard
2. Update pass store:

   1 2	pass edit sightengine/api-user pass edit sightengine/api-secret

3. Update Cloudflare Worker secrets:

   1 2 3 4 5 6 7 8

   for SECRET in SIGHTENGINE_API_USER SIGHTENGINE_API_SECRET; do PASS_KEY=$(echo $SECRET | tr '[:upper:]' '[:lower:]' | sed 's/_/-/g' | sed 's/sightengine-/sightengine\//') curl -s -X PUT \ "https://api.cloudflare.com/client/v4/accounts/${ACCOUNT_ID}/workers/scripts/verifieddit-badges/secrets" \ -H "Authorization: Bearer ${CF_TOKEN}" \ -H "Content-Type: application/json" \ --data "{\"name\": \"${SECRET}\", \"text\": \"$(pass ${PASS_KEY})\", \"type\": \"secret_text\"}" done

Pulumi Secrets

Pulumi encrypts secrets in the stack state file. The state backend is Scaleway Object Storage (fr-par).

Setting a Pulumi Secret

1
2

cd infra
pulumi config set --secret <key> <value>

Viewing Encrypted Config

1

pulumi config --show-secrets

State Backend Location

s3://sanmarcsoft-pulumi-state (fr-par)

Access via Scaleway Object Storage credentials.

Security Checklist

When rotating any secret:

• Update pass store first (source of truth)
• Update all consumers (Workers, Pulumi, CI/CD)
• Test each consumer still works
• Revoke the old secret/token
• Document the rotation date

Troubleshooting

• “Unauthorized” after rotation: Check all consumers were updated. A single missed consumer will fail.
• Pulumi state locked: Another pulumi up may be running. Wait or force unlock with pulumi cancel.
• Pass store sync: Ensure pass store changes are committed and pushed to the git remote for backup.