Securely Managing Secrets in Azure Pipelines

1. What is Secret Management?

Secret Management is the process of securely storing, accessing, and protecting sensitive information used by applications and CI/CD pipelines.

Examples of secrets include:

Database passwords and connection strings
API keys
JWT signing keys
Access tokens
Service credentials
Certificates and private keys

Main purpose: Prevent sensitive information from being exposed in source code, YAML files, pipeline logs, or unauthorized environments.

2. How do we manage secrets in Azure Pipelines?

Azure DevOps provides three common approaches.

Method	Description	Recommended use
Secret Pipeline Variables	Store encrypted secret values in pipeline settings	Simple, pipeline-specific secrets
Variable Groups	Centrally manage and share secret variables	Multiple pipelines
Azure Key Vault	Securely store and centrally manage secrets in Azure	Enterprise and Production environments

Recommended approach: Use Azure Key Vault for Production secrets and access them securely from Azure Pipelines using a service connection.

3. Method 1 – Secret Pipeline Variables

Azure DevOps allows you to create secret variables directly in the pipeline.

Steps

Open Azure DevOps → Pipelines.
Select your pipeline and click Edit.
Open Variables.
Create a variable named dbPassword.
Enter the password and select Keep this value secret.
Save the pipeline.

YAML example

steps:
- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: |
      Write-Host "Running database operation"
      # DB_PASSWORD is available to this process
  env:
    DB_PASSWORD: $(dbPassword)

The secret is mapped to an environment variable for the task.

Important: Secret variables are not automatically exposed as environment variables. Explicit mapping is recommended.

Azure DevOps attempts to mask secret values in pipeline logs, but you must still avoid printing or exposing them.

4. Method 2 – Variable Groups

Variable Groups allow multiple pipelines to share secrets securely.

Example

Create a Variable Group named:

PROD-Secrets

Variable	Value	Type
`dbPassword`	********	Secret
`apiKey`	********	Secret
`jwtSigningKey`	********	Secret
`environmentName`	Production	Normal

Reference the group in YAML:

variables:
- group: PROD-Secrets

steps:
- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: |
      Write-Host "Production configuration loaded"
  env:
    DB_PASSWORD: $(dbPassword)
    API_KEY: $(apiKey)

The pipeline must be authorized to use the Variable Group.

Advantage: Multiple applications can reuse centrally managed secrets without storing them in individual YAML files.

5. Method 3 – Azure Key Vault (Recommended for Production)

Azure Key Vault is an Azure service used to securely store and manage secrets, cryptographic keys, and certificates.

Instead of storing sensitive values directly in Azure DevOps, you store them in Azure Key Vault.

How it works

Azure Key Vault
  |
  | Stores:
  | - Database Password
  | - API Key
  | - JWT Signing Key
  |
  v
Azure Service Connection
  |
  v
Azure DevOps Pipeline
  |
  v
Retrieve Required Secrets
  |
  v
Deploy ASP.NET Core Web API
  |
  v
Application Uses Secrets Securely

Step 1 – Create Azure Key Vault

In the Azure Portal:

Create an Azure Key Vault.
Open Objects → Secrets.
Add secrets such as DbPassword and ApiKey.
Configure appropriate permissions for the identity that will access them.

Step 2 – Create an Azure Service Connection

Navigate to:

Azure DevOps → Project Settings → Service connections

Create an Azure Resource Manager service connection.

Where supported, use workload identity federation rather than storing a client secret.

Grant the connection's identity the required Key Vault permissions.

Step 3 – Retrieve Secrets in YAML

Use the AzureKeyVault@2 task.

steps:
- task: AzureKeyVault@2
  displayName: 'Retrieve Secrets'
  inputs:
    azureSubscription: 'Production-ServiceConnection'
    KeyVaultName: 'kv-employee-prod'
    SecretsFilter: 'DbPassword,ApiKey'
    RunAsPreJob: false

- task: PowerShell@2
  displayName: 'Use Secrets'
  inputs:
    targetType: 'inline'
    script: |
      Write-Host "Secrets retrieved securely"
  env:
    DB_PASSWORD: $(DbPassword)
    API_KEY: $(ApiKey)

The Key Vault task retrieves the selected secrets and makes them available as secret pipeline variables for subsequent tasks in the job.

The example assumes the service connection has the necessary Key Vault permissions and network access.

6. How do we manage secrets for DEV, SIT, UAT and Production?

Use separate Key Vaults for different environments where isolation is required.

Azure Key Vault
       |
       +-- kv-employee-dev
       |
       +-- kv-employee-sit
       |
       +-- kv-employee-uat
       |
       +-- kv-employee-prod

Each deployment stage accesses only its authorized secrets.

Environment	Key Vault
DEV	`kv-employee-dev`
SIT	`kv-employee-sit`
UAT	`kv-employee-uat`
Production	`kv-employee-prod`

This provides stronger isolation and reduces the risk of non-production pipelines accessing Production credentials.

7. How does ASP.NET Core use these secrets?

Consider an ASP.NET Core Web API deployed to Azure App Service.

Instead of placing the database password inside appsettings.Production.json, configure the connection string securely through App Service application settings or Key Vault references.

For example, the App Service connection-string setting can be named:

ConnectionStrings__DefaultConnection

ASP.NET Core can read it using:

var connectionString = builder.Configuration
    .GetConnectionString("DefaultConnection");

For Azure-hosted applications, Managed Identity with Azure Key Vault is often preferable because the application can retrieve secrets without storing additional credentials.

Where supported, passwordless authentication to Azure SQL is even better than storing a database password.

8. Important Security Best Practices

Practice	Why it matters
Never hardcode secrets in YAML	Prevents exposure in source control
Never commit passwords to Git	Protects credentials from repository access
Use Azure Key Vault	Centralizes secret management
Use secret variables	Helps protect sensitive pipeline values
Restrict Variable Group permissions	Prevents unauthorized pipeline access
Use least-privilege access	Limits what a compromised identity can access
Use workload identity federation	Avoids long-lived service connection secrets
Rotate secrets regularly	Reduces exposure from compromised credentials
Avoid logging secrets	Prevents accidental disclosure
Separate Production secrets	Improves environment isolation

Important: Never assume that log masking guarantees complete protection. Secrets can still leak through scripts, command-line arguments, generated files, or external tools.

9. Important Interview Questions

Q1. How do you securely store passwords in Azure Pipelines?

Use Secret Pipeline Variables, secret Variable Groups, or Azure Key Vault. For Production, Azure Key Vault is generally preferred.

Q2. What is the difference between a Secret Variable and Azure Key Vault?

A Secret Variable is securely stored within Azure DevOps. Azure Key Vault is a dedicated Azure service for managing secrets, keys, and certificates, with centralized access control and auditing capabilities.

Q3. How does Azure Pipeline access Azure Key Vault?

Using an authorized Azure service connection and the AzureKeyVault@2 task, or through a Key Vault-linked Variable Group.

Q4. How do you prevent developers from accessing Production secrets?

Use restricted Variable Group permissions, Key Vault RBAC, separate service connections, and Production environment approvals.

Q5. Can secret variables be displayed in pipeline logs?

Azure DevOps attempts to mask registered secrets, but developers should never intentionally print them. Masking is not guaranteed for every possible exposure.

Q6. What happens if an API key is accidentally committed to Git?

Immediately revoke or rotate the key, investigate its exposure, update the secure secret store, and remove the exposed value from the repository history where appropriate. Deleting the latest commit alone is insufficient.

10. Interview-ready Answer

"In Azure DevOps, we securely manage passwords, connection strings, API keys, and other sensitive values using Secret Variables, Variable Groups, and Azure Key Vault.

For Production environments, we prefer Azure Key Vault. We configure an Azure service connection using workload identity federation and retrieve secrets securely during deployment.

We maintain separate secrets for DEV, SIT, UAT, and Production and restrict access using appropriate permissions.

We never hardcode secrets in YAML files or source code, and we avoid exposing sensitive values in pipeline logs.

For Azure-hosted applications, we also use Managed Identity wherever possible to eliminate unnecessary stored credentials."

Key takeaway: Azure Key Vault stores and manages secrets, Azure Pipelines retrieves them through authorized identities, and the deployed application receives them securely without hardcoding sensitive values.