Azure Pipelines – Agents, Agent Pools, Microsoft-hosted vs Self-hosted Agents

1. What is an Agent in Azure Pipelines?

An Agent is a computing machine that executes the jobs defined in an Azure Pipeline.

When a pipeline runs, Azure DevOps assigns a job to an available agent. The agent executes the steps, such as restoring packages, building code, running tests, and deploying applications.

An agent can run on:

• Windows

• Linux

• macOS

An agent may be hosted by Microsoft or managed by your organization.

Simple example

Suppose you have an ASP.NET Core Web API application.

When you push code to Azure Repos:

Developer
    |
    v
Azure Repos (Git)
    |
    v
Azure Pipeline Triggered
    |
    v
Agent
    |
    |-- Restore NuGet Packages
    |-- Build .NET Application
    |-- Run Unit Tests
    |-- Publish Application
    |
    v
Build Artifact Generated

Important: Azure Pipelines controls the workflow, while the agent performs the actual execution.

2. What is an Agent Pool?

An Agent Pool is a collection of agents available to execute pipeline jobs.

Instead of assigning jobs to a particular machine manually, Azure Pipelines selects an available compatible agent from the specified pool.

For example, a company has three build servers:

Why do we need Agent Pools?

• Centralized management of build and deployment agents.

• Multiple pipelines can share available agents.

• Supports parallel job execution.

• Helps manage permissions and access.

• Allows different pools for different requirements.

For example, a company might maintain separate Windows and Linux agent pools.

3. Types of Agents in Azure Pipelines

Azure Pipelines primarily supports two agent-hosting models.

A. Microsoft-hosted Agent

A Microsoft-hosted agent is a virtual machine managed by Microsoft.

Microsoft provides and maintains the machine, operating system, and commonly used development tools.

When a job starts, Azure Pipelines provisions a fresh agent environment for that job.

Example YAML:

pool:
  vmImage: 'windows-latest'

steps:
- script: dotnet restore
  displayName: 'Restore Packages'

- script: dotnet build
  displayName: 'Build Application'

Here:

• vmImage specifies the agent image.

• windows-latest selects a Microsoft-hosted Windows image.

• The agent executes the build commands.

• The virtual machine is discarded after the job finishes.

Advantages:

• No server installation or maintenance.

• Preinstalled development tools.

• Fresh environment for each job.

• Supports Windows, Linux, and macOS.

Disadvantages:

• Limited control over the machine.

• Custom tools may need installation during each job.

• Job execution time and capacity limits depend on the Azure DevOps plan.

• Cannot directly access private on-premises resources without appropriate network connectivity.

B. Self-hosted Agent

A Self-hosted Agent is installed and maintained by your organization.

It can run on:

• An on-premises Windows server.

• A Linux server.

• An Azure Virtual Machine.

• Another supported machine controlled by your organization.

You install the Azure Pipelines agent software, register it with an agent pool, and configure the required development tools.

Example YAML:

pool:
  name: 'MyCompanyPool'

steps:
- script: dotnet restore
  displayName: 'Restore Packages'

- script: dotnet build
  displayName: 'Build Application'

Here:

• MyCompanyPool is a custom agent pool.

• Azure Pipelines selects an available compatible agent from the pool.

• The job runs on your organization's machine.

Advantages:

• Full control over installed software and tools.

• Can access internal resources when network permissions allow.

• Useful for private corporate networks.

• Tools and caches can persist between jobs.

• Supports customized build environments.

Disadvantages:

• Requires installation and configuration.

• Your organization maintains the server and agent software.

• Security updates and maintenance are your responsibility.

• Available capacity depends on your infrastructure.

4. Microsoft-hosted vs Self-hosted Agents

5. How Does Azure Pipelines Select an Agent?

When a pipeline starts:

1. Azure Pipelines identifies the agent pool specified in YAML.

2. It checks available agents and their capabilities.

3. It selects an eligible agent that satisfies the job requirements.

4. The agent executes the pipeline steps.

5. The agent reports the job result to Azure DevOps.

Example: Agent capabilities and demands

Suppose a build requires Visual Studio.

You can specify agent demands:

pool:
  name: 'MyCompanyPool'
  demands:
  - Agent.OS -equals Windows_NT
  - msbuild

steps:
- script: dotnet build

The demands help Azure Pipelines select a self-hosted agent with the required capabilities.

Important: Agent pools provide execution resources, while parallel-job capacity determines how many jobs can execute concurrently.

6. Real-world Example – ASP.NET Core Application

Consider an enterprise application with the following environments:

Azure Repos
    |
    v
CI Pipeline
    |
    v
Microsoft-hosted Agent
    |
    |-- Restore NuGet Packages
    |-- Build ASP.NET Core Web API
    |-- Execute Unit Tests
    |-- Publish Artifact
    |
    v
CD Pipeline
    |
    v
Self-hosted Agent
    |
    |-- Download Artifact
    |-- Deploy to Internal SIT Server
    |-- Deploy to Internal UAT Server
    |-- Deploy to Production (Approval)

In this scenario:

Microsoft-hosted Agent: Used for building and testing the application because the build does not require access to internal corporate servers.

Self-hosted Agent: Used for deployment because the agent can be installed inside the company's private network and configured to access internal IIS servers.

This is a common hybrid approach. However, using a self-hosted agent for deployment is not mandatory if the target environment is accessible securely through another supported deployment method.

7. How Do You Configure a Self-hosted Agent?

In Azure DevOps:

1. Open your Azure DevOps organization.

2. Navigate to Organization Settings → Agent pools.

3. Create a new pool or select an existing pool.

4. Select New agent.

5. Choose the operating system.

6. Download the agent package.

7. Install and configure the agent on your server.

8. Register the agent with the selected pool.

9. Start the agent service.

10. Verify that the agent appears online.

You can then reference the pool in your pipeline:

pool:
  name: 'MyCompanyPool'

The agent requires appropriate authentication during registration, and the pipeline must have permission to use the pool.

8. Important Interview Questions

Q1. What is the difference between an Agent and an Agent Pool?

An Agent is a machine that executes pipeline jobs. An Agent Pool is a collection of agents from which Azure Pipelines selects an available agent.

Q2. Can multiple pipelines use the same Agent Pool?

Yes. Multiple pipelines can use the same pool, subject to permissions and available capacity.

Q3. Can one agent execute multiple jobs simultaneously?

A standard Azure Pipelines agent executes one job at a time. Multiple agents are needed to run multiple agent jobs concurrently.

Q4. When should we use a self-hosted agent?

Use a self-hosted agent when you need custom software, specialized hardware, persistent build environments, or access to private corporate resources.

Q5. Does a Microsoft-hosted agent retain files after a job completes?

No. Microsoft-hosted agents use fresh virtual machines for jobs, so local changes do not persist between jobs. Use pipeline artifacts or caching to transfer or reuse necessary data.

Q6. Can we use both Microsoft-hosted and self-hosted agents in the same pipeline?

Yes. Different jobs or stages can specify different agent pools.

Q7. What happens if no agent is available?

The job remains queued until an eligible agent and the required parallel-job capacity become available, or the job is cancelled or times out.

9. Interview-ready Answer

An Agent in Azure Pipelines is a computing machine responsible for executing pipeline jobs, such as building applications, running tests, and deploying software.

An Agent Pool is a collection of agents used to manage and distribute pipeline workloads.

Azure Pipelines supports Microsoft-hosted and self-hosted agents.

Microsoft-hosted agents are managed by Microsoft and provide ready-to-use environments without requiring infrastructure maintenance.

Self-hosted agents are installed and managed by our organization. They provide greater control over tools, configurations, and access to internal networks.

In a real project, we can use Microsoft-hosted agents for CI builds and self-hosted agents for deploying applications to internal SIT, UAT, and Production servers.

Key point: An Agent executes jobs, an Agent Pool manages agents, and the choice between Microsoft-hosted and self-hosted depends mainly on infrastructure control, software requirements, and network accessibility.