Docker Compose – Complete Detailed Explanation

1. What is Docker Compose?

Docker Compose is a tool used to define, configure, create, and manage multiple Docker containers together using a single YAML configuration file.

Normally, when working with Docker, we create and run containers individually using docker run commands.

For example, imagine you are developing an ASP.NET Core Web API application that requires:

1. ASP.NET Core Web API

2. SQL Server Database

3. Redis Cache

Without Docker Compose, you need to create and start each container separately.

docker run -d --name sqlserver ...
docker run -d --name redis ...
docker run -d --name myapi ...

You also need to configure:

• Container networking.

• Port mapping.

• Environment variables.

• Volume mapping.

• Container dependencies.

Managing all these configurations separately becomes difficult.

Docker Compose solves this problem.

Instead of executing multiple Docker commands, you define all the services in one YAML file called:

compose.yaml

Then execute a single command:

docker compose up -d

Docker Compose automatically creates and starts all the required containers based on the YAML configuration.

2. Why Do We Need Docker Compose?

Consider a real-world application architecture.

Suppose you are developing an e-commerce application.

Your application contains:

• Angular frontend.

• ASP.NET Core Web API.

• SQL Server database.

• Redis cache.

Without Docker Compose:

Developer
   |
   |---- docker run Angular
   |
   |---- docker run ASP.NET Core API
   |
   |---- docker run SQL Server
   |
   |---- docker run Redis

You must manually configure each container.

With Docker Compose:

             Docker Compose
                   |
             compose.yaml
                   |
          ---------------------
          |        |          |
       Angular    API      SQL Server
          |        |          |
          |        |          |
          |        +----------+
          |             |
          |           Redis
          |
       All containers
       managed together

You can start the entire application using one command.

docker compose up -d

Main advantages:

1. Multiple Container Management: Manage multiple containers together.

2. Single Configuration: Define all container configurations in one YAML file.

3. Networking: Automatically creates a network for communication between services.

4. Volume Management: Configure persistent storage for containers.

5. Environment Variables: Pass configuration values to containers.

6. Easy Startup: Start all services using one command.

7. Easy Shutdown: Stop all services using one command.

8. Reusability: Share the Compose file with other developers.

3. What is a Service in Docker Compose?

A service represents an application component that Docker Compose manages.

For example:

Important distinction:

• Service: Defines how a container or group of containers should run.

• Container: The actual running instance created from the service configuration.

• Image: The template used to create a container.

Example:

services:
  api:
    image: myapi:v1

Here:

• services is the root section.

• api is the service name.

• image specifies the Docker image.

• myapi:v1 is the image name and tag.

Docker Compose uses this configuration to create a container for the API service.

4. What is a compose.yaml File?

compose.yaml is the configuration file used by Docker Compose.

It contains instructions about how Docker should create and manage application services.

Example:

services:

  web:
    image: nginx:latest
    ports:
      - "8080:80"

  redis:
    image: redis:latest

Let's understand each line.

Line 1:

services:

Defines the services required by your application.

Line 2:

web:

Defines a service named web.

Line 3:

image: nginx:latest

Specifies that Docker should use the Nginx image.

Line 4:

ports:
  - "8080:80"

Maps the Windows host port 8080 to the container port 80.

Line 5:

redis:

Defines another service named redis.

Line 6:

image: redis:latest

Uses the Redis image.

When you execute:

docker compose up -d

Docker Compose creates and starts both services.

5. Docker Compose Architecture

Let's understand how Docker Compose works internally.

The overall process:

1. Developer executes docker compose up.

2. Docker Compose reads compose.yaml.

3. Docker checks whether the required images are available.

4. If images are missing, Docker downloads or builds them.

5. Docker creates the required network and volumes.

6. Docker creates containers for the configured services.

7. Docker starts the containers.

6. Important Docker Compose Properties

Let's understand the commonly used properties in compose.yaml.

6.1 services

Defines all application services.

services:
  api:
    image: myapi:v1

  db:
    image: redis:latest

Here, we have two services: api and db.

6.2 image

Specifies which Docker image should be used to create the container.

services:
  web:
    image: nginx:latest

Docker uses the Nginx image from Docker Hub if it is not already available locally.

6.3 build

Used when you want Docker Compose to build an image using a Dockerfile.

Example project structure:

MyApplication/
│
├── compose.yaml
│
└── MyApi/
    ├── Dockerfile
    ├── Program.cs
    └── MyApi.csproj

Compose configuration:

services:
  api:
    build:
      context: ./MyApi
      dockerfile: Dockerfile

Explanation:

• build: Tells Docker Compose to build an image.

• context: Specifies the build context directory.

• dockerfile: Specifies the Dockerfile name.

When you execute:

docker compose up -d --build

Docker Compose builds the image and starts the container.

6.4 ports

Used to expose container ports to the host machine.

Syntax:

ports:
  - "HOST_PORT:CONTAINER_PORT"

Example:

services:
  api:
    image: myapi:v1
    ports:
      - "8080:8080"

Explanation:

You can access the application through:

http://localhost:8080

6.5 volumes

Used to persist data or share files between the host and container.

Example:

services:
  db:
    image: mysql:8
    volumes:
      - mysql-data:/var/lib/mysql

volumes:
  mysql-data:

Here:

• mysql-data is a named Docker volume.

• /var/lib/mysql is the data directory inside the MySQL container.

Even if the container is deleted, the data remains in the Docker volume.

6.6 environment

Used to pass environment variables to containers.

Example:

services:
  api:
    image: myapi:v1
    environment:
      ASPNETCORE_ENVIRONMENT: Development
      ConnectionStrings__DefaultConnection: "Server=db;Database=MyDB"

ASP.NET Core can read these environment variables through its configuration system.

For database passwords and other sensitive information, avoid hardcoding production secrets in Compose files. Use appropriate secrets management.

6.7 depends_on

Defines the startup and shutdown dependency between services.

Example:

services:
  api:
    image: myapi:v1
    depends_on:
      - db

  db:
    image: mysql:8

This indicates that the database service should be started before the API service.

Important: Basic depends_on controls startup ordering. It does not guarantee that the database is ready to accept connections.

For readiness checking, use a health check and a service_healthy dependency.

Example:

services:
  api:
    image: myapi:v1
    depends_on:
      db:
        condition: service_healthy

  db:
    image: mysql:8
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 5

6.8 restart

Defines when Docker should restart a container.

services:
  api:
    image: myapi:v1
    restart: always

Common restart policies:

6.9 container_name

Allows you to assign a custom name to a container.

services:
  api:
    image: myapi:v1
    container_name: my-api-container

You can inspect it using:

docker ps

Note: Explicit container names prevent scaling that service to multiple containers using Compose.

6.10 networks

Defines communication networks between services.

services:
  api:
    image: myapi:v1
    networks:
      - app-network

  db:
    image: mysql:8
    networks:
      - app-network

networks:
  app-network:

Both containers can communicate using service names.

For example, the API can connect to MySQL using:

Server=db

It does not need the database container's IP address.

6.11 command

Overrides the default command defined in the Docker image.

services:
  web:
    image: nginx:latest
    command: ["nginx", "-g", "daemon off;"]

6.12 entrypoint

Overrides the default entrypoint of the Docker image.

services:
  api:
    image: myapi:v1
    entrypoint: ["dotnet", "MyApi.dll"]

Difference:

• command: Usually provides or overrides the default command arguments.

• entrypoint: Specifies the main executable or startup process.

7. Practical Example – Multiple Containers Using Docker Compose

Let's create a practical example using:

1. Nginx

2. Redis

We will run both containers using Docker Compose.

Step 1: Create a Project Folder

Open Windows CMD.

mkdir DockerComposeDemo
cd DockerComposeDemo

Step 2: Create compose.yaml

Create a file named:

compose.yaml

Add the following configuration:

services:

  web:
    image: nginx:latest
    container_name: compose-nginx
    ports:
      - "8080:80"

  redis:
    image: redis:latest
    container_name: compose-redis
    ports:
      - "6379:6379"

Project structure:

DockerComposeDemo/
│
└── compose.yaml

Step 3: Start the Containers

Execute:

docker compose up -d

What happens?

1. Docker checks for the Nginx image.

2. Docker downloads the image if necessary.

3. Docker checks for the Redis image.

4. Docker downloads the image if necessary.

5. Docker creates the containers.

6. Docker starts both containers.

Both services should be running.

Step 4: Check Running Containers

Execute:

docker compose ps

Expected output (container IDs and other details will differ):

NAME            IMAGE          STATUS
compose-nginx   nginx:latest   Up
compose-redis   redis:latest   Up

You can also execute:

docker ps

Step 5: Access Nginx

Open your browser:

http://localhost:8080

You should see the Nginx welcome page.

Step 6: Test Redis

Execute:

docker exec -it compose-redis redis-cli

Inside Redis CLI:

SET username Syed
GET username

Expected output:

"OK"
"Syed"

Exit Redis CLI:

exit

Step 7: Stop All Containers

Execute:

docker compose stop

This stops both containers but does not delete them.

Step 8: Start Again

docker compose start

Both existing containers start again.

Step 9: Remove Containers

docker compose down

This removes the Compose containers and its default network.

Important: Named volumes are not removed by default.

8. Docker Compose Networking

One of the most important features of Docker Compose is automatic networking.

Suppose you have two services:

services:
  api:
    image: myapi:v1

  db:
    image: mysql:8

Docker Compose automatically creates a default network for the application.

Conceptually:

       Docker Compose Network
       (DockerComposeDemo_default)
                  |
          -----------------
          |               |
      API Container   DB Container
          |               |
          |---------------|
          |
     Server=db

The API container can communicate with the database using the service name:

db

Example connection string:

Server=db;Port=3306;Database=MyDB;

Important points:

1. Services in the same Compose network can communicate.

2. Service names are used for DNS-based service discovery.

3. You generally don't need to know container IP addresses.

4. Container-to-container communication uses container ports.

5. Host port mapping is not required for internal communication.

For example:

ports:
  - "3307:3306"

Here:

• Windows accesses MySQL using port 3307.

• Other containers access MySQL using db:3306.

9. Docker Compose vs Docker Run

Both are used to create and run containers, but their approaches are different.

Example using Docker Run:

docker run -d -p 8080:80 nginx
docker run -d -p 6379:6379 redis

Example using Docker Compose:

docker compose up -d

Both approaches can run the same two services.

10. Important Docker Compose Commands

These are the commands you should know.

Important: Docker Compose V2 uses the command docker compose (with a space). The older standalone version used docker-compose (with a hyphen).

11. Practical Example – ASP.NET Core Web API + SQL Server

Now let's understand a more realistic example from .NET development.

Suppose you have an ASP.NET Core Web API application that requires SQL Server.

Without Docker Compose:

Windows
  |
  |---- ASP.NET Core API
  |
  |---- SQL Server Container

You need to manually configure and start SQL Server.

With Docker Compose:

             Docker Compose
                   |
               compose.yaml
                   |
           -------------------
           |                 |
      API Container    SQL Container
           |                 |
           |-----------------|
             Docker Network

Step 1: Create Project Structure

DockerComposeDotNet/
│
├── compose.yaml
│
└── MyApi/
    ├── Dockerfile
    ├── Program.cs
    └── MyApi.csproj

Step 2: Create Dockerfile

Inside the MyApi folder, create a Dockerfile.

Example for a .NET 8 application:

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build

WORKDIR /src

COPY . .

RUN dotnet publish -c Release -o /app/publish

FROM mcr.microsoft.com/dotnet/aspnet:8.0

WORKDIR /app

COPY --from=build /app/publish .

EXPOSE 8080

ENTRYPOINT ["dotnet", "MyApi.dll"]

Replace MyApi.dll with your actual published application DLL name.

Step 3: Create compose.yaml

services:

  api:
    build:
      context: ./MyApi
      dockerfile: Dockerfile

    ports:
      - "8080:8080"

    environment:
      ASPNETCORE_ENVIRONMENT: Development

      ConnectionStrings__DefaultConnection: "Server=db,1433;Database=MyDB;User Id=sa;Password=YourStrong!Passw0rd;TrustServerCertificate=True;"

    depends_on:
      db:
        condition: service_healthy

  db:
    image: mcr.microsoft.com/mssql/server:2022-latest

    environment:
      ACCEPT_EULA: "Y"
      MSSQL_SA_PASSWORD: "YourStrong!Passw0rd"

    ports:
      - "1433:1433"

    volumes:
      - sql-data:/var/opt/mssql

    healthcheck:
      test: ["CMD-SHELL", "/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P \"$$MSSQL_SA_PASSWORD\" -C -Q \"SELECT 1\" || exit 1"]
      interval: 10s
      timeout: 5s
      retries: 10
      start_period: 30s

volumes:
  sql-data:

Note: This is a development example. Use a secure secret-management approach for production credentials. The SQL Server health check assumes the SQL Server 2022 image contains the tools18 sqlcmd utility at the specified path.

Step 4: Run the Application

Execute:

docker compose up -d --build

Docker Compose will:

1. Build the ASP.NET Core API image.

2. Pull the SQL Server image if required.

3. Create the SQL Server volume.

4. Create the default network.

5. Start the SQL Server container.

6. Wait for the SQL Server health check.

7. Start the API container.

Access the API:

http://localhost:8080

The API connects to SQL Server using:

Server=db,1433

Notice that we used db instead of localhost.

Why?

Because the API and SQL Server are running in separate containers.

Inside the API container, localhost refers to the API container itself, not the SQL Server container.

12. Docker Compose Build vs Image

This is an important concept.

You can define a service using either image or build.

Using image:

services:
  web:
    image: nginx:latest

Docker uses an existing image, pulling it if necessary.

Using build:

services:
  api:
    build:
      context: ./MyApi

Docker builds an image using the Dockerfile in the specified build context.

Using both:

services:
  api:
    build:
      context: ./MyApi
    image: myapi:v1

Docker builds the image from the Dockerfile and tags the resulting image as myapi:v1.

13. Docker Compose Profiles

Profiles allow you to selectively start certain services.

For example, suppose your application contains:

• API

• Database

• Redis

• Monitoring tool

You may not need the monitoring tool every time.

Example:

services:

  api:
    image: myapi:v1

  db:
    image: mysql:8

  redis:
    image: redis:latest

  monitoring:
    image: adminer
    profiles:
      - debug
    ports:
      - "8081:8080"

Start normal services:

docker compose up -d

Start services with the debug profile:

docker compose --profile debug up -d

The monitoring service starts only when its profile is enabled or it is explicitly targeted.

14. Docker Compose Scaling

Suppose you want to run multiple instances of your API service.

You can use:

docker compose up -d --scale api=3

This creates three containers for the API service.

Conceptually:

          Docker Compose
                |
          API Service
                |
       -------------------
       |        |        |
    API-1    API-2    API-3

Important considerations:

• Do not use a fixed container_name when scaling.

• Do not publish the same fixed host port for every replica.

• Use a reverse proxy or load balancer to distribute incoming requests.

• Your application should be designed to support multiple instances.

15. Docker Compose Lifecycle

Let's understand the complete lifecycle.

Remember:

Warning: docker compose down -v can permanently remove the named volumes and their stored data.

16. Docker Compose vs Kubernetes

Both technologies manage containerized applications, but they serve different purposes.

When to use Docker Compose?

• Local development.

• Learning Docker.

• Testing applications.

• Running multiple dependent containers.

• Small deployments.

When to use Kubernetes?

• Large-scale containerized applications.

• Production orchestration.

• Advanced scaling.

• High availability.

• Complex deployments across multiple nodes.

17. Important Points to Remember

1. Docker Compose manages multiple containers using a single YAML file.

2. The default configuration file is compose.yaml.

3. A service represents an application component.

4. Docker Compose automatically creates a default network.

5. Containers can communicate using service names.

6. ports maps host ports to container ports.

7. volumes provides persistent storage.

8. depends_on controls service startup ordering.

9. docker compose up -d starts services in the background.

10. docker compose down removes containers and the default network.

11. docker compose down -v also removes associated named volumes.

12. Docker Compose is particularly useful for local development and testing.

18. Important Docker Compose Interview Questions

Here are some important questions to prepare.

1. What is Docker Compose?

2. Why do we need Docker Compose?

3. What is the difference between Docker and Docker Compose?

4. What is a service in Docker Compose?

5. What is the purpose of the compose.yaml file?

6. What is the difference between image and build in Docker Compose?

7. What is the purpose of depends_on?

8. Does depends_on guarantee that a database is ready before the API starts?

9. How does networking work in Docker Compose?

10. How can two containers communicate with each other?

11. What is the difference between ports and expose?

12. How do you persist database data using Docker Compose?

13. What is the difference between docker compose stop and docker compose down?

14. What happens when you execute docker compose down -v?

15. How do you pass environment variables in Docker Compose?

16. What are Docker Compose profiles?

17. How do you scale services in Docker Compose?

18. Can Docker Compose build an image using a Dockerfile?

19. What is the difference between Docker Compose and Kubernetes?

20. Can Docker Compose be used in production?