C# / .NET Compendium PLEN
.NET / C# / Distributed architecture

Docker and Kubernetes - the basics in the context of microservices

Docker - containerising a single service

Docker lets you package an application (code + runtime + dependencies + configuration) into a portable, isolated image that runs identically regardless of the machine it is run on.

# Dockerfile for an ASP.NET Core Web API (exactly that OrderService we talked about)
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /src
COPY *.csproj .
RUN dotnet restore
COPY . .
RUN dotnet publish -c Release -o /app

FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS runtime
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "OrderService.dll"]

A multi-stage build - the first stage (build) has the full SDK (compiler, tools), the second stage (runtime) has only the runtime needed to execute - the final image is much smaller, because it doesn't contain unnecessary build tools.

docker build -t order-service:latest .
docker run -p 5000:80 order-service:latest

Docker Compose - running many related services locally

This is a practical tool for local development of a microservices architecture - in a single file you describe the whole infrastructure (services + database + Kafka), which docker-compose up starts all at once:

version: '3.8'
services:
  order-service:
    build: ./OrderService
    ports:
      - "5001:80"
    environment:
      - ConnectionStrings__DefaultConnection=Server=order-db;Database=OrderDb;User=sa;Password=Pass123!
      - Kafka__BootstrapServers=kafka:9092
    depends_on:
      - order-db
      - kafka

  inventory-service:
    build: ./InventoryService
    ports:
      - "5002:80"
    depends_on:
      - inventory-db
      - kafka

  order-db:
    image: mcr.microsoft.com/mssql/server:2022-latest
    environment:
      - ACCEPT_EULA=Y
      - SA_PASSWORD=Pass123!
    ports:
      - "1433:1433"

  kafka:
    image: confluentinc/cp-kafka:latest
    ports:
      - "9092:9092"
    environment:
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
    depends_on:
      - zookeeper

  zookeeper:
    image: confluentinc/cp-zookeeper:latest

A key architectural observation - each microservice (order-service, inventory-service) has its own dedicated database (order-db, inventory-db) - this is a direct realisation of the principle we discussed with the Saga: there is no single, shared database between services, hence the need for a Saga/Outbox for coordination instead of a classic SQL transaction.

Kubernetes - orchestration in production (a scale that Docker Compose doesn't handle)

Docker Compose is great locally, but it isn't meant for large-scale production - no automatic scaling, no self-healing on failure, no distribution across many physical machines. Kubernetes (K8s) takes care of that.

Pod - the smallest unit of deployment

apiVersion: v1
kind: Pod
metadata:
  name: order-service-pod
spec:
  containers:
    - name: order-service
      image: order-service:latest
      ports:
        - containerPort: 80

A Pod is one or more containers (most often one) sharing network and storage - it is the basic unit that Kubernetes manages (starts, restarts, scales).

Deployment - managing many replicas of a Pod

apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service-deployment
spec:
  replicas: 3 # 3 INDEPENDENT instances of OrderService, for high availability and throughput
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
    spec:
      containers:
        - name: order-service
          image: order-service:latest
          ports:
            - containerPort: 80
          env:
            - name: ConnectionStrings__DefaultConnection
              valueFrom:
                secretKeyRef:
                  name: order-db-secret
                  key: connectionString

A Deployment guarantees that the specified number of replicas is always running (replicas: 3) - if one Pod crashes (a crash, a machine failure), Kubernetes automatically creates a new one to restore the desired number. This is exactly self-healing, which Docker Compose doesn't provide.

A link to Kafka Consumer Groups - if these 3 replicas of OrderService listen to the same Kafka topic as one consumer group, Kubernetes automatically keeps 3 instances running to share the partitions among themselves, exactly as we discussed earlier.

Service - a stable access point to changing Pods

apiVersion: v1
kind: Service
metadata:
  name: order-service
spec:
  selector:
    app: order-service # routes traffic to ALL the Pods with this label
  ports:
    - port: 80
      targetPort: 80
  type: ClusterIP # accessible only INSIDE the cluster (other services can connect)

The problem a Service solves: Pods are ephemeral - they can be created/deleted/restarted, each with its own, changing IP address. A Service provides a stable name (order-service) through which other services (e.g. InventoryService calling OrderService over HTTP) can always connect, regardless of which specific Pods are currently running "underneath" - Kubernetes automatically routes traffic (load balancing) to healthy Pods.

// InventoryService calling OrderService - uses the STABLE Service name, not a specific Pod IP
builder.Services.AddHttpClient<IOrderServiceClient, OrderServiceClient>(client =>
    client.BaseAddress = new Uri("http://order-service")); // "order-service" to nazwa K8s Service

ConfigMap and Secret - configuration and sensitive data

apiVersion: v1
kind: Secret
metadata:
  name: order-db-secret
type: Opaque
stringData:
  connectionString: "Server=order-db;Database=OrderDb;User=sa;Password=Pass123!"

Secret (for sensitive data, like connection strings with passwords) and ConfigMap (for ordinary configuration) let you separate configuration from the container image - the same order-service:latest image can be deployed in different environments (dev/staging/prod) with different configuration, without rebuilding the image.

Summary - how it all fits together into one architecture

Docker packages a single microservice into a portable, isolated image. Docker Compose runs the whole architecture locally (many services + databases + Kafka) for development, while naturally forcing you to think of it as a set of independent services with their own databases (hence the need for a Saga/Outbox for coordination). Kubernetes carries this over to production at large scale - a Deployment guarantees the desired number of always-running replicas (self-healing, automatic scaling), and a Service gives a stable entry point to a dynamically changing set of Pods, through which other services communicate (e.g. over HTTP or by publishing/consuming events from Kafka, as we discussed earlier).

Buy me a coffee