Docker and Kubernetes - the basics in the context of microservices
Docker - containerising a single service
Docker lets you package an application (code + runtime + dependencies + configuration) into a portable, isolated image that runs identically regardless of the machine it is run on.
# Dockerfile for an ASP.NET Core Web API (exactly that OrderService we talked about)
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /src
COPY *.csproj .
RUN dotnet restore
COPY . .
RUN dotnet publish -c Release -o /app
FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS runtime
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "OrderService.dll"]
A multi-stage build - the first stage (build) has the full SDK (compiler, tools), the second stage (runtime) has only the runtime needed to execute - the final image is much smaller, because it doesn't contain unnecessary build tools.
docker build -t order-service:latest .
docker run -p 5000:80 order-service:latest
Docker Compose - running many related services locally
This is a practical tool for local development of a microservices architecture - in a single file you describe the whole infrastructure (services + database + Kafka), which docker-compose up starts all at once:
version: '3.8'
services:
order-service:
build: ./OrderService
ports:
- "5001:80"
environment:
- ConnectionStrings__DefaultConnection=Server=order-db;Database=OrderDb;User=sa;Password=Pass123!
- Kafka__BootstrapServers=kafka:9092
depends_on:
- order-db
- kafka
inventory-service:
build: ./InventoryService
ports:
- "5002:80"
depends_on:
- inventory-db
- kafka
order-db:
image: mcr.microsoft.com/mssql/server:2022-latest
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=Pass123!
ports:
- "1433:1433"
kafka:
image: confluentinc/cp-kafka:latest
ports:
- "9092:9092"
environment:
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
depends_on:
- zookeeper
zookeeper:
image: confluentinc/cp-zookeeper:latest
A key architectural observation - each microservice (order-service, inventory-service) has its own dedicated database (order-db, inventory-db) - this is a direct realisation of the principle we discussed with the Saga: there is no single, shared database between services, hence the need for a Saga/Outbox for coordination instead of a classic SQL transaction.
Kubernetes - orchestration in production (a scale that Docker Compose doesn't handle)
Docker Compose is great locally, but it isn't meant for large-scale production - no automatic scaling, no self-healing on failure, no distribution across many physical machines. Kubernetes (K8s) takes care of that.
Pod - the smallest unit of deployment
apiVersion: v1
kind: Pod
metadata:
name: order-service-pod
spec:
containers:
- name: order-service
image: order-service:latest
ports:
- containerPort: 80
A Pod is one or more containers (most often one) sharing network and storage - it is the basic unit that Kubernetes manages (starts, restarts, scales).
Deployment - managing many replicas of a Pod
apiVersion: apps/v1
kind: Deployment
metadata:
name: order-service-deployment
spec:
replicas: 3 # 3 INDEPENDENT instances of OrderService, for high availability and throughput
selector:
matchLabels:
app: order-service
template:
metadata:
labels:
app: order-service
spec:
containers:
- name: order-service
image: order-service:latest
ports:
- containerPort: 80
env:
- name: ConnectionStrings__DefaultConnection
valueFrom:
secretKeyRef:
name: order-db-secret
key: connectionString
A Deployment guarantees that the specified number of replicas is always running (replicas: 3) - if one Pod crashes (a crash, a machine failure), Kubernetes automatically creates a new one to restore the desired number. This is exactly self-healing, which Docker Compose doesn't provide.
A link to Kafka Consumer Groups - if these 3 replicas of OrderService listen to the same Kafka topic as one consumer group, Kubernetes automatically keeps 3 instances running to share the partitions among themselves, exactly as we discussed earlier.
Service - a stable access point to changing Pods
apiVersion: v1
kind: Service
metadata:
name: order-service
spec:
selector:
app: order-service # routes traffic to ALL the Pods with this label
ports:
- port: 80
targetPort: 80
type: ClusterIP # accessible only INSIDE the cluster (other services can connect)
The problem a Service solves: Pods are ephemeral - they can be created/deleted/restarted, each with its own, changing IP address. A Service provides a stable name (order-service) through which other services (e.g. InventoryService calling OrderService over HTTP) can always connect, regardless of which specific Pods are currently running "underneath" - Kubernetes automatically routes traffic (load balancing) to healthy Pods.
// InventoryService calling OrderService - uses the STABLE Service name, not a specific Pod IP
builder.Services.AddHttpClient<IOrderServiceClient, OrderServiceClient>(client =>
client.BaseAddress = new Uri("http://order-service")); // "order-service" to nazwa K8s Service
ConfigMap and Secret - configuration and sensitive data
apiVersion: v1
kind: Secret
metadata:
name: order-db-secret
type: Opaque
stringData:
connectionString: "Server=order-db;Database=OrderDb;User=sa;Password=Pass123!"
Secret (for sensitive data, like connection strings with passwords) and ConfigMap (for ordinary configuration) let you separate configuration from the container image - the same order-service:latest image can be deployed in different environments (dev/staging/prod) with different configuration, without rebuilding the image.
Summary - how it all fits together into one architecture
Docker packages a single microservice into a portable, isolated image. Docker Compose runs the whole architecture locally (many services + databases + Kafka) for development, while naturally forcing you to think of it as a set of independent services with their own databases (hence the need for a Saga/Outbox for coordination). Kubernetes carries this over to production at large scale - a
Deploymentguarantees the desired number of always-running replicas (self-healing, automatic scaling), and aServicegives a stable entry point to a dynamically changing set of Pods, through which other services communicate (e.g. over HTTP or by publishing/consuming events from Kafka, as we discussed earlier).