Master Node Affinity & Pod Placement
Hands-On Lab Guide
This comprehensive lab covers Kubernetes pod scheduling mechanisms, from basic manual scheduling to advanced node affinity rules. You'll learn how the Kubernetes scheduler makes decisions and how to control pod placement across your cluster.
Static pods are managed directly by the kubelet daemon on a specific node, without the API server observing them. Master components (kube-apiserver, kube-scheduler, kube-controller-manager, etcd) run as static pods.
/etc/kubernetes/manifests/kubectl get pods -n kube-system
kube-scheduler-master-node).
The simplest form of scheduling is using the nodeName field to assign a pod to a specific node.
apiVersion: v1
kind: Pod
metadata:
name: busybox-manual
spec:
nodeName: node-22 # Hard-coded node assignment
containers:
- name: busybox
image: busybox
command: ["sleep", "3600"]
kubectl apply -f busybox-manual.yaml
kubectl get pods -o wide
First, add labels to your nodes to categorize them:
# Label nodes by color
kubectl label node node-20 color=blue
kubectl label node node-21 color=green
kubectl label node node-22 color=red
# Label nodes by size
kubectl label node node-20 size=small
kubectl label node node-21 size=small
kubectl label node node-22 size=medium
# Verify labels
kubectl get nodes --show-labels
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
spec:
replicas: 5
selector:
matchLabels:
app: backend
template:
metadata:
labels:
app: backend
spec:
nodeSelector:
size: small # Only schedule on nodes with size=small
containers:
- name: backend
image: nginx:latest
ports:
- containerPort: 80
kubectl apply -f backend-nodeselector.yaml
kubectl get pods -o wide
# Edit deployment to use non-existent label
kubectl edit deployment backend
# Change: size: small -> size: large
# Scale to trigger rescheduling
kubectl scale deployment backend --replicas=0
kubectl scale deployment backend --replicas=5
# Check pod status
kubectl get pods
kubectl describe pod <pod-name> to see: "0/3 nodes are available: 3 node(s) didn't match Pod's node affinity/selector"
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
spec:
replicas: 5
selector:
matchLabels:
app: backend
template:
metadata:
labels:
app: backend
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: size
operator: In
values:
- small
- medium
containers:
- name: backend
image: nginx:latest
In: Label value must be in the listNotIn: Label value must not be in the listExists: Label key must exist (any value)DoesNotExist: Label key must not existGt: Greater than (numeric values)Lt: Less than (numeric values)apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
spec:
replicas: 5
selector:
matchLabels:
app: backend
template:
metadata:
labels:
app: backend
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 50
preference:
matchExpressions:
- key: size
operator: In
values:
- small
- weight: 30
preference:
matchExpressions:
- key: size
operator: In
values:
- medium
containers:
- name: backend
image: nginx:latest
┌─────────┐ ┌───────────┐ ┌─────────┐ ┌─────────────┐
│ Pending │ --> │ Scheduled │ --> │ Running │ --> │ Terminated │
└─────────┘ └───────────┘ └─────────┘ └─────────────┘
│ │
└─────────────────────────────────────────────────────┘
(Image Pull / Errors)
# Create pod with required affinity for color=red
kubectl apply -f pod-with-affinity.yaml
# Verify pod is running on node-22 (color=red)
kubectl get pods -o wide
# Change node label
kubectl label node node-22 color=blue --overwrite
# Pod continues running! (IgnoredDuringExecution)
kubectl get pods -o wide
# Delete and recreate pod
kubectl delete pod <pod-name>
# New pod will be Pending (no node with color=red)
kubectl get pods
nodeSelectorTerms:
- matchExpressions: # Item 1
- key: color
operator: In
values:
- red
- matchExpressions: # Item 2
- key: size
operator: In
values:
- small
values:
- small
- medium
- large
kubectl label node node-20 env=dev
kubectl label node node-21 env=uat
kubectl label node node-22 env=prod
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: env
operator: In
values:
- uat
kubectl label node node-22 performance=high
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: performance
operator: In
values:
- high
# List all pods with status
kubectl get pods
# Detailed pod information
kubectl describe pod <pod-name>
# Check node labels
kubectl get nodes --show-labels
# Check node resources
kubectl describe node <node-name>
# Modify affinity rules
kubectl edit deployment <deployment-name>
# Or update labels
kubectl label node <node-name> <key>=<value>
env: dev, uat, prodtier: frontend, backend, databasezone: availability zonesdisktype: ssd, hddsize: small, medium, largeAlways test with:
# Label operations
kubectl label node <node> <key>=<value>
kubectl label node <node> <key>- # Remove label
kubectl get nodes --show-labels
# Pod operations
kubectl get pods -o wide
kubectl describe pod <pod-name>
kubectl delete pod <pod-name>
# Deployment operations
kubectl scale deployment <name> --replicas=<count>
kubectl edit deployment <name>
kubectl rollout restart deployment <name>
# Debugging
kubectl get events --sort-by='.lastTimestamp'
kubectl logs <pod-name>
nodeNamenodeSelector