Description
When StatefulSet replicas share a node, one node failure can affect several Pods. Pod anti-affinity or topology spread constraints can control placement, and selectors must match actual Pod labels.
Required rules enforce scheduling conditions, while preferred rules express priorities. Spreading across hosts alone does not protect against zone failure or establish the application’s replication quorum.
Potential impact
- A node failure can interrupt several replicas together.
- Strict placement rules can leave Pods Pending when too few suitable nodes exist.
Remediation
- Configure podAntiAffinity or topology spread constraints for workload failure and quorum requirements. Check actual Pod labels and topologyKey.
- Verify the node count, capacity and storage placement required by mandatory rules, and test actual placement. IgnoredDuringExecution rules do not automatically move already running Pods.
Examples
These existing ZooKeeper image excerpts compare placement rules only. Use a supported image for operation and separately provide the zk-hs headless Service. The after rule requires at least three distinct suitable nodes.
Before
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: zk
spec:
serviceName: zk-hs
replicas: 3
selector:
matchLabels:
app: zk
template:
metadata:
labels:
app: zk
spec:
containers:
- name: zookeeper
image: k8s.gcr.io/kubernetes-zookeeper:1.0-3.4.10
Three replicas are defined without a spreading rule in this excerpt. Check actual placement and other policies.
After
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: zk
spec:
serviceName: zk-hs
replicas: 3
selector:
matchLabels:
app: zk
template:
metadata:
labels:
app: zk
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app: zk
topologyKey: kubernetes.io/hostname
containers:
- name: zookeeper
image: k8s.gcr.io/kubernetes-zookeeper:1.0-3.4.10
The rule prevents app: zk Pods from being scheduled on the same host. Too few suitable nodes can prevent all replicas from running.