Description
When GKE node auto-repair is disabled, unhealthy nodes can require operator intervention and prolong an outage. The current Google provider enables management.auto_repair by default; omission alone does not mean it is disabled.
Auto-repair supports availability and operational stability. It does not resolve every failure immediately or replace data backups.
Potential impact
- Unhealthy nodes can persist and disrupt workloads.
- Manual recovery can increase response time.
- Node repair itself can also disrupt workloads.
Remediation
- Verify
management.auto_repair = trueand actual service state for Standard node pools. - Maintain failure alerts and manual response procedures, and prepare workloads for node replacement.
- Manage auto-upgrades for security patches separately.
Examples
These excerpts show Standard cluster and node-pool settings. Supply the project, networking, node service account and other operational settings separately.
Before
hcl
resource "google_container_cluster" "cluster" {
name = "my-gke-cluster"
location = "us-central1"
remove_default_node_pool = true
initial_node_count = 1
}
resource "google_container_node_pool" "node_pool" {
name = "my-node-pool"
location = "us-central1"
cluster = google_container_cluster.cluster.name
node_count = 1
management {
auto_repair = false
}
}
After
hcl
resource "google_container_cluster" "cluster" {
name = "my-gke-cluster"
location = "us-central1"
remove_default_node_pool = true
initial_node_count = 1
}
resource "google_container_node_pool" "node_pool" {
name = "my-node-pool"
location = "us-central1"
cluster = google_container_cluster.cluster.name
node_count = 1
management {
auto_repair = true
}
}
Explanation:
- Before: Auto-repair is explicitly disabled for the node pool.
- After:
auto_repair = trueenables automatic repair according to the service’s health-check criteria.