Review GKE node pool auto-repair settings

Combine node auto-repair with failure monitoring to reduce prolonged outages.

Description

When GKE node auto-repair is disabled, unhealthy nodes can require operator intervention and prolong an outage. The current Google provider enables management.auto_repair by default; omission alone does not mean it is disabled.

Auto-repair supports availability and operational stability. It does not resolve every failure immediately or replace data backups.

Potential impact

  • Unhealthy nodes can persist and disrupt workloads.
  • Manual recovery can increase response time.
  • Node repair itself can also disrupt workloads.

Remediation

  • Verify management.auto_repair = true and actual service state for Standard node pools.
  • Maintain failure alerts and manual response procedures, and prepare workloads for node replacement.
  • Manage auto-upgrades for security patches separately.

Examples

These excerpts show Standard cluster and node-pool settings. Supply the project, networking, node service account and other operational settings separately.

Before

hcl
resource "google_container_cluster" "cluster" {
  name                     = "my-gke-cluster"
  location                 = "us-central1"
  remove_default_node_pool = true
  initial_node_count       = 1
}

resource "google_container_node_pool" "node_pool" {
  name       = "my-node-pool"
  location   = "us-central1"
  cluster    = google_container_cluster.cluster.name
  node_count = 1

  management {
    auto_repair = false
  }
}

After

hcl
resource "google_container_cluster" "cluster" {
  name                     = "my-gke-cluster"
  location                 = "us-central1"
  remove_default_node_pool = true
  initial_node_count       = 1
}

resource "google_container_node_pool" "node_pool" {
  name       = "my-node-pool"
  location   = "us-central1"
  cluster    = google_container_cluster.cluster.name
  node_count = 1

  management {
    auto_repair = true
  }
}

Explanation:

  • Before: Auto-repair is explicitly disabled for the node pool.
  • After: auto_repair = true enables automatic repair according to the service’s health-check criteria.

References