Description
Databricks cluster autoscale configuration needs min_workers and max_workers values suited to the workload. Omitting either leaves the intended scaling range unclear and may cause the configuration to be rejected.
Potential impact
Incomplete capacity settings can prevent cluster deployment or make it harder to manage required performance and spending limits.
Remediation
Specify both worker counts and choose a range supported by the service that fits the workload and budget.
Examples
The revised example sets a range of 1–50 workers. These values are illustrative, not recommendations for every workload. Runtime and node-type data sources are omitted.
Before
hcl
resource "databricks_cluster" "example" {
cluster_name = "Shared Autoscaling"
spark_version = data.databricks_spark_version.latest.id
node_type_id = data.databricks_node_type.smallest.id
autotermination_minutes = 20
autoscale {
min_workers = 1
}
}
After
hcl
resource "databricks_cluster" "example" {
cluster_name = "Shared Autoscaling"
spark_version = data.databricks_spark_version.latest.id
node_type_id = data.databricks_node_type.smallest.id
autotermination_minutes = 20
autoscale {
min_workers = 1
max_workers = 50
}
}