Review Databricks diagnostic log collection

Description

Databricks handles data processing, notebook execution and cluster changes. Missing activity records can make incident investigation and permission-abuse analysis difficult.

Azure diagnostic settings export selected categories supported by the workspace. Their coverage differs from the audit log system table, so verify which collection path provides the events you need.

Potential impact

Missing records can make it harder to identify who changed a cluster or executed a job.
Audit logs can contain sensitive information; excessive access to the destination can create a separate disclosure risk.

Remediation

Attach diagnostic settings to the workspace and select categories needed for investigations, such as authentication, file operations, clusters, notebooks and jobs. Check available categories and the Premium-tier requirement.
Configure an approved destination such as Log Analytics, a Storage Account or Event Hub. Verify incoming logs, access permissions and retention.
Use the appropriate additional collection path, such as the audit log system table, for required account-level events or events unavailable through diagnostic settings.

Examples

The examples compare a workspace with diagnostic settings for selected categories. Supply the referenced resource group and Log Analytics workspace separately.

Before

hcl

resource "azurerm_databricks_workspace" "example" {
  name                = "secure-databricks-ws"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  sku                 = "premium"
}

Only the workspace is defined. Check other settings or audit-log collection paths as well.

After

hcl

resource "azurerm_monitor_diagnostic_setting" "example" {
  name                       = "databricks-diagnostic-logs"
  target_resource_id         = azurerm_databricks_workspace.example.id
  log_analytics_workspace_id = azurerm_log_analytics_workspace.example.id

  enabled_log {
    category = "accounts"
  }

  enabled_log {
    category = "Filesystem"
  }

  enabled_log {
    category = "clusters"
  }

  enabled_log {
    category = "notebook"
  }

  enabled_log {
    category = "jobs"
  }
}

resource "azurerm_databricks_workspace" "example" {
  name                = "secure-databricks-ws"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  sku                 = "premium"
}

Five categories are sent to Log Analytics. This list does not cover every audit event; verify investigation coverage and actual receipt.