Databricks platform: control plane, compute planes, Delta Lake and Unity Catalog
- The control plane hosts the web application and backend services in the Databricks account, while data is processed in classic or serverless compute planes.
- Classic compute runs in the customer's cloud account, whereas serverless compute runs on capacity Databricks manages, starting quickly and removing cluster configuration work.
- Every Delta table keeps a transaction log of JSON commits and Parquet checkpoints, which provides ACID writes, table history and time travel to earlier versions.
- Unity Catalog organizes data as catalog.schema.object and governs tables, views, volumes, functions and models with one set of grants, lineage and audit logs.
- All-purpose compute suits interactive notebooks, jobs and serverless compute suit automated runs, and SQL warehouses serve BI and SQL analytics at the best price.