DatabricksLive · 108 questions · 12 notes

Databricks Certified Data Engineer Associate

Free practice exam, study notes with architecture diagrams and the official blueprint for Databricks Certified Data Engineer Associate.

Passing score
70%
Domains
7
Exam code
Data Engineer Associate

Official Databricks Certified Data Engineer Associate exam blueprint

Domains and weights from Databricks's exam guide.

DomainWeight
Databricks Intelligence Platform6%
Data Ingestion and Loading21%
Data Transformation and Modeling22%
Working with Lakeflow Jobs16%
Implementing CI/CD10%
Troubleshooting, Monitoring, and Optimization10%
Governance and Security15%

Free Databricks Certified Data Engineer Associate sample questions

The first questions of the free diagnostic. Pick your answer, then check it. Why every option is right or wrong, and the What-If twist, open in the workspace.

  1. Question 1 · Data Transformation and Modeling

    The daily customer append failed this morning with a schema mismatch error, because the incoming DataFrame has a new column, loyalty_tier, that the table does not have. The column must simply be added to the table without rewriting it, and existing rows must be kept as they are. The write:

    Python
    (df.withColumn("load_date", current_date())
       .write.format("delta")
       .mode("append")
       .saveAsTable("main.crm.customers"))

    What should the engineer change?

    1. AWrite with .option("mergeSchema", "true")
    2. BWrite with mode("overwrite") and .option("overwriteSchema", "true")
    3. CCast loyalty_tier to STRING before writing
    4. DDrop and recreate the table before every load
    Check the answer

    Answer: A. Delta schema enforcement rejects unknown columns; mergeSchema evolves the schema on write, and overwriteSchema replaces it.

    Official docs: Databricks Documentation: Update table schema
    Unlock Full 4-Option Autopsy & What-If Twist in Workspace
  2. Question 2 · Working with Lakeflow Jobs

    A Lakeflow Job starts this Structured Streaming query every hour on jobs compute. Each run must process all data that arrived since the last run and then stop, without leaving any cluster or other compute running between the hourly schedules:

    Python
    (spark.readStream.table("main.bronze.events")
       .writeStream
       .option("checkpointLocation", checkpoint_path)
       .trigger(<trigger>)
       .toTable("main.silver.events"))

    Which setting should replace <trigger>?

    1. ALeave the trigger unset so the default micro-batch mode runs
    2. BA processingTime trigger of a few seconds
    3. CavailableNow=True
    4. DAn hourly processingTime trigger with the cluster kept running
    Check the answer

    Answer: C. availableNow processes all available data in batches and then stops; the default and processingTime triggers keep running.

    Official docs: Databricks Documentation: Configure Structured Streaming trigger intervals
    Unlock Full 4-Option Autopsy & What-If Twist in Workspace
  3. Question 3 · Data Ingestion and Loading

    Millions of JSON files land continuously in an S3 path behind a Unity Catalog external location. A data engineer must load only the new files into a Delta table as they arrive, with minimal setup and without tracking file names by hand. What should they use?
    1. ACREATE TABLE AS SELECT from read_files every hour
    2. BA Lakeflow Connect managed connector
    3. CAuto Loader (the cloudFiles source) in a streaming read
    4. DA scheduled COPY INTO statement over the whole path
    Check the answer

    Answer: C. Auto Loader incrementally processes new files in cloud storage and is recommended for very large numbers of files.

    Official docs: Databricks Documentation: What is Auto Loader?
    Unlock Full 4-Option Autopsy & What-If Twist in Workspace
Answer all 10 free Databricks Certified Data Engineer Associate questions

Databricks Certified Data Engineer Associate study notes: key concepts

What each note covers. The full notes, with summaries, examples and memory hooks, open with a free account.

Architecture flow · Databricks platform: control plane, compute planes, Delta Lake and Unity Catalog

How the Databricks platform layers fit together
  1. 1Control plane in the Databricks account

    Workspace web application

    Job scheduler and backend services

    • leads to Classic clusters and SQL warehouses and Serverless jobs, pipelines and warehouses · Starts runs on

    Unity Catalog metastore

    • leads to Delta tables with _delta_log · Governs access to
  2. 2Compute planes

    Classic clusters and SQL warehouses

    • leads to Delta tables with _delta_log · Reads and writes

    Serverless jobs, pipelines and warehouses

    • leads to Unity Catalog volumes · Reads and writes
  3. 3Cloud object storage

    Delta tables with _delta_log

    Unity Catalog volumes

Drawn fromDatabricks Documentation: Databricks components overview

Databricks platform: control plane, compute planes, Delta Lake and Unity Catalog

  • The control plane hosts the web application and backend services in the Databricks account, while data is processed in classic or serverless compute planes.
  • Classic compute runs in the customer's cloud account, whereas serverless compute runs on capacity Databricks manages, starting quickly and removing cluster configuration work.
  • Every Delta table keeps a transaction log of JSON commits and Parquet checkpoints, which provides ACID writes, table history and time travel to earlier versions.
  • Unity Catalog organizes data as catalog.schema.object and governs tables, views, volumes, functions and models with one set of grants, lineage and audit logs.
  • All-purpose compute suits interactive notebooks, jobs and serverless compute suit automated runs, and SQL warehouses serve BI and SQL analytics at the best price.

Ingestion: Auto Loader, COPY INTO, read_files, Lakeflow Connect and volumes

  • Auto Loader discovers new files incrementally and records processed files in its checkpoint, so each file is ingested exactly once even across restarts.
  • With the default addNewColumns mode, a new column stops the stream once after updating the schema, so running it in a job with retries restarts it automatically.
  • COPY INTO skips files it has already loaded, which makes reruns safe, while the force option reloads every file and can create duplicates.
  • Lakeflow Connect managed connectors ingest SaaS applications, databases and file services into Unity Catalog without custom connector code to maintain.
  • External volumes govern files in an existing cloud path in place, while managed volumes store files in Unity Catalog managed storage.

Structured Streaming on Databricks: triggers, checkpoints, output modes and watermarks

  • A streaming query's offsets and state live in its checkpoint, so changing the checkpoint location starts a new query that reprocesses the source from the beginning.
  • availableNow processes everything available at start, possibly in several batches, and then stops, which suits jobs scheduled hourly or daily without idle compute.
  • The default trigger keeps starting micro-batches as data arrives, giving low latency but requiring compute that stays up all the time.
  • Append mode only emits rows that will not change, so a windowed aggregation needs withWatermark to finalize windows before they can be written.
  • Update mode emits rows that changed since the last trigger, which suits running counts on dashboards that accept revised numbers.

Delta Lake tables: MERGE, time travel, RESTORE, VACUUM, schema evolution and change data feed

  • Time travel reads an earlier table version for targeted recovery, while RESTORE TABLE moves the whole table back and undoes every later change.
  • VACUUM deletes unreferenced data files past the retention threshold, so older versions lose their files and time travel to them fails afterwards.
  • MERGE fails when multiple source rows match the same target row, so the source is deduplicated first, for example keeping the latest row per key.
  • mergeSchema lets an append add new columns while keeping existing rows, whereas overwriteSchema with overwrite replaces both the data and the schema.
  • Change data feed records inserts, updates and deletes with a change type, so downstream jobs process only changed rows instead of comparing snapshots.

Medallion architecture and Lakeflow pipelines: streaming tables, materialized views, expectations and AUTO CDC

  • Bronze keeps raw data for replay, silver cleans and conforms it, and gold publishes business aggregates, so each layer has one clear job.
  • Streaming tables process each appended row once, while materialized views keep precomputed results correct when source rows are later corrected.
  • An expectation without an action only warns, DROP ROW drops failing rows and logs counts, and FAIL UPDATE stops the update for manual review.
  • AUTO CDC orders out-of-order changes with SEQUENCE BY and stores them as SCD Type 1 for current values or SCD Type 2 for full history.
  • Lakeflow pipelines work out dependencies between datasets and refresh them in order, so no hand-written orchestration or merge code is needed.

Transforming with Spark SQL and PySpark: joins, deduplication, arrays, views and SQL functions

  • A broadcast join sends the small table to every executor, so the large table joins in place without the costly shuffle of both sides.
  • dropDuplicates keeps an arbitrary row per key, so keeping the latest record requires row_number over a window ordered by the update timestamp.
  • explode drops rows whose array is empty or NULL, while explode_outer keeps them once with NULL values for the element.
  • FILTER returns the array elements that match a lambda, EXISTS returns one Boolean, and AGGREGATE folds the array into a single value.
  • Temporary views are session-scoped, global temporary views are shared on one cluster through global_temp, and permanent views live in Unity Catalog.
Read the full Databricks Certified Data Engineer Associate study notes

Databricks Certified Data Engineer Associate FAQ

What is the passing score for Databricks Certified Data Engineer Associate?

The passing score for Databricks Certified Data Engineer Associate is 70%.

Which domains does Databricks Certified Data Engineer Associate cover?

The official exam guide lists 7 domains: Databricks Intelligence Platform (6%); Data Ingestion and Loading (21%); Data Transformation and Modeling (22%); Working with Lakeflow Jobs (16%); Implementing CI/CD (10%); Troubleshooting, Monitoring, and Optimization (10%); Governance and Security (15%).

Is there a free Databricks Certified Data Engineer Associate practice test?

Yes. Gyan AI's free diagnostic is 10 Databricks Certified Data Engineer Associate questions with no sign-up, each with an explanation for every option and a link to the official documentation. A free account unlocks the full question set and study notes.

Are these the real Databricks Certified Data Engineer Associate exam questions?

No. Real exam content is confidential under vendor exam agreements, and using leaked "dumps" can get a certification revoked. Gyan AI questions are original and written to the published skills outline.