
Databricks Certified Associate Developer for Apache Spark checks that you can write and tune Spark applications with the DataFrame API and Spark SQL in Python. It covers Spark architecture, reading and writing data, transformations, joins, UDFs, performance tuning, Structured Streaming, Spark Connect and the pandas API on Spark. Spark appears in most data engineering job posts, so this is a strong first certification for data engineering freshers. Databricks published the current version on 8 October 2026.
This guide follows the official Databricks exam page and exam guide: exam facts, the seven sections with weights, the PySpark you need, a 4-week plan and 15 original practice questions.
Quick facts (Databricks, November 2026)
| Item | Detail |
|---|---|
| Questions | 45 scored multiple-choice questions, plus up to 10 unscored (up to 55 in total) |
| Time | 90 minutes (time for unscored items is included) |
| Fee | US$200 plus applicable taxes |
| Delivery | Online proctored or test centre; no test aids |
| Language | English |
| Prerequisites | None; about six months of hands-on Spark recommended |
| Validity | 2 years; recertify by passing the current exam |
Read the official exam page and its exam guide before you book.
Exam sections and weights
| Section | Weight | Key objectives |
|---|---|---|
| 1. Apache Spark Architecture and Components | 20% | Driver, executors, cores and memory; DataFrames and Datasets; SparkSession; caching and storage levels; jobs, stages and tasks; partitions and shuffles; lazy evaluation; Spark modules |
| 2. Using Spark SQL | 20% | Reading and writing JDBC and files; querying files directly; save modes; partitionBy; saving persistent tables; temporary views |
| 3. Developing DataFrame/Dataset API Applications | 30% | Column and row operations; explode; deduplication; aggregates; dates; joins (including broadcast and cross) and unions; schemas; UDFs; broadcast variables and accumulators |
| 4. Troubleshooting and Tuning | 10% | Repartition vs coalesce; data skew; reducing shuffles; Adaptive Query Execution; driver and executor logs; out-of-memory errors |
| 5. Structured Streaming | 10% | Micro-batches, exactly-once and fault tolerance; output modes and sinks; windows and aggregations; deduplication with and without watermarks |
| 6. Using Spark Connect to Deploy Applications | 5% | Spark Connect features; client, cluster and local deployment modes |
| 7. Using Pandas API on Spark | 5% | Why use the pandas API on Spark; pandas UDFs |
PySpark you must be able to read
from pyspark.sql import functions as F
orders = (spark.read.format("csv")
.option("header", True).schema(order_schema)
.load("/data/orders/"))
clean = (orders
.dropDuplicates(["order_id"])
.filter(F.col("amount") > 0)
.withColumn("order_date", F.to_date(F.from_unixtime("ts")))
.withColumn("item", F.explode("items")))
daily = (clean.join(F.broadcast(stores), "store_id", "left")
.groupBy("order_date", "city")
.agg(F.sum("amount").alias("revenue"),
F.approx_count_distinct("customer_id").alias("customers")))
(daily.write.mode("overwrite")
.partitionBy("order_date")
.saveAsTable("sales.daily_revenue"))
clean.createOrReplaceTempView("orders_v")
spark.sql("SELECT city, COUNT(*) FROM orders_v GROUP BY city").show()
# Structured Streaming with a watermark and a window
events = spark.readStream.table("raw.events")
counts = (events
.withWatermark("event_time", "10 minutes")
.dropDuplicates(["event_id", "event_time"])
.groupBy(F.window("event_time", "5 minutes"), "page")
.count())
(counts.writeStream.outputMode("append")
.option("checkpointLocation", "/chk/page_counts")
.toTable("analytics.page_counts"))
Things candidates often mix up
- Transformations vs actions: select, filter and join are lazy; count, show, collect and write trigger a job.
- Narrow vs wide: filter and withColumn stay within a partition; groupBy, join and distinct cause a shuffle and a new stage.
- repartition vs coalesce: repartition does a full shuffle and can increase or decrease partitions; coalesce only reduces partitions and avoids a full shuffle.
- cache vs persist: cache() on a DataFrame uses the default storage level (MEMORY_AND_DISK); persist() lets you choose one.
- Save modes: append, overwrite, ignore and error/errorifexists (the default).
4-week study plan
| Week | Focus | Lab (Databricks Free Edition or local PySpark) |
|---|---|---|
| 1 | Section 1 and 2: architecture, execution hierarchy, lazy evaluation, reading and writing, save modes, temp views | Read CSV, JSON and Parquet; write a partitioned table; read the Spark UI for one job |
| 2 | Section 3: column functions, explode, dedup, dates, aggregates, joins, UDFs, broadcast variables and accumulators | Rebuild a 10-step pandas analysis in PySpark |
| 3 | Section 4 and 7: partitions, skew, AQE, caching, logs; pandas API on Spark and pandas UDFs | Make a skewed join slow, then fix it with broadcast or AQE |
| 4 | Section 5 and 6: streaming, watermarks, output modes, Spark Connect, deployment modes; review | Run a windowed streaming count with a checkpoint; book the exam |
15 practice questions with answers
These are original questions written for this guide, not exam questions.
1. Which process plans the job and sends tasks to executors?
The driver.
2. Put these in order from largest to smallest: task, job, stage.
Job, stage, task.
3. What triggers a new stage?
A shuffle caused by a wide transformation such as groupBy or join.
4. Is df.filter(...) executed immediately?
No. Transformations are lazy; nothing runs until an action.
5. Which method reduces 200 partitions to 10 without a full shuffle?
coalesce(10).
6. What is the default save mode when writing a DataFrame?
errorifexists (error).
7. How do you make a DataFrame queryable with spark.sql for the current session?
createOrReplaceTempView.
8. Which function turns an array column into one row per element?
explode.
9. Joining a 2 TB table with a 20 MB table: how do you avoid shuffling the big one?
Broadcast the small table (a broadcast join).
10. What are accumulators used for?
Adding up values (such as counters) from executors back to the driver.
11. Why are Python UDFs often slow, and what is faster?
Rows are serialised between the JVM and Python one at a time; built-in functions or vectorised pandas UDFs are faster.
12. Name one thing Adaptive Query Execution does at runtime.
Coalesces shuffle partitions, switches to a broadcast join, or splits skewed partitions.
13. What does a watermark do in Structured Streaming?
Sets how late data may arrive, so Spark can drop old state and finalise windows.
14. Which output mode writes only new result rows?
Append.
15. What does Spark Connect change?
Client applications talk to a remote Spark cluster through a thin client API, separating the client from the driver.
After this certification
- Continue with the Databricks Data Engineer Associate and then the Professional.
- Prepare for interviews with Databricks interview questions.
- Learn it hands-on in our Python course and Multi-Cloud Data, AI and DevOps course; see live data engineering jobs.
Cloud Soft Solutions in Ameerpet, Hyderabad teaches Python, Spark, Databricks and cloud data engineering with real-time projects. See the APEX program and all courses.
📞 Call +91 96660 19191 · WhatsApp · Flat No. 513, 5th Floor, Nilagiri Block, Aditya Enclave, Ameerpet, Hyderabad
Frequently Asked Questions
How many questions are on the Spark Developer Associate exam?
45 scored multiple-choice questions plus up to 10 unscored, in 90 minutes.
How much does it cost?
US$200 plus applicable taxes.
Which section has the most weight?
Developing Apache Spark DataFrame/Dataset API applications, at 30%.
Which language is used?
The exam uses Python (PySpark) and Spark SQL.
When was the current version published?
8 October 2026.
How long is it valid?
2 years.



