Spark 3 5 Updates
Knowledge base for Spark 3.1→3.5 updates not covered by the Learning Spark 2nd Ed book. Use for: (A) Pandas UDF and Arrow APIs (exam Sec 7 obj 2 'Create and invoke Pandas UDF'); (B) new built-in functions added in 3.3-3.5 (exam Sec 3); (C) AQE evolution post-3.0 — default-on, new sub-configs, SMJ→SHJ conversion (exam Sec 4). NOT covered here: Pandas API on Spark (see skill pandas-on-spark), Spark…
Pandas On Spark
Knowledge base for Pandas API on Spark (pyspark.pandas) — Apache Spark 3.5. Use when preparing for Sec 7 objective 'Explain advantages of using Pandas API on Spark', migrating pandas code to scale on Spark, or configuring pyspark.pandas options (default index type, ops_on_diff_frames, checkpointing). Sources: spark.apache.org 3.5.7 docs + Databricks blog (Koalas merger). NOT covered here: Pandas…
Apache Spark
Knowledge base from \"Learning Spark, 2nd Edition\" by Damji, Wenig, Das & Lee. Use when working with Apache Spark / PySpark, preparing for the Databricks Certified Associate Developer for Apache Spark exam, or applying the authors' frameworks for Spark architecture, DataFrame API, Spark SQL, Structured Streaming, tuning, and Datasets.
Spark Connect
Knowledge base for Spark Connect (Apache Spark 3.4–3.5). Use when preparing for Sec 6 of the Databricks Certified Associate Developer for Apache Spark exam, configuring a Spark Connect client/server, or migrating PySpark code to the decoupled client-server architecture. Sources: spark.apache.org 3.5.7 docs + Databricks engineering blog.
Spark Cert 3 5
Meta-skill orchestrator for preparing the Databricks Certified Associate Developer for Apache Spark 3.5 exam (Python). Coordinates the sub-skills apache-spark, spark-connect, pandas-on-spark, and spark-3.5-updates. Activates on requests like "let's study Spark", "review <topic>", "quiz on <topic>", "mock exam", "what do I still need to study", "weak spots", "progress check".