Question 1

A data engineer needs to write a Streaming DataFrame as Parquet files. Given the code:Given the code:

Options :

A : format("parquet") .option("location", "path/to/destination/dir")

B : CopyEdit .option("format", "parquet") .option("destination", "path/to/destination/dir")

C : .option("format", "parquet") .option("location", "path/to/destination/dir")

D : format("parquet") .option("path", "path/to/destination/dir")

Answer: D

Question 2

A Spark application suffers from too many small tasks due to excessive partitioning. How can this be fixed without a full shuffle?

Options :

A : Use the distinct() transformation to combine similar partitions

B : Use the coalesce() transformation with a lower number of partitions

C : Use the sortBy() transformation to reorganize the data

D : Use the repartition() transformation with a lower number of partitions

Answer: B

Question 3

Which Spark configuration controls the number of tasks that can run in parallel on the executor? Options:

Options :

A : spark.executor.cores

B : spark.task.maxFailures

C : spark.driver.cores

D : spark.executor.memory

Answer: A

Question 4

A data engineer observes that an upstream streaming source sends duplicate records, where duplicates share the same key and have at most a 30-minute difference inevent_timestamp. The engineer adds: dropDuplicatesWithinWatermark("event_timestamp", "30 minutes") What is the result?

Options :

A : It is not able to handle deduplication in this scenario

B : It removes duplicates that arrive within the 30-minute window specified by the watermark

C : It removes all duplicates regardless of when they arrive

D : It accepts watermarks in seconds and the code results in an error

Answer: B

Question 5

A Spark engineer must select an appropriate deployment mode for the Spark jobs. What is the benefit of using cluster mode in Apache Spark™?

Options :

A : In cluster mode, resources are allocated from a resource manager on the cluster, enabling better performance and scalability for large jobs

B : In cluster mode, the driver is responsible for executing all tasks locally without distributing them across the worker nodes.

C : In cluster mode, the driver runs on the client machine, which can limit the application's ability to handle large datasets efficiently.

D : In cluster mode, the driver program runs on one of the worker nodes, allowing the application to fully utilize the distributed resources of the cluster.

Answer: D

Higher Test Marks with Free Online Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Exam Practice

Buying Options: