Question 1

If we want to create a constant integer 1 as a new column ‘new_column’ in a dataframe df, which code block we should select ?

Options :

A : df.withColumn(“new_column”, 1)

B : df.withColumn(new_column, lit(1))

C : df.withColumn”'new_column”, lit(“1”))

D : df.withColumnRenamed('new_column', lit(1))

E : df.withColumn(“new_column”, lit(1))

Answer: A

Question 2

When the property spark.sql.optimizer.dynamicPartitionPruning.enabled is set to true, what optimization happens in spark ?

Options :

A : It allows you to dynamically switch join strategies.

B : Spark dynamically handles skew in sort-merge join by splitting (and replicating if needed) skewed partitions with this property enabled.

C : You to read only as much data as you need.

D : It allows you to dynamically select physical plans based on cost.

Answer: C

Question 3

Given that the number of partitions of dataframe df is 4 and we want to write a parquet file in a given path. Choose the correct number of files after a successful write operation.

Options :

A : 4

B : 1

C : 8

Answer: A

Question 4

Which of the following code blocks returns DataFrame transactionsDf sorted in descending order by column predError, showing missing values last?

Options :

A : transactionsDf.sort(asc_nulls_last("predError"))

B : transactionsDf.orderBy("predError").desc_nulls_last()

C : transactionsDf.sort("predError", ascending=False)

D : transactionsDf.desc_nulls_last("predError")

E : transactionsDf.orderBy("predError").asc_nulls_last()

Answer: C

Question 5

The code block displayed below contains an error. The code block should read the csv file located at path

data/transactions.csv into DataFrame transactionsDf, using the first row as column header

and casting the columns in the most appropriate type. Find the error.

First 3 rows of transactions.csv:

1.transactionId;storeId;productId;name

2.1;23;12;green grass

3.2;35;31;yellow sun

4.3;23;12;green grass

Code block:

transactionsDf = spark.read.load("data/transactions.csv", sep=";", format="csv", header=True)

Options :

A : The DataFrameReader is not accessed correctly.

B : The transaction is evaluated lazily, so no file will be read.

C : Spark is unable to understand the file type.

D : The code block is unable to capture all columns.

E : The resulting DataFrame will not have the appropriate schema.

Answer: A

Higher Test Marks with Free Online Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 Exam Practice

Buying Options: