Question 1

When using CLI or REST API to get results from jobs with multiple tasks, which statement correctly describes the response structure?

Options :

A : Each run of a job will have a unique job_id; all tasks within this job will have a unique job_id

B : Each run of a job will have a unique job_id; all tasks within this job will have a unique task_id

C : Each run of a job will have a unique orchestration_id; all tasks within this job will have a unique run_id

D : Each run of a job will have a unique run_id; all tasks within this job will have a unique task_id

E : Each run of a job will have a unique run_id; all tasks within this job will also have a unique run_id

Answer: D

Question 2

A data engineer is analyzing a Spark job via the Spark UI. They have the following summary metrics for 27 completed tasks in a particular stage

Which conclusion can the data engineer draw from the above statistics ?

Options :

A : All task are operating over partitions with even amounts of data

B : All task are operating over empty or near empty partitions

C : All tasks are operating over partitions with larger skewed amounts of data.

D : Number of tasks are operating over partitions with larger skewed amounts of data.

E : Number of tasks are operating over empty or near empty partitions

Answer: D

Question 3

The data engineering team has a singleplex bronze table called ‘orders_raw’ where new orders data is appended every night. They created a new Silver table called ‘orders_cleaned’ in order to provide a more refined view of the orders data.

The team wants to create a batch processing pipeline to process all new records inserted in the orders_raw table and propagate them to the orders_cleaned table.

Which solution minimizes the compute costs to propagate this batch of data?

Options :

A : Use time travel capabilities in Delta Lake to compare the latest version of orders_raw with one version prior, then write the difference to the orders_cleaned table.

B : Use Spark Structured Streaming to process the new records from orders_raw in batch mode using the trigger availableNow option

C : Use Spark Structured Streaming-s foreachBatch logic to process the new records from orders_raw using trigger(processingTime=”24 hours")

D : Use batch overwrite logic to reprocess all records in orders_raw and overwrite the orders_cleaned table

E : Use insert-only merge into the orders_cleaned table using orders_raw data based on a composite key

Answer: B

Question 4

All records from an Apache Kafka producer are being ingested into a single Delta Lake table with the following schema:

key BINARY, value BINARY, topic STRING, partition LONG, offset LONG, timestamp LONG

There are 5 unique topics being ingested. Only the "registration" topic contains Personal Identifiable Information (PII). The company wishes to restrict access to PII. The company also wishes to only retain records containing PII in this table for 14 days after initial ingestion. However, for non-PII information, it would like to retain these records indefinitely.

Which of the following solutions meets the requirements?

Options :

A :

All data should be deleted biweekly; Delta Lake-s time travel functionality should be leveraged to maintain a history of non-PII information.

B :

Data should be partitioned by the registration field, allowing ACLs and delete statements to be set for the PII directory

C :

Because the value field is stored as binary data, this information is not considered PII and no special precautions should be taken

D :

Separate object storage containers should be specified based on the partition field, allowing isolation at the storage level

E :

Data should be partitioned by the topic field, allowing ACLs and delete statements to leverage partition boundaries.

Answer: E

Question 5

A junior data engineer seeks to leverage Delta Lake's Change Data Feed functionality to create a Type 1 table representing all of the values that have ever been valid for all rows in a bronze table created with the property delta.enableChangeDataFeed = true. They plan to execute the following code as a daily job:

Databricks-Certified-Professional-Data-Engineer-page61-image24

Which statement describes the execution and results of running the above query multiple times?

Options :

A : Each time the job is executed, newly updated records will be merged into the target table, overwriting previous values with the same primary keys.

B : Each time the job is executed, the entire available history of inserted or updated records will be appended to the target table, resulting in many duplicate entries.

C : Each time the job is executed, the target table will be overwritten using the entire history of inserted or updated records, giving the desired result.

D : Each time the job is executed, the differences between the original and current versions are calculated; this may result in duplicate entries for some records.

E : Each time the job is executed, only those records that have been inserted or updated since the last execution will be appended to the target table, giving the desired result.

Answer: B

Higher Test Marks with Free Online Databricks-Certified-Professional-Data-Engineer Exam Practice

Buying Options: