Learn with Databricks : Databricks-Certified-Data-Engineer-Professional training material for 100% pass

A junior data engineer is working to implement logic for a Lakehouse table named silver_device_recordings. The source data contains 100 unique fields in a highly nested JSON structure.
The silver_device_recordings table will be used downstream to power several production monitoring dashboards and a production model. At present, 45 of the 100 fields are being used in at least one of these applications.
The data engineer is trying to determine the best approach for dealing with schema declaration given the highly-nested structure of the data and the numerous fields.
Which of the following accurately presents information about Delta Lake and Databricks that may impact their decision-making process?

• A. Because Databricks will infer schema using types that allow all observed data to be processed, setting types manually provides greater assurance of data quality enforcement.
• B. Schema inference and evolution on .Databricks ensure that inferred types will always accurately match the data types used by downstream systems.
• C. Because Delta Lake uses Parquet for data storage, data types can be easily evolved by just modifying file footer information in place.
• D. Human labor in writing code is the largest cost associated with data engineering workloads; as such, automating table declaration logic should be a priority in all migration workloads.
• E. The Tungsten encoding used by Databricks is optimized for storing string data; newly-added native support for querying JSON strings means that string types are always most efficient.

Explanation: Only visible for Exam4Labs members. You can sign-up / login (it's free).

A user wants to use DLT expectations to validate that a derived table report contains all records from the source, included in the table validation_copy.
The user attempts and fails to accomplish this by adding an expectation to the report table definition.

Which approach would allow using DLT expectations to validate all expected records are present in this table?

• A. Define a function that performs a left outer join on validation_copy and report and report, and check against the result in a DLT expectation for the report table
• B. Define a SQL UDF that performs a left outer join on two tables, and check if this returns null values for report key values in a DLT expectation for the report table.
• C. Define a temporary table that perform a left outer join on validation_copy and report, and define an expectation that no report key values are null
• D. Define a view that performs a left outer join on validation_copy and report, and reference this view in DLT expectations for the report table

Explanation: Only visible for Exam4Labs members. You can sign-up / login (it's free).

The security team is exploring whether or not the Databricks secrets module can be leveraged for connecting to an external database.
After testing the code with all Python variables being defined with strings, they upload the password to the secrets module and configure the correct permissions for the currently active user. They then modify their code to the following (leaving all other variables unchanged).

Which statement describes what will happen when the above code is executed?

• A. An interactive input box will appear in the notebook; if the right password is provided, the connection will succeed and the encoded password will be saved to DBFS.
• B. The connection to the external table will fail; the string "redacted" will be printed.
• C. An interactive input box will appear in the notebook; if the right password is provided, the connection will succeed and the password will be printed in plain text.
• D. The connection to the external table will succeed; the string value of password will be printed in plain text.
• E. The connection to the external table will succeed; the string "redacted" will be printed.

Explanation: Only visible for Exam4Labs members. You can sign-up / login (it's free).

A data engineer wants to create a cluster using the Databricks CLI for a big ETL pipeline. The cluster should have five workers, one driver of type i3.xlarge, and should use the '14.3.x- scala2.12' runtime. Which command should the data engineer use?

• A. databricks compute add 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster-name Data Engineer_cluster
• B. databricks compute create 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster- name Data Engineer_cluster
• C. databricks clusters create 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster- name DataEngineer_cluster
• D. databricks clusters add 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster-name Data Engineer_cluster

Explanation: Only visible for Exam4Labs members. You can sign-up / login (it's free).

While reviewing a query's execution in the Databricks Query Profiler, a data engineer observes that the Top Operators panel shows a Sort operator with high Time Spent and Memory Peak metrics. The Spark UI also reports frequent data spilling. How should the data engineer address this issue?

• A. Convert the sort operation to a filter operation.
• B. Repartition the DataFrame to a single partition before sorting.
• C. Switch to a broadcast join to reduce memory usage.
• D. Increase the number of shuffle partitions to better distribute data.

Explanation: Only visible for Exam4Labs members. You can sign-up / login (it's free).