Databricks Certified-Data-Engineer-Professional : Databricks Certified Data Engineer Professional

A data engineer is configuring a Lakeflow Declarative Pipeline to process CDC (Change Data Capture) data from a source. The source events sometimes arrive out of order, and multiple updates may occur with the same update_timestamp but with different update_sequence_id.
What should the data engineer do to ensure events are sequenced correctly?

• A. Use dropDuplicates() to remove out-of-order and duplicate records in LDP.
• B. Set track_history_column_list to [event_timestamp, event_id] in AUTO CDC APIs.
• C. Use SEQUENCE BY STRUCT(event_timestamp, update_sequence_id) in AUTO CDC APIs.
• D. Use a window function to sort update_sequence_id within the same partition, i.e., update_timestamp in the LDP pipeline.

The data engineering team maintains the following code:

Assuming that this code produces logically correct results and the data in the source table has been de-duplicated and validated, which statement describes what will occur when this code is executed?

• A. The gold_customer_lifetime_sales_summary table will be overwritten by aggregated values calculated from all records in the silver_customer_sales table as a batch job.
• B. The silver_customer_sales table will be overwritten by aggregated values calculated from all records in the gold_customer_lifetime_sales_summary table as a batch job.
• C. A batch job will update the gold_customer_lifetime_sales_summary table, replacing only those rows that have different values than the current version of the table, using customer_id as the primary key.
• D. An incremental job will detect if new rows have been written to the silver_customer_sales table; if new rows are detected, all aggregates will be recalculated and used to overwrite the gold_customer_lifetime_sales_summary table.
• E. An incremental job will leverage running information in the state store to update aggregate values in the gold_customer_lifetime_sales_summary table.

A data engineer wants to refactor the following DLT code, which includes multiple table definitions with very similar code.

In an attempt to programmatically create these tables using a parameterized table definition, the data engineer writes the following code.

The pipeline runs an update with this refactored code, but generates a different DAG showing incorrect configuration values for these tables.
How can the data engineer fix this?

• A. Convert the list of configuration values to a dictionary of table settings, using different input the for loop.
• B. Load the configuration values for these tables from a separate file, located at a path provided by a pipeline parameter.
• C. Wrap the loop inside another table definition, using generalized names and properties to replace with those from the inner table
• D. Convert the list of configuration values to a dictionary of table settings, using table names as keys.

A platform team lead is responsible for automating the individual teams attribution towards SQL Warehouse usage. The requirement is to identify the SQL warehouse usage at the individual user's level and generate a daily report to be shared with an executive team that includes leaders from all business units. How should the platform lead generate an automated report that can be shared daily?

• A. Use the system tables to capture the audit and billing usage data and create a dashboard with daily refresh schedules and shared with the executive team.
• B. Let the users run the SQL query and then directly report the usage to the executives. The ownership of the SQL warehouse usage will be with the individual teams.
• C. Use the system tables to capture the audit and billing usage data and share the queries with the executive team. This enables the executives to execute the query and see the latest results any time.
• D. Restrict users from running any SQL query unless they provide all the query details so that the attribution can be calculated and shared with the executive team.

Predictive Optimization is an automated Databricks service enabled by default for Unity Catalog Managed tables. It helps maintain Delta tables by continuously optimizing them to ensure optimal performance and costs. Which two operations does Predictive Optimization run to maintain the Delta tables? (Choose two.)

• A. BUCKETING
• B. OPTIMIZE
• C. ANALYZE
• D. PARTITION BY
• E. COMPACT