DP-750 Questions and Answers

The correct answer is A — Yes.

df.dropna(subset=[ ' order_amount ' ]) is the idiomatic PySpark way to remove rows where a specific column contains a null. It inspects only the columns listed in subset and drops any row where those columns are null. The resulting DataFrame contains only rows where order_amount is not null — exactly what the requirement asks for.

The subset parameter is important: without it, dropna() would drop rows where ANY column is null, which could incorrectly exclude rows that have nulls in other columns but a valid order_amount. By specifying subset=[ ' order_amount ' ], the filter is applied precisely and only to the column in question.

This method is semantically equivalent to df.filter(df.order_amount.isNotNull()) and to the SQL clause WHERE order_amount IS NOT NULL. Both are correct — dropna with a subset is arguably the more readable Pythonic approach.

Delta Sharing is designed to share governed data securely with recipients outside an Azure Databricks workspace. The external organization can query the shared data from its own compatible compute platform without receiving workspace access or requiring a Databricks SQL warehouse. Unity Catalog centrally controls which tables, views, or other objects are included in the share and which recipients can access them. Moving files to an SFTP server creates additional copies and requires custom transfer and security administration. Lakeflow Connect is intended for ingesting data into Databricks rather than sharing curated data externally. Granting workspace access or creating a SQL warehouse would require the recipient to use Databricks-managed resources. Delta Sharing therefore provides the required open access model, centralized governance, and minimal administrative effort.

An Azure Storage account provides durable, comparatively low-cost retention for diagnostic and audit logs that are accessed infrequently. A lifecycle or retention policy can preserve the logs for 90 days and then remove them automatically. This matches an investigation-only access pattern without paying the ingestion and indexing charges associated with Log Analytics. Azure Monitor metrics stores numerical monitoring measurements, not the complete audit-event records required here. Azure Event Hubs is a streaming transport intended to forward events to consumers and is not the final long-term retention destination. Log Analytics is appropriate when teams need frequent querying, dashboards, and alerting, but those capabilities introduce unnecessary cost for logs reviewed only during occasional investigations. Storage therefore best satisfies centralized retention and cost requirements.

Two actions are needed to reclaim storage while keeping tables queryable:

Set delta.deletedFileRetentionDuration and delta.logRetentionDuration to 90 days on each table. These properties define the retention floor — VACUUM will not touch anything newer than this threshold, so no data needed for time travel within 90 days can be accidentally removed.

Run VACUUM on each table. VACUUM is the Delta Lake command that physically removes data files and transaction log entries older than the retention duration. Importantly, VACUUM runs as a background operation — it uses Delta Lake ' s MVCC (multi-version concurrency control) to ensure that concurrent reads against the table continue uninterrupted while cleanup happens. Tables are fully available throughout.

OPTIMIZE compacts small files for query performance but doesn ' t delete anything. Manually deleting files outside the Delta protocol would corrupt the table.

The correct answer is A — INSERT INTO.

INSERT INTO appends all rows from the source to the target. Existing rows in table2 are untouched — the historical data stays intact — and the current year ' s rows from table1 are added. That is exactly what the requirement asks: ' preserve any existing data and minimise processing effort. '

Option B (CREATE TABLE AS SELECT) would fail if table2 already exists, or if used with CREATE OR REPLACE it wipes table2 entirely before writing — historical data gone. Option C (INSERT OVERWRITE) replaces the entire table content with only the table1 rows, also destroying table2 ' s existing data. Option D (CREATE OR REPLACE TABLE AS SELECT) explicitly drops and recreates the table, eliminating all historical records.

INSERT INTO is deliberately the simplest possible command here — one line, no special options, no risk of accidental data loss.

The databricks bundle init command initializes a new bundle project from a selected default or custom project template. It prompts for relevant project values and creates the local project structure and starter configuration files. It does not deploy resources; deployment is performed with databricks bundle deploy. It does not merely validate an existing configuration, because validation uses databricks bundle validate. Running deployed bundle resources requires databricks bundle run followed by the resource key. The distinction matters in an automated bundle lifecycle: initialization scaffolds the project, validation checks configuration, deployment creates or updates workspace resources, and run starts a configured job or pipeline. Therefore, option A accurately describes the initialization command’s function. Microsoft Learn

Production ingestion: Job compute

BI: Serverless SQL warehouse

Job compute is designed for automated production workloads executed through Lakeflow Jobs. Its lifecycle can be tied to the job run, providing workload isolation and avoiding the cost of maintaining an interactive all-purpose cluster continuously. It is therefore appropriate for scheduled ingestion and transformation processing. A serverless SQL warehouse is designed for BI and Databricks SQL workloads. It provides rapid startup, automatic infrastructure management, scaling, and optimized SQL-query execution for dashboards and reporting tools. All-purpose compute is intended primarily for interactive notebook development and exploration, while shared compute does not provide the same job-specific isolation or SQL-serving experience. Consequently, job compute should support production ingestion, and a serverless SQL warehouse should serve the BI workload.

The right ingestion tool depends on the source characteristics:

File-based telemetry and maintenance data → Auto Loader (cloudFiles). It monitors ADLS Gen2 for new arrivals, handles schema inference and evolution for the frequent schema drift, and requires minimal operational effort.

Real-time telemetry from Event Hubs → Spark Structured Streaming with the azure-eventhubs-spark connector. This provides exactly-once semantics and checkpoint-based recovery, satisfying ' resume processing after failures without reprocessing. '

Structured maintenance data from PostgreSQL → JDBC connector. Databricks supports direct JDBC reads from relational databases with pushdown predicates.

Daily CSV ERP extracts (50–100 GB) → COPY INTO or Auto Loader. Both support idempotent incremental batch loading into Delta tables with minimal code.

The correct answer is A. Photon is Azure Databricks ' native vectorized query engine, written in C++, designed to accelerate data ingestion and SQL-heavy workloads significantly over the standard Spark JVM path. Enabling it on a job compute cluster directly addresses Contoso ' s requirement for ' fast and consistent performance for BI workloads ' and ' production ingestion workloads that can scale automatically during telemetry spikes. '

Photon integrates transparently — no code changes are needed — and pairs well with autoscaling job clusters to handle the bursty 40,000-sensor telemetry load.

Option B contradicts the isolation requirement: Contoso explicitly needs production and development separated, not merged onto shared compute. Option C with a fixed large node gives peak capacity at all times, driving up costs even during quiet periods. Option D disabling autoscaling is the opposite of what ' s needed — telemetry spikes require elastic scaling, not a locked node count.