Online Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 Questions Video

Explanation

Result of correct code block:

+-------------------+

|attributes_exploded|

+-------------------+

| winter|

| cooling|

+-------------------+

To solve this question, you need to know about explode(). This operation helps you to split up arrays into single rows. If you did not have a chance to familiarize yourself with this method yet, find

more examples in the documentation (link below).

Note that explode() is a method made available through pyspark.sql.functions – it is not available as a method of a DataFrame or a Column, as written in some of the answer options.

More info: pyspark.sql.functions.explode — PySpark 3.1.2 documentation

Static notebook | Dynamic notebook: See test 2, QUESTION NO: 18 (Databricks import instructions)

Explanation

Correct code block:

itemsDf.withColumn("itemNameElements", split("itemName"," "))

Output of code block:

+------+----------------------------------+-------------------+------------------------------------------+

|itemId|itemName |supplier |itemNameElements |

+------+----------------------------------+-------------------+------------------------------------------+

|1 |Thick Coat for Walking in the Snow|Sports Company Inc.|[Thick, Coat, for, Walking, in, the, Snow]|

|2 |Elegant Outdoors Summer Dress |YetiX |[Elegant, Outdoors, Summer, Dress] |

|3 |Outdoors Backpack |Sports Company Inc.|[Outdoors, Backpack] |

+------+----------------------------------+-------------------+------------------------------------------+

The key to solving this QUESTION NO: is that the split method definitely needs a second argument here (also look at the link to the documentation below). Given the values in column itemName in

DataFrame itemsDf, this should be a space character " ". This is the character we need to split the words in the column.

More info: pyspark.sql.functions.split — PySpark 3.1.1 documentation

Static notebook | Dynamic notebook: See test 1, QUESTION NO: 46 (Databricks import instructions)

Explanation

Caching means storing a copy of a partition on an executor, so it can be accessed quicker by subsequent operations, instead of having to be recalculated. cache() is a lazily-evaluated method of the

DataFrame. Since count() is an action (while filter() is not), it triggers the caching process.

More info: pyspark.sql.DataFrame.cache — PySpark 3.1.2 documentation, Learning Spark, 2nd Edition, Chapter 7

Static notebook | Dynamic notebook: See test 2, QUESTION NO: 20 (Databricks import instructions)

Explanation

Correct code block:

transactionsDf.select((col("storeId").between(20, 30)) & (col("productId")==2))

Although this QUESTION NO: may make you think that it asks for a filter or where statement, it does not. It asks explicity to return a column with booleans – this should point you to the select

statement.

Another trick here is the rarely used between() method. It exists and resolves to ((storeId >= 20) AND (storeId <= 30)) in SQL. geq() and leq() do not exist.

Another riddle here is how to chain the two conditions. The only valid answer here is &. Operators like && or and are not valid. Other boolean operators that would be valid in Spark are | and.

Static notebook | Dynamic notebook: See test 1, QUESTION NO: 42 (Databricks import instructions)