Import a text file content that contains statements that are intended to be in a zeppelin notebook, in multiple cells

Viewed 8

Accompanying a book I'm learning from, are files like this one, obviously depicting a Zeppelin notebook content:

spark
// COMMAND ----------

// in Scala
val myRange = spark.range(1000).toDF("number")


// COMMAND ----------

// in Scala
val divisBy2 = myRange.where("number % 2 = 0")


// COMMAND ----------

divisBy2.count()


// COMMAND ----------

// in Scala
val flightData2015 = spark
  .read
  .option("inferSchema", "true")
  .option("header", "true")
  .csv("/data/flight-data/csv/2015-summary.csv")


// COMMAND ----------

flightData2015.take(3)


// COMMAND ----------

flightData2015.sort("count").explain()


// COMMAND ----------

spark.conf.set("spark.sql.shuffle.partitions", "5")


// COMMAND ----------

flightData2015.sort("count").take(2)


// COMMAND ----------

flightData2015.createOrReplaceTempView("flight_data_2015")


// COMMAND ----------

// in Scala
val sqlWay = spark.sql("""
SELECT DEST_COUNTRY_NAME, count(1)
FROM flight_data_2015
GROUP BY DEST_COUNTRY_NAME
""")

val dataFrameWay = flightData2015
  .groupBy('DEST_COUNTRY_NAME)
  .count()

sqlWay.explain
dataFrameWay.explain


// COMMAND ----------

spark.sql("SELECT max(count) from flight_data_2015").take(1)


// COMMAND ----------

// in Scala
import org.apache.spark.sql.functions.max

flightData2015.select(max("count")).take(1)


// COMMAND ----------

// in Scala
val maxSql = spark.sql("""
SELECT DEST_COUNTRY_NAME, sum(count) as destination_total
FROM flight_data_2015
GROUP BY DEST_COUNTRY_NAME
ORDER BY sum(count) DESC
LIMIT 5
""")

maxSql.show()


// COMMAND ----------

// in Scala
import org.apache.spark.sql.functions.desc

flightData2015
  .groupBy("DEST_COUNTRY_NAME")
  .sum("count")
  .withColumnRenamed("sum(count)", "destination_total")
  .sort(desc("destination_total"))
  .limit(5)
  .show()


// COMMAND ----------

// in Scala
flightData2015
  .groupBy("DEST_COUNTRY_NAME")
  .sum("count")
  .withColumnRenamed("sum(count)", "destination_total")
  .sort(desc("destination_total"))
  .limit(5)
  .explain()


// COMMAND ----------

But how may I load such a text file as a notebook, if it is intended for,
a way where each // COMMAND ------- would be splat into a Zeppelin cell?

From what I can see in Zeppelin, I can only:

  • either import a notebook file, but then its of .json kind or .ipynb.
  • or paste the whole content of my file into a single cell, then split it manually.

How can I handle this kind of file the most conveniently?

0 Answers
Related