Why DeltaTable.forPath throws "[path] is not a Delta table"?

user_16651530 0
Viewed 242

I'm trying to read a delta lake table which I loaded previously using Spark and I'm using IntelliJ IDE.

val dt = DeltaTable.forPath(spark, "/some/path/")

Now when I'm trying to read the table again I'm getting below error, it was working fine but suddenly it throws error like these, what might be the reason for this?

Note:

Checked the files in the DeltaLake path - it looks good. Colleague was able to read the same DeltaLake file.

Exception in thread "main" org.apache.spark.sql.AnalysisException: `/some/path/` is not a Delta table.
    at org.apache.spark.sql.delta.DeltaErrors$.notADeltaTableException(DeltaErrors.scala:260)
    at io.delta.tables.DeltaTable$.forPath(DeltaTable.scala:593)
    at com.datalake.az.core.DeltaLake$.delayedEndpoint$com$walmart$sustainability$datalake$az$core$DeltaLake$1(DeltaLake.scala:66)
    at com.datalake.az.core.DeltaLake$delayedInit$body.apply(DeltaLake.scala:18)
    at scala.Function0.apply$mcV$sp(Function0.scala:39)
    at scala.Function0.apply$mcV$sp$(Function0.scala:39)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
    at scala.App.$anonfun$main$1$adapted(App.scala:80)
    at scala.collection.immutable.List.foreach(List.scala:431)
    at scala.App.main(App.scala:80)
    at scala.App.main$(App.scala:78)
    at com.datalake.az.core.DeltaLake$.main(DeltaLake.scala:18)
    at com.datalake.az.core.DeltaLake.main(DeltaLake.scala)
delta-lake
1 Answers

AnalysisException: /some/path/ is not a Delta table.

AnalysisException is thrown when the given path has no transaction log under _delta_log directory.

There could be other issues but that's the first check.

BTW By the stacktrace I figured you may not be using the latest and greatest Delta Lake 2.0.0. Please upgrade as soon as possible as it brings tons of improvements you don't want to miss.

user_13053440
Related