Why does single test fail with "Error XSDB6: Another instance of Derby may have already booted the database"?

user_688578 0
Viewed 2217

I use Spark 1.6.

We have a HDFS write method that wrote to HDFS using SqlContext. Now we needed to switch over to using HiveContext. When we did that existing unit tests do not run and give the error

Error XSDB6: Another instance of Derby may have already booted the database <local path>\metastore_db

This happens whether I run a single test via IntelliJ test runner or via maven on the command line.

As I understand the issue happens when multiple HiveContexts or multiple processes are trying to access the metastore_db. However I am running a single test and no other jobs on my local machine so I fail to understand where the multiple processes are coming from

apache-spark-1.6 hdfs apache-spark-sql apache-spark derby
3 Answers

Even I was getting the same error though I was running a test suite.

I could run the individual test file successfully but when I ran suite a few tests kept failing. There were many tests doing IO in local file system using SparkSession.

In that situation, use after method in every test file(in my case, it was missing in 1-2 files) to close this session.

after { 
 sparkSession.stop() 
}
user_59938540
Related