('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/simba/spark/lib/64/libsparkodbc_sb64.so' : file not found (0) (SQLDriverConnect)")

user_3235848 0
Viewed 1049

I was attempting to get the Databricks ODBC driver working on Golang. I followed the directions on https://github.com/alexbrainman/odbc in order to setup the Go library and the system library.

Instead of setting up the MSSQL ODBC driver, I installed Databricks' ODBC driver and configured the system to use it. I re-used a the mssql_test.go file to create a testbed to test the connection to Databricks, using a DSN/connection string I built using the guide from their website.

However, when attempting to connect to the database, I get the error:

('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/simba/spark/lib/64/libsparkodbc_sb64.so' : file not found (0) (SQLDriverConnect)")

No matter how I change the connection string, LD_LIBRARY paths, or anything else, I cannot seem to get things to work. It repeatedly gives this error, even though if I do an os.Stat() on this file in Go, I can see it clearly exists in the execution environment.

I'm running Go/the driver inside the a Debian linux container user docker, on my M1 Macbook.

arm64 databricks odbc
2 Answers

In this case, the reason that the file cannot be found is that the driver has not been compiled for ARM. This post on Github illustrates the issue.

We can confirm by running file on the library. It yields...

$> file /opt/simba/spark/lib/64/libsparkodbc_sb64.so
/opt/simba/spark/lib/64/libsparkodbc_sb64.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped

You can see it is compiled for x86-64, but the platform is actually ARM. No good!

There are a few different options to solve this problem:

  1. if you have the source of the driver, recompile the driver to target ARM.
  2. if you're building on docker, you might be able to rebuild your docker container to use the x86 platform instead. You can do this with the --platform linux/amd64 build flag to either your Dockerfile (e.g. FROM --platform=linux/amd64 golang:1.17 or to your build command.
  3. build/test/deploy your code in an x86 environment.

In the case of Databricks, they are aware of the issue, and an ARM-based driver is forthcoming.

user_32358480

If you have similar situation, check this:

ldd /opt/simba/spark/lib/64/libsparkodbc_sb64.so

It will tell you whether there are some depending files missing.

user_99355770
Related