Calling elocation geocode API returns empty in Spark

I have this odd problem when calling eLocations geocoding API via Spark where i will always get an empty body even on addresses I know will return a coordinate. I am developing a geocoding app using Spark (2.3.3) and scala. I am also using scalaj to call the REST API. So the line of code which calls the API is as such:

def getGeoCoderLocation(sc: SparkSession, req: String, url: String, proxy_host: String, proxy_port: String, response_format: String): scala.collection.Map[String, (String, String)] = {
    import sc.implicits._

    val httpresponse = Http(url).proxy(proxy_host, proxy_port.toInt).postForm.params(("xml_request", req), ("format", response_format)).asString

    println(httpresponse.body)
    println(httpresponse.contentType.getOrElse(""))
    println(httpresponse.headers)
    println(httpresponse)
    if(!httpresponse.contentType.getOrElse("").contains("text/html")) {
      val body = httpresponse.body
      val httpresponse_body = parseJSON(Option(body).getOrElse("[{\"x\":, \"y\":}]"))

      val location = for (it <- 0 until httpresponse_body.length) yield {
        (Option(httpresponse_body(it)(0).x).getOrElse("").toString, Option(httpresponse_body(it)(0).y).getOrElse("").toString, it)
      }

      val locDF = location.toDF(Seq("LONGITUDE", "LATITUDE", "row"): _*)//.withColumn("row", monotonically_increasing_id())
      locDF.show(20, false)
      locDF.rdd.map { r => (Option(r.get(2)).getOrElse("").toString, (Option(r.get(0)).getOrElse("").toString, Option(r.getString(1)).getOrElse("").toString)) }.collectAsMap()
    }
    else {
      val locDF = Seq(("","","-")).toDF(Seq("LONGITUDE", "LATITUDE", "row"): _*)//.withColumn("row", monotonically_increasing_id())
      locDF.show(20, false)
      locDF.rdd.map { r => (Option(r.get(2)).getOrElse("").toString, (Option(r.get(0)).getOrElse("").toString, Option(r.getString(1)).getOrElse("").toString)) }.collectAsMap()
    }
  }

Where

• url = http://elocation.oracle.com/elocation/lbs
• proxy_host = (ip of proxy)
• proxy_port = (port number)
• req = "<?xml version=\"1.0\" standalone=\"yes\"?>\n<geocode_request vendor=\"elocation\">\n\t(address_list)\n\t\t|<list of requests>|\n\t</address_list>\n</geocode_request>"
• response_format = JSON

So when I print the body it will always be [{}] (i.e. empty JSON Array) when I run my app in Spark. When I run the same request without a spark-submit i will get a proper Array of JSON objects (e.g. java -jar test.jar).

Is there a setting in Spark which blocks the app from receiving REST responses? We are using Cloudera 5.16.x

I have also tried setting the proxy information using --conf "spark.executor.extraJavaOptions=-Dhttp.proxyHost=(ip) -Dhttp.proxyPort=(port) -Dhttps.proxyHost=(ip) -Dhttps.proxyPort=(port)" but I will get:

Exception in thread "main" org.apache.hadoop.security.KerberosAuthException: Login failure for user: (principal) from keytab (keytab) javax.security.auth.login.LoginException: Cannot locate KDC

Please help as I don't know where to look to solve this as i have never encounter this before.