Unable to connect/submit a job to a remote Yarn cluser

I'm trying to submit a spark job to a yarn cluster on an Hadoop HDP 2.6.3 environment(With kerberos) using a java application. I have the required core-site.xml,hdfs-site.xml,yarn-site.xml all under the resources folder.

Setting SparkSession in the following way:

SparkSession sparkSession = SparkSession
                .builder()
                .appName("Sample App")
                .master("yarn") //tried with "local" as well
                .getOrCreate();

I keep getting this following message: Client: Retrying connect to server: xxx. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)

...

Client: Retrying connect to server: xxx. Already tried 49 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)

Where the xxx is the address under the yarn.resourcemanager.address property in the yarn-site.xml file.

I'm using the same spark-sql&spark-yarn version that are used in the server:

    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_2.11</artifactId>
        <version>2.2.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-yarn_2.11</artifactId>
        <version>2.2.0</version>
    </dependency>

Running the jar using spark-submit from my local environment doesn't work. However if I copy the jar to the HDP server and submit the jar from there, it does work.

What am I missing?