In our Spark application we have log4j.properties under our src/main/resources folder.
But this is not being taken by Spark while submitting the application.
We found that we should use either one of the below approaches to override the Spark's default log4j.properties in SPARK_HOME/conf folder.
spark-submit ... --conf "spark.executor.extraJavaOptions=-Dlog4j.configuration=file:/tmp/log4j.properties" ...
or
spark-submit ... --files /tmp/log4j.properties#log4j.properties ...
Is there a way to make the log4j.properties inside src/main/resources folder of the JAR itself to override the Spark's default file?
Thanks in Advance!!!