I have a hadoop cluster, on which there are two data nodes and one name node. In addition, I have a worker node from which I run hadoop commands and which points to the hadoop data nodes for execution.
When I run an application, it runs and stores the application log in /app-logs/hdfs/logs. The hadoop nodes that run the application are installed with version 3.1.1.3.
When I then try to run yarn logs -applicationId application_1613400075634_0173 on the worker node, the yarn command does not return a value, and instead prints these lines:
File /app-logs/hdfs/logs-tfile/application_1613400075634_0173 does not exist.
File /app-logs/hdfs/bucket-logs-tfile/0173/application_1613400075634_0173 does not exist.
Can not find any log file matching the pattern: [ALL] for the application: application_1613400075634_0173
Can not find the logs for the application: application_1613400075634_0173 with the appOwner: hdfs
The worker node is installed with hadoop binaries version 3.3.
What do I need to change in the configuration in order for the yarn command to look at /app-logs/hdfs/logs instead of /app-logs/hdfs/logs-tfile or bucket-logs-tfile? The file /app-logs/hdfs/logs/application_1613400075634_0173 does exist, but yarn looks at different directories.
Running the yarn command on the hadoop nodes themselves does work. It's only on the worker node that it doesn't.