We upgraded CDH version from 5.x to 6.3.3 and also hive from 1.x to 2.1.1. When we run hive map join, there is no impact to the result, but we got the following error message
Caused by: main ERROR RandomAccessFileManager [java.io.IOException: Could not create directory /var/log/hive/operation_logs/a2e15f81-0e7e-4d56-8b8b-7a4768ced8ae]
When I run hive map join, the table size is less than 10KB.
Our HiveServer2 is installed in node1, the /var/log/hive/operation_logs/ folder exist in node1, but the operation_logs folder is empty, seems like it didn't write any log file there. Since the error message mentioned folder creation issue, I tried to create the operation_logs folder manually in edge node(node2). And when I run the hive map join, there is no error message, and I can see the log created in node2.
So I just wonder why HiveServer2 write the log in the edge node instead of it's own node? And does anyone know how can we fix it?