I have two machines I want to create a cluster on Hadoop/Yarn. Both IMAC and RAZER have the Ubuntu 20.04 with Hadoop installed on both. Both have SSH-keys that can communicate without password.
For the master, I set the hdfs-site.xml as following:
<configuration>
<property>
<name>dfs.namenode.name.dir</name>
<value>/usr/local/hadoop-3.2.1/data/nameNode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/usr/local/hadoop-3.2.1/data/dataNode</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
I set the core-site.xml as following:
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop-3.2.1/hdfs/tmp</value>
<description>A base for other temporary directories</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://192.168.1.191:8020</value>
</property>
</configuration>
I performed hdfs namenode -format and even performed a chmod 777 /usr/local/hadoop-3.2.1
When I performed the hstart (start-dfs.sh), I get the following error:
Starting namenodes on [192.168.1.191]
Starting datanodes
@192.168.1.121: ERROR: Cannot set priority of datanode process 134254
Starting secondary namenodes
I couldn't get the remote machine to show up on the live nodes on the dashboard. I've tried most forums and their suggestions, but I'm still having this problem, please help. Thanks,
Vinh