follow the step to install the hadoop:
STEPS TO INSTALL HADOOP IN DISTRIBUTED MODE
STEP 1:-
#cd /home/masternode
# mkdirhadoopinfra
# chown -R root:roothadoopinfra
# chmod -R 777 hadoopinfra
STEP 1:- Open the /root/.bashrc file and add the following lines in the file
export HADOOP_HOME=/usr/lib/hadoop_package/hadoop-2.5.6
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
STEP 2:- Now apply all the changes into the current running system.
source ~/.bashrc
Now Starting Hadoop Configuration
STEP 3:-You can find all the Hadoop configuration files in the location “$HADOOP_HOME/etc/hadoop”. You need to make required changes in those configuration files according to your Hadoop infrastructure.
$ cd $HADOOP_HOME/etc/hadoop
In order to develop Hadoop programs using java, you have to reset the java environment variables in hadoop-env.sh file by replacing JAVA_HOME value with the location of java in your system.
gedit hadoop-env.sh and add java path to this file
export JAVA_HOME=/usr/lib/hadoop_package/jdk1.8.0_161
STEP 4:- core-site.xml
The core-site.xml file contains information such as the port number used for Hadoop instance, memory allocated for the file system, memory limit for storing the data, and the size of Read/Write buffers.
Open the core-site.xml and add the following properties in between the and tags.
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value> // hdfs://192.168.64.131:9000 <master ip>
</property>
</configuration>
STEP 5:- hdfs-site.xml
The hdfs-site.xml file contains information such as the value of replication data, the namenode path, and the datanode path of your local file systems. It means the place where you want to store the Hadoop infra.
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>file:///home/masternode/hadoopinfra/hdfs/namenode </value>
</property>
<property>
<name>dfs.data.dir</name>
<value>file:///home/masternode/hadoopinfra/hdfs/datanode </value >
</property>
</configuration>
STEP 6:- yarn-site.xml
This file is used to configure yarn into Hadoop. Open the yarn-site.xml file and add the following properties in between the , tags in this file.
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
STEP 7:- mapred-site.xml
This file is used to specify which MapReduce framework we are using. By default, Hadoop contains a template of yarn-site.xml. First, you need to copy the file from mapred-site,xml.template to mapred-site.xml file using the following command.
$ cpmapred-site.xml.template mapred-site.xml
Open mapred-site.xml file and add the following properties in between the , tags in this file.
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
STEP 8:- Verifying Hadoop Installation
The following steps are used to verify the Hadoop installation.
start-all.sh
jps