Hadoop: start-dfs.sh throwing syntax errors

user_3877418 0
Viewed 847

I am trying to start hadoop by running ./start-dfs.sh, but i am getting some syntax errors. Could anybody please help?

    Gurupads-MacBook-Air:sbin guru$ sudo ./start-dfs.sh
    Starting namenodes on [localhost]
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-functions.sh: line 398: syntax error near unexpected token `<'
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-functions.sh: line 398: `  done < <(for text in "${input[@]}"; do'
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 70: hadoop_deprecate_envvar: command not found
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 87: hadoop_bootstrap: command not found
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 104: hadoop_parse_args: command not found
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 105: shift: : numeric argument required
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 244: hadoop_need_reexec: command not found
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 252: hadoop_verify_user_perm: command not found
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/hdfs: line 213: hadoop_validate_classname: command not found
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/hdfs: line 214: hadoop_exit_with_usage: command not found
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 263: hadoop_add_client_opts: command not found
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 270: hadoop_subcommand_opts: command not found
    /Users/guru/homebrew/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 273: hadoop_generic_java_subcmd_handler: command not found
    Starting datanodes
    ERROR: Attempting to operate on hdfs datanode as root
    ERROR: but there is no HDFS_DATANODE_USER defined. Aborting operation.
    Starting secondary namenodes [Gurupads-MacBook-Air.local]
    ERROR: Attempting to operate on hdfs secondarynamenode as root
    ERROR: but there is no HDFS_SECONDARYNAMENODE_USER defined. Aborting operation.
    2018-09-18 21:51:24,380 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
hadoop
1 Answers

The error states that, you are running hdfs datanode as root user and there is no HDFS_DATANODE_USER is defined.

Solution:

Edit /Users/guru/homebrew/Cellar/hadoop/3.1.1/etc/hadoop/hadoop-env.sh file by adding below lines at the end of file:

export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root

If you are using Yarn then you can add below lines in same file:

export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root

Restart hadoop using start-dfs.sh script.

user_63235320
Related