Permission denied at hdfs

user_891494 0
Viewed 125278

I am new to hadoop distributed file system, I have done complete installation of hadoop single node on my machine.but after that when i am going to upload data to hdfs it give an error message Permission Denied.

Message from terminal with command:

hduser@ubuntu:/usr/local/hadoop$ hadoop fs -put /usr/local/input-data/ /input
put: /usr/local/input-data (Permission denied)

hduser@ubuntu:/usr/local/hadoop$

After using sudo and adding hduser to sudouser:

hduser@ubuntu:/usr/local/hadoop$ sudo bin/hadoop fs -put /usr/local/input-data/ /inwe
put: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="":hduser:supergroup:rwxr-xr-x

hduser@ubuntu:/usr/local/hadoop$
hdfs hadoop shell permissions security
6 Answers

I had similar situation and here is my approach which is somewhat different:

 HADOOP_USER_NAME=hdfs hdfs dfs -put /root/MyHadoop/file1.txt /

What you actually do is you read local file in accordance to your local permissions but when placing file on HDFS you are authenticated like user hdfs. You can do this with other ID (beware of real auth schemes configuration but this is usually not a case).

Advantages:

  1. Permissions are kept on HDFS.
  2. You don't need sudo.
  3. You don't need actually appropriate local user 'hdfs' at all.
  4. You don't need to copy anything or change permissions because of previous points.
user_2046650

For Hadoop 3.x, if you try to create a file on HDFS when unauthenticated (e.g. user=dr.who) you will get this error.

It is not recommended for systems that need to be secure, however if you'd like to disable file permissions entirely in Hadoop 3 the hdfs-site.xml setting has changed to:

<property>
  <name>dfs.permissions.enabled</name>
  <value>false</value>
</property>

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml

user_13994910
Related