Passing xargs values as argument to subsequent YARN command

user_4447530 0
Viewed 935

Need help with passing arguments from xargs to subsequent Hadoop command.

I have results like below from this command. 

yarn application -list -appStates RUNNING | grep <user> | awk '{print $1}'

Results of above command as below.

17/09/19 12:24:36 INFO client.ConfiguredRMFailoverProxyProvider:
Failing over to rm21 Total Application-Id
application_1505017974932_14847 
application_1505017974932_14815
application_1505017974932_14810 
application_1505017974932_14784
application_1505017974932_14796 
application_1505017974932_14778

I need to be able to pass above output one by one to below command.

yarn application -movetoqueue <application_ID of above output> -queue myqueue

Could you please help here in finding the right command to achieve the same?

Similar setup is working for me in below command but not in the above as above command has n

a) 

yarn application -list -appStates RUNNING | grep <user> | awk '{print $1}' | xargs -n 1 yarn application -kill

This is another way..

b.) for app in yarn application -list -appStates RUNNING | grep <user> | awk '{print $1}’; do yarn application -kill "$app"; done

Appreciate your help on this.

hadoop-yarn xargs hadoop grep linux
1 Answers

xargs reads the STDIN stream data and converts each line into space separated arguments to the command. Some reasons why xargs might not work in your case:

  1. Using xargs, the argument to this command "yarn application -movetoqueue 'application_ID' -queue myqueue" is passed at the end of it.
  2. Not all the applications/tools accept multiple space separated arguments.
  3. The result of your command is a string containing special characters and it should be quoted.

So, I would suggest the following:

for appid in `yarn application -list -appStates RUNNING | grep user|awk '{print $1}'` ; do 
    yarn application -movetoqueue "${appid}" -queue myqueue;
done
user_32971350
Related