bash + how to capture word from a long output

user_10194070 0
Viewed 148

I have the following output from the following command

zookeeper-shell.sh 19.2.6.4  get /brokers/ids/1010

output is

Connecting to 19.2.6.4

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
{"listener_security_protocol_map":{"PLAINTEXT":"PLAINTEXT"},"endpoints":["PLAINTEXT://kafka1.lulu.com:6667"],"rack":"/default-rack","jmx_port":9997,"port":6667,"host":"kafka1.lulu.com","version":4,"timestamp":"1630507307906"}

The main target is to capture the machine name kafka1 – from above output

So I successes to do the following long command syntax

zookeeper-shell.sh 119.2.6.4  get /brokers/ids/1010 | sed s'/\/\// /g' | sed s'/:/ /g' | sed s'/,/ /g' | sed s'/"/ /g' | sed s'/\./ /g'| awk '{for (i=1;i<=NF;i++) print $i}' | grep -i kafka | sort | uniq

and the results is: ( as expected results )

kafka1

the case is that I feel bad with my approach , its too long and not so elegant

can we get suggestion ( with awk/sed/perl one liner ) , that are much better from my syntax?

language-agnostic regex
4 Answers

With your shown samples, attempts please try following awk code. Since I don't have zookeeper command with me, I had written this code and tested it as per your shown output only.

zookeeper-shell.sh 19.2.6.4  get /brokers/ids/1010 | 
awk '
/WatchedEvent state/{
  found=1
  next
}
found && match($0,/"PLAINTEXT:\/\/[^:]*/){
  print substr($0,RSTART+13,RLENGTH-13)
}
'

Explanation: Adding detailed explanation for above awk code.

awk '                                         ##Starting awk program from here.
/WatchedEvent state/{                         ##Checking condition if line contains WatchedEvent state
  found=1                                     ##Then set found to 1 here.
  next                                        ##next will skip all further statements from here.
}
found && match($0,/"PLAINTEXT:\/\/[^:]*/){    ##Checking condition if found is SET then match regex "PLAINTEXT:\/\/[^:]* in match function of awk.
  print substr($0,RSTART+13,RLENGTH-13)       ##Printing sub string of matched regex used in match function above.
}
'
user_58665800

The text you want to parse is JSON so use a JSON-aware tool like jq for the bulk of the work, e.g. using cat file since I don't have the command you use to produce output:

$ cat file | jq -Rr 'fromjson? | .endpoints[]'
PLAINTEXT://kafka1.lulu.com:6667

$ cat file | jq -Rr 'fromjson? | .endpoints[]' | awk -F'[/.]' '{print $3}'
kafka1
user_17450010

Using perl, you can do:

$zookeeper_command | perl -MJSON::PP=decode_json -wnE'/^\{"/ or next; $j = decode_json($_); ($s) = (split /\./, $j->{host})[0]; say $s'

detailing the command:

  • -MJSON::PP=decode_json => imports decode_json from JSON::PP module (it's a core module.
  • /^\{"/ or next; => skip lines doesn't look like a json string.
  • $j = decode_json($_); => store into $j a data structure fom json string.
  • ($s) = (split /\./, $j->{host})[0]; => split string kafka1.lulu.com and store in $s only first portion.

It can also be written in a shorter form (and less readable too):

$zookeeper_command | perl -MJSON::PP=decode_json -wnE'say decode_json($_)->{host}=~s/\..*$//r if/^\{"/'
user_134568080

You can use following script to filter out data of interest, what would allow you to avoid typing long command lines.

use strict;
use warnings;
use feature 'say';

use JSON;

my $data;

while( <> ) {
    next unless /^\{.*?\}$/;   # skip all but JSON string
    
    my $data = from_json($_);  # restore data structure
    my $host = (split('\.',$data->{host}))[0]; # extract info of interest
    
    say $host;                 # output it
}

Run as zookeeper-shell.sh 19.2.6.4 get /brokers/ids/1010 | script.pl.

Note: make script executable chmod +x script.pl and store it in your $HOME/bin directory which add in your variable $PATH.

user_123133090
Related