How to combine all the words of a sentence extracted with a regex?

Viewed 152

I would like to combine with a linux command, if possible, all the words that start with a capital letter, excluding the one at the beginning of the line. The goal is to create edges between these words. For example:

My friend John met Beatrice and Lucio.

The result I would like to have should be:

  • John, Beatrice
  • John, Lucio
  • Beatrice, Lucio

I managed to get all the words that start with a capital letter, excluding the word at the beginning of the line through a regex. The regex is:

*cat gov.json | grep -oP "\b([A-Z][a-z']*)(\s[A-Z][a-z']*)*\b | ^(\s*.*?\s).*" > nodes.csv*

The nodes managed to enter them individually in column, ie:

  • John
  • Beatrice
  • Lucio

The goal now is to create the possible combinations between names that start with a capital letter and put them into a file. Any suggestions?

5 Answers

If order of the pairs in the output doesn't matter:

$ cat tst.awk
BEGIN { FS="[^[:alpha:]]+"; OFS=", " }
{
    for (i=2; i<=NF; i++) {
        if ($i ~ /^[[:upper:]]/) {
            words[$i]
        }
    }
}
END {
    for (word1 in words) {
        for (word2 in words) {
            if (word1 != word2) {
                print word1, word2
            }
        }
        delete words[word1]
    }
}

$ awk -f tst.awk file
Beatrice, Lucio
Beatrice, John
Lucio, John

If the order does matter then:

$ cat tst.awk
BEGIN { FS="[^[:alpha:]]"; OFS=", " }
{
    for (i=2; i<=NF; i++) {
        if ($i ~ /^[[:upper:]]/) {
            if ( !seen[$i]++ ) {
                words[++numWords] = $i
            }
        }
    }
}
END {
    for (word1nr=1; word1nr<=numWords; word1nr++) {
        word1 = words[word1nr]
        for (word2nr=word1nr+1; word2nr<=numWords; word2nr++) {
            word2 = words[word2nr]
            print word1, word2
        }
    }
}

$ awk -f tst.awk file
John, Beatrice
John, Lucio
Beatrice, Lucio

In the above, file contains the original input, e.g. My friend John met Beatrice and Lucio.

Here is another awk script doing the task, building the output while reading input.

script.awk allowing duplicate names.

BEGIN {FPAT =  " [[:upper:]][[:alpha:]]+"}
{
    for (i = 1; i <= NF; i++ ) {
        for (name in namesArr) {
            namePairsArr[pairsCount++] = namesArr[name] $i;
        }
        namesArr[namesCount++] = $i;
    }   
}
END {for (i = 0; i < pairsCount; i++) print namePairsArr[i];}

If duplicate names not allowed, script.awk is:

BEGIN {FPAT =  " [[:upper:]][[:alpha:]]+"}
{
    for (i = 1; i <= NF; i++ ) {
        if (nameSeenArr[$i]) continue;
        nameSeenArr[$i] = 1;
        for (name in namesArr) {
              namePairsArr[pairsCount++] = namesArr[name] $i;
        }
        namesArr[namesCount++] = $i;
    }
}
END {for (i = 0; i < pairsCount; i++) print namePairsArr[i];}**

run

awk -f script.awk gov.json > nodes.csv

sample input file:

My friend John met Beatrice and Lucio
My friend Johna met Beatricea and Lucioa

sample output:

 John Beatrice
 John Lucio
 Beatrice Lucio
 John Johna
 Beatrice Johna
 Lucio Johna
 John Beatricea
 Beatrice Beatricea
 Lucio Beatricea
 Johna Beatricea
 John Lucioa
 Beatrice Lucioa
 Lucio Lucioa
 Johna Lucioa
 Beatricea Lucioa

When you have the results in an array, you can loop with

names=( John Beatrice Lucio )
max=${#names[@]}

for ((i1=0; i1<max; i1++)); do
  for ((i2=i1+1; i2<max; i2++)); do
    echo "${names[$i1]}, ${names[$i2]}"
  done
done

When you don't have to much names (length arglist) you can use

set -- John Beatrice Lucio
for a; do
    shift
    for b; do
        printf "%s, %s\n" "$a" "$b"
    done
done

If the only problem would be the first word of our string, we would simply add a . in our expression:

.([A-Z][a-z']+)

or we would be adding an space as a left boundary:

(\s+)([A-Z][a-z']+)(\s+)?

Our desired word is in this capturing group:

([A-Z][a-z']+)

Demo

yet this would fail if we'd have such cases as:

My friend Alice O'Neal met Beatrice and Lucio

for which we can expand our desired output group.

RegEx

If this expression wasn't desired and you wish to modify it, please visit this link at regex101.com.

RegEx Circuit

jex.im visualizes regular expressions:

enter image description here

If you want all the possible pairs of names, let's say you have created names.txt containing:

    John
    Beatrice
    Lucio

I'd try this in bash:

  $ for n in `cat names.txt`
  > do for m in `cat names.txt`
  >   do if [ $m != $n ]; then
  >        echo $n, $m
  >      fi
  >   done
  > done
  John, Beatrice
  John, Lucio
  Beatrice, John
  Beatrice, Lucio
  Lucio, John
  Lucio, Beatrice
Related