awk: How to get the whole string from a lookup file

user_14590226 0
Viewed 46

This question is closely related to this question:

There are many tsv files containing classification elements like this:

doc1.tsv

<http://uri.gbv.de/terminology/bk/86.56> 
<http://uri.gbv.de/terminology/bk/58.28>

There is also a lookup table that additionally contains the classnames:

lookup.tsv

<http://uri.gbv.de/terminology/bk/44.38>        Pharmakologie
<http://uri.gbv.de/terminology/bk/44.43>        Medizinische Mikrobiologie
<http://uri.gbv.de/terminology/bk/58.28>        Pharmazeutische Technologie
<http://uri.gbv.de/terminology/bk/86.56>        Gesundheitsrecht. Lebensmittelrecht

Raman Sailopal already gave a very good hint on the solution which is the following:

awk 'FNR==NR{ urls[$1]=$2 } FNR!=NR { print $1"\t"urls[$1] }' lookup.tsv doc1.tsv >> result.tsv

Unfortunately, the command just gets the first string up to a blank, but as in "Gesundheitsrecht. Lebensmittelrecht" the whole text is required. What has to be added?

awk shell
3 Answers

You can save $0 instead of $2 in the loopup table, as it seems like you want to print the whole line

$ awk 'FNR==NR { urls[$1]=$0 } FNR!=NR { print urls[$1] }' lookup.tsv doc1.tsv
<http://uri.gbv.de/terminology/bk/86.56>        Gesundheitsrecht. Lebensmittelrecht
<http://uri.gbv.de/terminology/bk/58.28>        Pharmazeutische Technologie
user_8875390

1st solution: With yoru shown samples, could you please try following, written and tested in GNU awk.

awk '
FNR==NR{
  arr[$0]
  next
}
match($0,/<.*> +/){
  val=substr($0,RSTART,RLENGTH)
  sub(/ +$/,"",val)
}
(val in arr)
' doc1 lookup

2nd solution: To get only value apart from URL from doc2 Input_file try following, 1st solution will give complete line.

awk '
FNR==NR{
  arr[$0]
  next
}
match($0,/<.*> +/){
  val=substr($0,RSTART,RLENGTH)
  sub(/ +$/,"",val)
}
(val in arr){
  print substr($0,RSTART+RLENGTH)
}
' doc1 lookup
user_58665800

You might be able to use grep -f:

$ grep -Fwf doc1.tsv lookup.tsv
<http://uri.gbv.de/terminology/bk/58.28>        Pharmazeutische Technologie
<http://uri.gbv.de/terminology/bk/86.56>        Gesundheitsrecht. Lebensmittelrecht

Note that even though -w is used then the lines from doc1.tsv will be matched against the whole line in lookup.tsv and not the first column

user_8875390
Related