Split sentences into separate lines

user_14383499 0
Viewed 871

I'm trying to split sentences in a file into separate lines using a shell script.

Now I would like to split the strings by !, ? or . . The output should be like this :

The file that I want to read from my_text.txt and contains

you want to learn shell script? First, you want to learn Linux command! then. you can learn shell script.

Now I would like to split the strings by " ! " or "? " or "." The output should be like this :

you want to learn shell script                 
First, you want to learn Linux command             
then           
you can learn shell script

I used this script :

while read p
do
   echo $p | tr "? ! ." "\n " 
done < my_text.txt

But the output is:

you want to learn shell script

First, you want to learn Linux command then you can learn shell script

Can somebody help?

tr awk bash
8 Answers

This could be done in a single awk using its global substitution option as follows, written and tested with shown samples only in GNU awk. Simply globally substituting ?,!,. with new line(by default ORS(output record separator) value as new line).

awk '{gsub(/\?|!|\./,ORS)} 1' Input_file
user_58665800 
$ sed 's/[!?.]/\n/g' file
you want to learn shell script
 First, you want to learn Linux command
 then
 you can learn shell script
user_17450010

Awk is ideal for this:

awk -F '[?.!]' '{ for (i=1;i<=NF;i++) { print $i } }' file

Set the field delimiters to ? or . or ! and then loop through each field and print the entry.

user_78404400

You can call 3 tr commands to split for ? ! and .

cat test_string.txt | tr "!" "\n" | tr "?" "\n" | tr "." "\n"
user_147336690

That's not how you use tr. Both arguments to it should be of the same length, otherwise the second one is extended to length of the first by repeating its last character*—that is, in this case, a space—to make one-by-one transliteration possible. In other words, given ? ! . and \n  as arguments, tr will replace ? with a line feed, and !, , and . with a space. What you're looking for is I guess:

$ tr '?!.' '\n' <file
you want to learn shell script
 First, you want to learn Linux command
 then
 you can learn shell script

Or, more portably:

tr '?!.' '[\n*]' <file

*This is what GNU tr does, POSIX leaves the behavior unspecified when arguments aren't of the same length.

user_102486780

In gnu-awk we can get it with gensub() function:

awk '{print gensub(/([.?!]\s*)/, "\n", "g", $0)}' file
you want to learn shell script
First, you want to learn Linux command
then
you can learn shell script
user_142594650

why limit yourself to new line \n being the RS ? Maybe something like this :

  • \056 is the period. \040 is space. i'll add the + in case there have been legacy practices of typing 2 spaces after each sentence and u wanna standardize it.
  • I presume question mark \044 is more frequent than exclamation \041. Only reason why i'm using all octal is that all those are ones that can wreck havor on a terminal when just a slight chance of didn't quoting and escaping properly.
  • Unlike FS or RS, OFS/ORS are constant strings (are they?), so typing in the characters will be safe.
  • the periods are taken care of by RS. No need special processing. So if the row contains neither ? nor ! , just print it as is, and move on (it'll handle the ". \n" )

.

mawk 'BEGIN { RS = "[\056][\040]+" ; ORS = ". \n"; 
              FS = "[\044][\040]+";  OFS = "? \n"; }
      ($0 !~ /[\041\044]/) { 
                              print; next; } 
             /[\041]/      { 
                              gsub("[\041][\040]+", "\041 \n"); }  
      ( NF==1 ) || ( $1=$1 )'

As fast as mawk is, a gsub ( ) or $1=$1 still costs money, so skip the costly parts unless it actually has a ? or ! mark.

Last line is the fun trick, done *outside the brace brackets. You've already done the ! the line before, so if no ? found (aka NF is 1), then that one evaluates true, which awk will short circuit and not execute part 2 , simply print.

But if you've found any ? marks, the assignment of $1=$1 will re-arrange them in new order, and because it's an assignment operation not equality-compare, it always come back successful if the assignment itself didn't fail, which will also serve as it self's always-true flag to print towards the end.

user_146721140

Awk's record separator variable RS should do the trick.

echo 'you want to learn shell script? First, you want to learn Linux command! then. you can learn shell script.' |
awk 'BEGIN{RS="[?.!] "}1'
user_93512550
Related