Remove characters from every line of a file in shell

user_4853331 0
Viewed 587

I have a shell script which is reading the value of a variable line by line. I need to remove certain characters from every line.

What I have - $sample_variable - 

Data 0 start; 1 ABCD0;2 EFGH0;3 IJKL0;4 MNOP0;5 QRST0;6 end;

What I want - 

start
ABCD0
EFGH0
IJKL0
MNOP0
QRST0
end

Code that I have written - 

IFS=$';' 
for j in $sample_variable
do  
    j=$j | cut -d ' ' -f3-
    echo $j
    j=${j// /''}
    echo $j
    echo $j >> output.txt
done

I am writing the output in a txt file.But, the file is getting written as output.txt -

start
1ABCD0
2EFGH0
3IJKL0
4MNOP0
5QRST0
6end

How do I remove the numbers occuring at the beginning also?

ksh shell file
4 Answers

If you were trying to remove all the digits I would say you could try using the tr tool, like so:

IFS=$';' for j in $sample_variable do j=$j | cut -d ' ' -f3- echo $j j=${j// /''} echo $j | tr -d [:digit:] echo $j | tr -d [:digit:] >> output.txt done

However if you want to only remove the initial digits you'll need a more versatile tool like sed, it would look like:

IFS=$';' for j in $sample_variable do j=$j | cut -d ' ' -f3- echo $j j=${j// /''} echo $j | sed -e 's/^[0-9]\?//' echo $j | sed -e 's/^[0-9]\?//' >> output.txt done

user_65512130

You may find sed handy as well, 

sample_data="Data 0 start 1 ABCD0;2 EFGH0;3 IJKL0;4 MNOP0;5 QRST0;6 end"
sed -e 's/^[^0]*0\ //' -e 's/;/\n/g' -e 's/\(^[^0-9][^0-9]*\ \)/\1\n/' -e 's/\ //g'

Example Use/Output

$ sample_data="Data 0 start 1 ABCD0;2 EFGH0;3 IJKL0;4 MNOP0;5 QRST0;6 end"
> echo "$sample_data" |
> sed -e 's/^[^0]*0\ //' -e 's/;/\n/g' -e 's/\(^[^0-9][^0-9]*\ \)/\1\n/' -e 's/\ //g'
start
1ABCD0
2EFGH0
3IJKL0
4MNOP0
5QRST0
6end
user_34221020

The following code solves the problem - 

i=0
IFS=$';' 
for j in $sample_variable
do 
    j=${j// /''}
    j=$(echo "$j" | tr -d [$i] | tr -d ["\n"])
    echo "$j" >> output.txt
    i=$((i+1))
done
}

So, I have taken a variable 'i' that will keep iterating through the loop. Using that variable I am able to remove the digits occuring only in the beginning of every line.

user_48533310

You need to remove everything in front of the space too:

j=${j//* /''}

The '' are unneded.

j=${j//* /}

The * matches any number of any characters. So if j has two spaces, then it will remove everything in front and including the second space. Depending on you needs it may be better to use:

j=${j##* }

or

j=${j#* }

See shell parameter expansion in bash.

It may be better to read the variable in an array and then process it:

sample_variable='1 ABCD;2 EFGH;3 IJKL;4 MNOP;5 QRST;'
IFS=';' read -r -a arr <<<"$sample_variable"

Then you can split the variable ignoring the anything in front the first space:

for j in "${arr[@]}"; do
     j=${j//* /}
     echo "$j"
done

But I would just do the following, but it will leave one empty trailing newline, so it may be needed to sed '$d' - remove last line:

<<<"$sample_variable" tr ';' '\n' | cut -d' ' -f2- | sed '$d'

Notes:

  • j=$j | cut -d ' ' -f3- doesn't do what you think it does. It executes j=$j which sets the variable to itself. Then it executes cut -d ' ' -f3- with no input, because the assignment doesn't print any output.
  • Remember to always quote your variable expansions.
user_90727530
Related