How to print just one field of a structured file in shell script?

Viewed 47

I'm trying to write a shell script in which I need to show as output the cars and their caractheristics this way:

Car: Chevrolet Chevelle Malibu
MPG: 18.0
Cylinders: 8
...

The file I have is like this:

Car;MPG;Cylinders;Displacement;Horsepower;Weight;Acceleration;Model;Origin

    Chevrolet Chevelle Malibu;18.0;8;307.0;130.0;3504.;12.0;70;US
    Buick Skylark 320;15.0;8;350.0;165.0;3693.;11.5;70;US
    Plymouth Satellite;18.0;8;318.0;150.0;3436.;11.0;70;US
    AMC Rebel SST;16.0;8;304.0;150.0;3433.;12.0;70;US
    Ford Torino;17.0;8;302.0;140.0;3449.;10.5;70;US
    Ford Galaxie 500;15.0;8;429.0;198.0;4341.;10.0;70;US
    Chevrolet Impala;14.0;8;454.0;220.0;4354.;9.0;70;US

I have tried doing this

#! /bin/bash
i=1
line=`wc -l < cars2.csv`
while [ $i -le $line ]
do
    name=`head -$i cars2.csv | tail -1 | cut -d: -f1`
    mpg=`head -$i cars2.csv | tail -1 | cut -d: -f2`
    echo "Name: $name"
    echo "MPG: $mpg "

let i=i+1
done

I've just done this but it doesn't work the way I want, for each echo it prints the whole line like this:

Name: Chevrolet Chevelle Malibu;18.0;8;307.0;130.0;3504.;12.0;70;US

MPG: Chevrolet Chevelle Malibu;18.0;8;307.0;130.0;3504.;12.0;70;US
3 Answers

This could be done relatively easily with awk:

 awk -F";" 'NR==1{for(i=1;i<=NF;++i){header[i]=$i}}NR>1{for(i=1;i<=NF;++i){print header[i]": "$i}}' yourfile.txt

awk processes files line-by-line.

This awk script is doing the following steps:

  1. Tell awk that a semicolon delimits each field in a line -F";"
  2. If this is the first line of the file then loop through each field and capture the value to an associative array where the key is the field number and the value is the field value: NR==1{for(i=1;i<=NF;++i){header[i]=$i}}
  3. If we are past line 1 then loop through each field and print out the header value from the array captured in step 2 as well as the value of the field for the current line: NR>1{for(i=1;i<=NF;++i){print header[i]": "$i}}

Car: Chevrolet Chevelle Malibu
MPG: 18.0
Cylinders: 8
Displacement: 307.0
Horsepower: 130.0
Weight: 3504.
Acceleration: 12.0
Model: 70
Origin: US
Car: Buick Skylark 320
MPG: 15.0
Cylinders: 8
Displacement: 350.0
Horsepower: 165.0
Weight: 3693.
Acceleration: 11.5
...

While I feel awk is probably most appropriate since it's installed by default, if you need to gain more control over the processing of the data, you can consider python. Using pandas that would look something like:

import pandas as pd

#import data
df = pd.read_csv("yourfile.txt", sep=";", header=0)

#sort by car column
df = df.sort_values(by = "Car", ignore_index = True)

#stack/unpivot everything
df = df.stack().reset_index()

#print it out
print(df.level_1.str.cat(df[0].map(str), sep=": ").to_string(index=False))

This problem is best solved with awk. If you're going to use a while/read loop in the shell, you want something like;

{ 
    read header # Discard the first line
    while IFS=\; read car mpg cyl disp horse weight acc model origin; do 
        echo "Car: $car"
        echo "MPG: $mpg"
        echo "Cylinders: $cyl"
    done
} < input

If you are using a shell that supports arrays (eg, bash), you might also like:

{
    IFS=\; read -a header
    while IFS=\; read car mpg cyl disp horse weight acc model origin; do
        echo "${header[0]}: $car"
        echo "${header[1]}: $mpg"
        echo "${header[2]}: $cyl"
    done
} < input

or

{
    IFS=\; read -a header
    while IFS=\; read -a value; do
        for i in 0 1 2; do
            echo "${header[$i]}: ${value[$i]}"
        done
    done
} < input

If you need it sorted by car name, sort the records first with sort(1), and strip the leading whitespace while your at at it.

# sort records (excluding header), by field 1
{ sed 1q cars2.csv
sed '1,2d; s/^[[:space:]]*//' cars2.csv |
sort -t \; -k1,1; } |

awk -F \; '
FNR==1 {
    for (i=1; i<NF; i++) {
        keys[i] = $i
    }
}
FNR!=1 {
    for (i=1; i<NF; i++) {
        print keys[i]": "$i
    }
}'
Related