Best way to bind shell output to a struct in Go?

user38643 0
Viewed 87

I have some chips running on some hardware and I want to bind the output of a shell command to a struct to do reporting/logging on.

Num item1: 2
INDEX LOAD MODEL_LOAD INST MEM  SHARE_MEM P2P_MEM DEVICE         NAMESPACE
1     2    3          4    50    600         700       1     1
1a     2b    3c          4c    5d    6e         7f       2     2
Num item2: 2
INDEX LOAD MODEL_LOAD INST MEM  SHARE_MEM P2P_MEM DEVICE         NAMESPACE
2a     2b    2c          3    0    0         0       1     1
1     0    0          0    0    0         0       2     2
**************************************************

Attempt

cat out.txt | grep -i "Num $1" -A 3 | grep -i nvme | tr -s ' ' | cut -d' ' -f1-7

This actually isn't too bad, I can pass in an arg like decoders or encoders and get the load metrics for each chip. However, I'm curious now the best way to bind this to a struct in Go.

Currently, what I can do is code up a custom deserializer from something like:

func main() {
    out, err := exec.Command("/bin/sh", "metrics.sh", "encoders").Output()
    if err != nil {
        fmt.Println(err)
        log.Fatal(err)
    }

    fmt.Println(string(out))
}

But I feel like there has to be a better way, like outputting as JSON and binding to a struct or something.

awk bash go
2 Answers

I would convert your input file to CSV, because it suits the original tabular data and also because the Go language have a CSV encoder/decoder in its standard library:

awk -v OFS=',' '
    $1 == "Num" {
        count = $3
        type = $2

        getline
        if ( !header++ ) {
            $(NF+1) = "ID"
            print
        }
        for ( id = 1; id <= count; id++ ) {
            getline
            $(NF+1) = type id
            print
        }
    }
' file.txt

Warning: the code doesn't CSV-escape the fields

INDEX,LOAD,MODEL_LOAD,INST,MEM,SHARE_MEM,P2P_MEM,DEVICE,NAMESPACE,ID
1,2,3,4,50,600,700,/dev/nvme0,/dev/nvme0n1,decoders:1
1a,2b,3c,4c,5d,6e,7f,/dev/nvme1,/dev/nvme1n1,decoders:2
2a,2b,2c,3,0,0,0,/dev/nvme0,/dev/nvme0n1,encoders:1
1,0,0,0,0,0,0,/dev/nvme1,/dev/nvme1n1,encoders:2
0,0,0,0,0,0,0,/dev/nvme0,/dev/nvme0n1,scalers:1
1,0,0,0,0,0,0,/dev/nvme1,/dev/nvme1n1,scalers:2

N.B. Writing a parser in Go for your input format shouldn't be that difficult

Fravadona0

How about starting directly with the text you care about in Go? You'll have far more control in Go than you'll ever have with shell utilities.

This is a little state machine that looks for the leading text, "Num", to indicate the beginning of a new Item. The next line is the header, which is skipped, and following lines are converted to a Row, which are added to that Item. At the boundary between items and at the end of the input text/file, the last Item is added to the set of all Items.

package main

import (
    "bufio"
    "fmt"
    "regexp"
    "strings"
)

var txt = `
Num item1: 2
INDEX LOAD MODEL_LOAD INST MEM  SHARE_MEM P2P_MEM DEVICE         NAMESPACE
1     2    3          4    50    600         700       1     1
1a     2b    3c          4c    5d    6e         7f       2     2
Num item2: 2
INDEX LOAD MODEL_LOAD INST MEM  SHARE_MEM P2P_MEM DEVICE         NAMESPACE
2a     2b    2c          3    0    0         0       1     1
1     0    0          0    0    0         0       2     2
Num item3: 1
INDEX LOAD MODEL_LOAD INST MEM  SHARE_MEM P2P_MEM DEVICE         NAMESPACE
i     iib    iic          iii    zero    zero         zero       i     i
**************************************************
`

var columns = regexp.MustCompile(`\s+`)

type Row struct {
    Index,
    Load,
    Model_Load,
    Inst_Mem,
    Share_Mem,
    P2p_Mem,
    Device,
    Namespace string
}

type Item []Row

func main() {
    r := strings.NewReader(txt)
    scanner := bufio.NewScanner(r)

    items := make([]Item, 0)

    var item Item
    for scanner.Scan() {
        line := scanner.Text()
        line = strings.TrimSpace(line)

        if len(line) == 0 ||
            strings.HasPrefix(line, "***") {
            continue
        }

        // find beginning of an "item": if any previous item, save it and
        // reset item to append future rows; skip header line; continue
        if strings.HasPrefix(line, "Num item") {
            if len(item) > 0 {
                items = append(items, item)
                item = make(Item, 0)
            }
            scanner.Scan() // skip header
            continue
        }

        cols := columns.Split(line, -1)
        row := Row{cols[0], cols[1], cols[2], cols[3], cols[4], cols[5], cols[6], cols[7]}
        item = append(item, row)
    }

    // deal with last/trailing item
    if len(item) > 0 {
        items = append(items, item)
    }

    for i, item := range items {
        fmt.Printf("Item %d\n", i+1)
        for _, row := range item {
            fmt.Println(row)
        }
    }
}

That prints the following:

Item 1
{1 2 3 4 50 600 700 1}
{1a 2b 3c 4c 5d 6e 7f 2}
Item 2
{2a 2b 2c 3 0 0 0 1}
{1 0 0 0 0 0 0 2}
Item 3
{i iib iic iii zero zero zero i}

I don't know of a better way to create the struct, but it is direct, and fairly clean.

Zach Young0
Related