Golang read fixed width text file from S3

Question

I'm trying to use this library to read a fixed width file from S3. It requires an io.Reader, and so far this is what I've got:

import (
    "bufio"
    "fmt"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
    "github.com/o1egl/fwencoder"
)

type leRecord struct {
    fullName       string
    firstName      string
    lastName       string
    primaryAddress string
    city           string
    state          string
    zipcode        string
    MailKey        string
}

func main() {
    getObjectReturnMaps("key", "bucket")

}

func getObjectReturnMaps(key, bucket string) {
    // Initialize an AWS session with configured credentials
    sess, err := session.NewSession(&aws.Config{
        Region: aws.String("us-east-1")},
    )
    if err != nil {
        panic(err)
    }

    // Define client
    s3Client := s3.New(sess)

    // Using the key, get the object from the bucket
    obj, err := s3Client.GetObject(&s3.GetObjectInput{
        Bucket: aws.String(bucket),
        Key:    aws.String(key),
    })
    if err != nil {
        fmt.Println(err)
    }

    // Read the body, parse it into variable
    var records []leRecord

    reader := bufio.NewReader(obj.Body)
    err = fwencoder.UnmarshalReader(reader, &records)
    if err != nil {
        fmt.Println("Error", err)
    }
    fmt.Println(records)
}

however printing the records variable returns blanks:

[{       } {       } {       } {       } {       }......]

but if I print the reader it clearly has values:

&{[32 32 88 10 84 82 69 65 68 87 69 76 76 32 32 68 65 78 73 69 76 32 32 82... ]

So I'm wondering where I'm going wrong. Is it because im passing a bufio.Reader instead of an io.Reader?

EDIT:

Here is some sample data:

John                        Smith                       J                       123 Street Name                       SampleTown                       NY                       12345                       X
Jane                        Smith                       J                       456 Name Street                       Anywhere                         TX                       78901                       X

There are no column headers in these files

user_1416660 · Answer

As commented by Marc, fwencoder, considers the first line of the file as header.

https://github.com/o1egl/fwencoder/blob/8a2f1ab8210ee2018fd171ea1bea94eaa64d043c/decoder.go#L120

if !isHeaderParsed {
        isHeaderParsed = true
        headersLength = len(lineRunes)
        columns, err = parseHeaders(line, columnNames)
        if err != nil {
            return err
        }
        continue
    }

So if you can control the files written to s3 try adding the header or if you are sure about the order of column you can rely on "encoding/csv" something like here

Golang read fixed width text file from S3

1 Answers