Serve Image File From Github Graph QL API Using Golang

Viewed 44

We have a Github repository that stores image files.

https://github.com/rollthecloudinc/ipe-objects/tree/dev/media

We would like to serve those image files via golang. The golang api runs on aws api gateway as a lambda function. The function in its current state which goes to a blank screen is below.

func GetMediaFile(req *events.APIGatewayProxyRequest, ac *ActionContext) (events.APIGatewayProxyResponse, error) {
    res := events.APIGatewayProxyResponse{StatusCode: 500}

    pathPieces := strings.Split(req.Path, "/")
    siteName := pathPieces[1]
    file, _ := url.QueryUnescape(pathPieces[3]) // pathPieces[2]

    log.Printf("requested media site: " + siteName)
    log.Printf("requested media file: " + file)

    // buf := aws.NewWriteAtBuffer([]byte{})

    // downloader := s3manager.NewDownloader(ac.Session)

    /*_, err := downloader.Download(buf, &s3.GetObjectInput{
        Bucket: aws.String(ac.BucketName),
        Key:    aws.String("media/" + file),
    })

    if err != nil {
        return res, err
    }*/

    ext := strings.Split(pathPieces[len(pathPieces)-1], ".")
    contentType := mime.TypeByExtension(ext[len(ext)-1])

    if ext[len(ext)-1] == "md" {
        contentType = "text/markdown"
    }

    suffix := ""
    if os.Getenv("GITHUB_BRANCH") == "master" {
        suffix = "-prod"
    }

    var q struct {
        Repository struct {
            Object struct {
                ObjectFragment struct {
                    Text     string
                    IsBinary bool
                    ByteSize int
                } `graphql:"... on Blob"`
            } `graphql:"object(expression: $exp)"`
        } `graphql:"repository(owner: $owner, name: $name)"`
    }
    qVars := map[string]interface{}{
        "exp":   githubv4.String(os.Getenv("GITHUB_BRANCH") + ":media/" + file),
        "owner": githubv4.String("rollthecloudinc"),
        "name":  githubv4.String(siteName + suffix),
    }

    err := ac.GithubV4Client.Query(context.Background(), &q, qVars)
    if err != nil {
        log.Print("Github latest file failure.")
        log.Panic(err)
    }
    // log.Printf(q.Repository.Object.ObjectFragment.Text)
    // json.Unmarshal([]byte(q.Repository.Object.ObjectFragment.Text), &obj)
    // log.Printf("END GithubFileUploadAdaptor::LOAD %s", id)

    log.Print("content")
    log.Print(q.Repository.Object.ObjectFragment.Text)

    res.StatusCode = 200
    res.Headers = map[string]string{
        "Content-Type": contentType,
    }
    res.Body = q.Repository.Object.ObjectFragment.Text //base64.StdEncoding.EncodeToString([]byte(q.Repository.Object.ObjectFragment.Text))
    res.IsBase64Encoded = true
    return res, nil
}

The full api file can viewed below but excludes the changes above for migration to Github. This api has been running fine using s3. However, we are now trying to migrate to Github for object storage instead. Have successfully implemented write but are having difficulties described above for reading the file using our lambda.

https://github.com/rollthecloudinc/verti-go/blob/master/api/media/main.go

Help requested to figure out how to serve image files from our Github repo using the golang lambda on aws which can be accessed here as a blank screen.

https://81j44yaaab.execute-api.us-east-1.amazonaws.com/ipe/media/Screen%20Shot%202022-02-02%20at%202.00.29%20PM.png

However, this repo is also a pages site which serves the image just fine.

https://rollthecloudinc.github.io/ipe-objects/media/Screen%20Shot%202022-02-02%20at%202.00.29%20PM.png

Thanks


Further debugging the Text property appears to be empty inside the log.

Log Record


The IsBinary property value being false lead use to the discovery of a typo. The name input for the graph QL invocation was missing -objects. Once the typo was corrected IsBinary started showing up true. However, the Text property value is still empty.

Log Record

Having managed to find some similar issues but for uploading many have suggested that graph QL isn't the right tool for uploading binary data to begin with. Therefore, rather than chase tail we have decided to try the Github REST v3 api. Specifically, the go-github package for golang instead.

https://github.com/google/go-github

Perhaps using the REST api instead will lead to successful results.

1 Answers

An additional step was necessary to fetch the blob contents of the object queried via the graph QL api. Once this was achieved the media file was served with success. This required using the go-github blob api to fetch the blob base64 contents from github.

https://81j44yaaab.execute-api.us-east-1.amazonaws.com/ipe/media/Screen%20Shot%202022-02-02%20at%202.00.29%20PM.png

GetMediaFile lambda

func GetMediaFile(req *events.APIGatewayProxyRequest, ac *ActionContext) (events.APIGatewayProxyResponse, error) {
    res := events.APIGatewayProxyResponse{StatusCode: 500}

    pathPieces := strings.Split(req.Path, "/")
    siteName := pathPieces[1]
    file, _ := url.QueryUnescape(pathPieces[3]) // pathPieces[2]

    log.Print("requested media site: " + siteName)
    log.Print("requested media file: " + file)

    // buf := aws.NewWriteAtBuffer([]byte{})

    // downloader := s3manager.NewDownloader(ac.Session)

    /*_, err := downloader.Download(buf, &s3.GetObjectInput{
        Bucket: aws.String(ac.BucketName),
        Key:    aws.String("media/" + file),
    })

    if err != nil {
        return res, err
    }*/

    ext := strings.Split(pathPieces[len(pathPieces)-1], ".")
    contentType := mime.TypeByExtension(ext[len(ext)-1])

    if ext[len(ext)-1] == "md" {
        contentType = "text/markdown"
    }

    suffix := ""
    if os.Getenv("GITHUB_BRANCH") == "master" {
        suffix = "-prod"
    }

    owner := "rollthecloudinc"
    repo := siteName + "-objects" + suffix

    var q struct {
        Repository struct {
            Object struct {
                ObjectFragment struct {
                    Oid githubv4.GitObjectID
                } `graphql:"... on Blob"`
            } `graphql:"object(expression: $exp)"`
        } `graphql:"repository(owner: $owner, name: $name)"`
    }
    qVars := map[string]interface{}{
        "exp":   githubv4.String(os.Getenv("GITHUB_BRANCH") + ":media/" + file),
        "owner": githubv4.String(owner),
        "name":  githubv4.String(repo),
    }

    err := ac.GithubV4Client.Query(context.Background(), &q, qVars)
    if err != nil {
        log.Print("Github latest file failure.")
        log.Panic(err)
    }

    oid := q.Repository.Object.ObjectFragment.Oid
    log.Print("Github file object id " + oid)

    blob, _, err := ac.GithubRestClient.Git.GetBlob(context.Background(), owner, repo, string(oid))
    if err != nil {
        log.Print("Github get blob failure.")
        log.Panic(err)
    }

    res.StatusCode = 200
    res.Headers = map[string]string{
        "Content-Type": contentType,
    }
    res.Body = blob.GetContent()
    res.IsBase64Encoded = true
    return res, nil
}

Full Source: https://github.com/rollthecloudinc/verti-go/blob/master/api/media/main.go

Related