Is it possible to get a webpage that requires JS in Golang?

user_5674583 0
Viewed 3375

Creating a micro service that gets couple websites main pages html. One of them performs a check for enabled JS and redirects to error page if no JS was detected.

Is there a way around it with Golang?

EDIT: attempted to play with this package (JavaScript interpreter) but with no luck..

EDIT2: its 2020, moved to use js Puppeteer

It uses embedded browser and is a very mature and packed with utilities library. for complex web apps embedded browser is really the only one to go

For backends written in other the js I would still use 'Puppeteer' as a micro service

hope this helps anyone in the future

thanks

server go javascript
3 Answers

Yes, it is possible. Like Gonzalez mentioned earlier, PhantomJS is a good choice. But there are some things I would like to clarify, first there is a problem using the phantomgo repository on Linux as the developer doesn't provide the binary of the PanthomJS for Linux.

The way this repository is used should be using the following instructions:

  1. go get github.com/k4s/phantomgo
  2. go get github.com/k4s/webrowser
  3. Download the pre-compiled binary for your platform of PhantomJS on this page.
  4. Add the binary to the $GOPATH/src/github.com/k4s/phantomgo/phantomgojs folder.
import (
    "fmt"
    "io/ioutil"
    "net/http"

    . "github.com/k4s/webrowser"
)

func main() {
    p := &Param{
        Method:       "GET",
        Url:          "http://google.com",
        Header:       http.Header{"Cookie": []string{"your cookie"}},
        UsePhantomJS: true,
    }
    brower := NewWebrowse()
    resp, err := brower.Download(p)
    if err != nil {
        fmt.Println(err)
    }
    body, err := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
    fmt.Println(resp.Cookies())
}
user_27630600
Related