Good day! There was a need to parse the html page .onion using the Go language. A framework for scraping Colly was taken for implementation, but a proxy is needed to connect to the tor. Colly has a corresponding function (Set Proxy Func()). For some unknown reason, none of the selected server addresses (including the proxy of the torus itself on 127.0.0.1:9050) does not allow you to connect to the desired resource (the timeout comes out, then eof comes instead of the code, then the incomprehensible error nor Implementation). Tell me, please, what solution can this problem have?
This is the proxy installation function
func SetLocalProxy(c *colly.Collector) {
rp, err := proxy.RoundRobinProxySwitcher("http://203.28.8.37:80")
if err != nil {
log.Fatal(err)
}
c.SetProxyFunc(rp)
}
And this is a parsing function
func Parsing(){
c := colly.NewCollector(colly.AllowURLRevisit())
c.SetRequestTimeout(0)
SetLocalProxy(c)
var leakDB string = "someURL"
c.OnHTML("a[href]", func(e *colly.HTMLElement) {
fmt.Println(e)
})
c.OnError(func(r *colly.Response, err error) {
fmt.Println("Request URL:", r.Request.URL, "failed with response:", r, "\nError:", err)
})
err := c.Visit(leakDB)
if err != nil {
return
}
}
In the example above, the proxy server "http" is specified, but this is just an example. socks5 was used for the Tor browser, which can be seen in the screenshot below. https://habrastorage.org/webt/62/aa/fe/62aafeeb79f70642613391.jpeg
There is an assumption that the problem is an external setup (proxy server on a PC)