httplib is an libs help you to curl remote url.
you can use Get to crawl data.
import "github.com/astaxie/beego/httplib"
str, err := httplib.Get("https://beego.me/").String()
if err != nil {
// error
}
fmt.Println(str)
POST data to remote url
req := httplib.Post("https://beego.me/")
req.Param("username","astaxie")
req.Param("password","123456")
str, err := req.String()
if err != nil {
// error
}
fmt.Println(str)
The default timeout is 60 seconds, function prototype:
SetTimeout(connectTimeout, readWriteTimeout time.Duration)
Example:
// GET
httplib.Get("https://beego.me/").SetTimeout(100 * time.Second, 30 * time.Second)
// POST
httplib.Post("https://beego.me/").SetTimeout(100 * time.Second, 30 * time.Second)
If you want to debug the request info, set the debug on
httplib.Get("https://beego.me/").Debug(true)
str, err := Get("https://beego.me/").SetBasicAuth("user", "passwd").String()
if err != nil {
// error
}
fmt.Println(str)
If request url is https, You can set the client support TSL:
httplib.SetTLSClientConfig(&tls.Config{InsecureSkipVerify: true})
More info about the tls.Config please visit https://golang.org/pkg/crypto/tls/#Config
some servers need to specify the protocol version of HTTP
httplib.Get("https://beego.me/").SetProtocolVersion("HTTP/1.1")
some http request need setcookie. So set it like this:
cookie := &http.Cookie{}
cookie.Name = "username"
cookie.Value = "astaxie"
httplib.Get("https://beego.me/").SetCookie(cookie)
httplib support mutil file upload, use req.PostFile()
req := httplib.Post("https://beego.me/")
req.Param("username","astaxie")
req.PostFile("uploadfile1", "httplib.pdf")
str, err := req.String()
if err != nil {
// error
}
fmt.Println(str)
See godoc for further documentation and examples.