How proper get content from url in GO?

huangapple go评论86阅读模式
英文:

How proper get content from url in GO?

问题

例如,我需要从页面https://aliexpress.ru/item/4001275226820.html获取内容。

在Postman和浏览器中,我可以获取到页面的HTML内容。

但是当我尝试使用GO语言时,无法获取内容。

package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
)

func main() {
	resp, err := http.Get("https://aliexpress.ru/item/4001275226820.html")
	defer resp.Body.Close()

	if err != nil {panic(err)}

	html, err1 := ioutil.ReadAll(resp.Body)
	if err1 != nil {panic(err1)}

	fmt.Printf("%s\n", html)
}

但是会出现"stopped after 10 redirects"的错误,我做错了什么?

英文:

For example I need content from page https://aliexpress.ru/item/4001275226820.html

In Postman and in browser I get html content of page

But when I try use GO i can`t get content

package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
)

func main() {
	resp, err := http.Get("https://aliexpress.ru/item/4001275226820.html")
	defer resp.Body.Close()

	if err != nil {panic(err)}

	html, err1 := ioutil.ReadAll(resp.Body)
	if err1 != nil {panic(err1)}

	fmt.Printf("%s\n", html)
}

But get panic "stopped after 10 redirects"
what am I doing wrong?

答案1

得分: 0

以下是翻译好的代码:

package main

import (
   "fmt"
   "net/http"
)

func cookies() ([]*http.Cookie, error) {
   req, err := http.NewRequest(
      "HEAD", "https://login.aliexpress.ru/sync_cookie_write.htm", nil,
   )
   if err != nil {
      return nil, err
   }
   val := req.URL.Query()
   val.Set("acs_random_token", "1")
   req.URL.RawQuery = val.Encode()
   res, err := new(http.Transport).RoundTrip(req)
   if err != nil {
      return nil, err
   }
   return res.Cookies(), nil
}

func main() {
   req, err := http.NewRequest(
      "HEAD", "https://aliexpress.ru/item/4001275226820.html", nil,
   )
   if err != nil {
      panic(err)
   }
   cooks, err := cookies()
   if err != nil {
      panic(err)
   }
   for _, cook := range cooks {
      if cook.Name == "xman_f" {
         req.AddCookie(cook)
      }
   }
   res, err := new(http.Transport).RoundTrip(req)
   if err != nil {
      panic(err)
   }
   fmt.Printf("%+v\n", res)
}
英文:

The below code gets me a 200 response. You might be able to simplify it, but should be enough to get you started:

package main

import (
   "fmt"
   "net/http"
)

func cookies() ([]*http.Cookie, error) {
   req, err := http.NewRequest(
      "HEAD", "https://login.aliexpress.ru/sync_cookie_write.htm", nil,
   )
   if err != nil {
      return nil, err
   }
   val := req.URL.Query()
   val.Set("acs_random_token", "1")
   req.URL.RawQuery = val.Encode()
   res, err := new(http.Transport).RoundTrip(req)
   if err != nil {
      return nil, err
   }
   return res.Cookies(), nil
}

func main() {
   req, err := http.NewRequest(
      "HEAD", "https://aliexpress.ru/item/4001275226820.html", nil,
   )
   if err != nil {
      panic(err)
   }
   cooks, err := cookies()
   if err != nil {
      panic(err)
   }
   for _, cook := range cooks {
      if cook.Name == "xman_f" {
         req.AddCookie(cook)
      }
   }
   res, err := new(http.Transport).RoundTrip(req)
   if err != nil {
      panic(err)
   }
   fmt.Printf("%+v\n", res)
}

huangapple
  • 本文由 发表于 2021年10月28日 16:31:55
  • 转载请务必保留本文链接:https://go.coder-hub.com/69750738.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定