英文:
Trying to log into Amazon.com to pull data, but getting an Enable Cookies response (using Go)
问题
我正在尝试使用Go语言登录我的亚马逊账户,以自动获取一些信息,但是登录时遇到了cookie的问题。以下是我使用的代码的简化版本:
package main
import (
"bytes"
"io/ioutil"
"net/http"
"net/http/cookiejar"
"net/url"
"strconv"
)
func CheckThis(AmazonUsername string, AmazonPassword string) error {
var LogonURL string
// 设置URL
LogonURL = "https://www.amazon.com/ap/signin"
// 构造一些表单数据
form := url.Values{}
form.Add("appAction", "SIGNIN")
form.Add("email", AmazonUsername)
form.Add("password", AmazonPassword)
form.Add("create", "0")
form.Add("appActionToken", "$VALUE")
form.Add("openid.pape.max_auth_age", "$VALUE==")
form.Add("openid.identity", "$VALUE=")
form.Add("openid.assoc_handle", "$VALUE")
form.Add("openid.mode", "$VALUE")
form.Add("openid.ns.pape", "$VALUE==")
form.Add("openid.claimed_id", "$VALUE=")
form.Add("pageId", "$VALUE")
form.Add("openid.ns", "$VALUE=")
// 亚马逊使用cookies
cookieJar, _ := cookiejar.New(nil)
// 在结构体中创建一个带有cookiejar的新客户端...
client := &http.Client{
Jar: cookieJar,
}
// 构造包含登录信息的表单的请求
req, _ := http.NewRequest("POST", LogonURL, bytes.NewBufferString(form.Encode()))
// 一些其他的头部信息
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
req.Header.Add("Content-Length", strconv.Itoa(len(form.Encode())))
req.Header.Add("Accept-Language", "en-US,en;q=0.8")
req.Header.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8")
req.Header.Add("Connection", "keep-alive")
req.Header.Add("Host", "www.amazon.com")
req.Header.Add("Referer", "https://www.amazon.com/ap/signin")
req.Header.Add("Upgrade-Insecure-Requests", "1")
req.Header.Add("Origin", "https://www.amazon.com")
req.Header.Add("Cache-Control", "max-age=0")
// 发送请求...
resp, _ := client.Do(req)
// 响应中的内容是什么?
charResponse, _ := ioutil.ReadAll(resp.Body)
resp.Body.Close()
// 将响应体写入一个标题为...的文本文件中
_ = WriteOutputToFile(string(charResponse), "response.html")
// 完成!
return nil
}
$VALUE的条目是因为我不确定这些字符串对我的账户是否重要,所以我将它们删除了;这些值是我从Chrome登录会话的开发者工具中获取的。出于简洁起见,我还删除了错误检查。
我保存的回复页面(在Chrome中打开response.html)如下所示:
为了保持客户端请求/响应中的cookie,我缺少什么?
或者,我是否遗漏了某些内容,导致我保存的响应页面在渲染HTML时尝试从亚马逊获取元素,并且cookie问题是因为浏览器在尝试查看Go应用程序的结果时缺少cookie信息?
英文:
I'm trying to use Go to log into my account on Amazon to automatically pull some information, but I'm having trouble logging in because it complains about cookies. Here's a sanitized version of the code I was using:
package main
import (
"bytes"
"io/ioutil"
"net/http"
"net/http/cookiejar"
"net/url"
"strconv"
)
func CheckThis(AmazonUsername string, AmazonPassword string) error {
var LogonURL string
// Set the url
LogonURL = "https://www.amazon.com/ap/signin"
// Craft some form data
form := url.Values{}
form.Add("appAction", "SIGNIN")
form.Add("email", AmazonUsername)
form.Add("password", AmazonPassword)
form.Add("create", "0")
form.Add("appActionToken", “$VALUE”)
form.Add("openid.pape.max_auth_age", "$VALUE==")
form.Add("openid.identity", "$VALUE=")
form.Add("openid.assoc_handle", "$VALUE")
form.Add("openid.mode", "$VALUE")
form.Add("openid.ns.pape", "$VALUE==")
form.Add("openid.claimed_id", "$VALUE=")
form.Add("pageId", "$VALUE")
form.Add("openid.ns", "$VALUE=")
// Amazon sells cookies
cookieJar, _ := cookiejar.New(nil)
// Create a new client with the cookiejar in the struct...
client := &http.Client{
Jar: cookieJar,
}
// Craft the request to send to the website with the form containing login info
req, _ := http.NewRequest("POST", LogonURL, bytes.NewBufferString(form.Encode()))
// Some more headers
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
req.Header.Add("Content-Length", strconv.Itoa(len(form.Encode())))
req.Header.Add("Accept-Language", "en-US,en;q=0.8")
req.Header.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8")
req.Header.Add("Connection", "keep-alive")
req.Header.Add("Host", "www.amazon.com")
req.Header.Add("Referer", "https://www.amazon.com/ap/signin")
req.Header.Add("Upgrade-Insecure-Requests", "1")
req.Header.Add("Origin", "https://www.amazon.com")
req.Header.Add("Cache-Control", "max-age=0")
// And we're off to the races...
resp, _ := client.Do(req)
// What was in the response?
charResponse, _ := ioutil.ReadAll(resp.Body)
resp.Body.Close()
// Write response body to a text file with title of…
_ = WriteOutputToFile(string(charResponse), “response.html")
// All done!
return nil
}
The $VALUE entries are because I'm not sure if the strings are significant to my account so I removed them; these are values I pulled from the developer tools of a Chrome login session. I also removed err checks for brevity.
The reply page (opening response.html on my drive within Chrome) looks like this:
What am I missing in order to keep the cookie with the client req/resp for the sign-in and later pages?
Or am I missing something where the response page I save is trying to pull elements from Amazon when I render the HTML, and the cookie issue is because the browser would be missing cookie information when I'm trying to view the results from the Go application?
答案1
得分: 1
我很确定亚马逊在每次尝试登录时都会使用不同的数据,所以最好解析登录表单。以下是示例代码:
package main
import (
"bytes"
"io/ioutil"
"net/http"
"net/http/cookiejar"
"net/url"
"github.com/PuerkitoBio/goquery"
"log"
)
func checkError(err error) {
if err != nil {
log.Fatal(err)
}
}
func CheckThis(AmazonUsername string, AmazonPassword string) error {
cookieJar, _ := cookiejar.New(nil)
client := &http.Client{
Jar: cookieJar,
}
res, err := client.Get("https://www.amazon.com/gp/sign-in.html/ref=ord_cart_unrec_signin")
checkError(err)
doc, err := goquery.NewDocumentFromResponse(res)
form := url.Values{}
doc.Find("form[name='signIn'] input").Each(func(i int, s *goquery.Selection) {
name, exist := s.Attr("name")
if exist {
value, exist := s.Attr("value")
if exist {
form.Add(name, value)
}
}
})
form.Set("email", AmazonUsername)
form.Set("password", AmazonPassword)
req, _ := http.NewRequest("POST", "https://www.amazon.com/ap/signin", bytes.NewBufferString(form.Encode()))
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
res, err = client.Do(req)
checkError(err)
defer res.Body.Close()
charResponse, _ := ioutil.ReadAll(res.Body)
ioutil.WriteFile("response.html", charResponse, 0777)
return nil
}
func main() {
CheckThis("", "")
}
希望对你有帮助!
英文:
I'm pretty sure Amazon uses different data every time when you try to login so better to parse login form. Here is example
package main
import (
"bytes"
"io/ioutil"
"net/http"
"net/http/cookiejar"
"net/url"
"github.com/PuerkitoBio/goquery"
"log"
)
func checkError(err error) {
if err != nil {
log.Fatal(err)
}
}
func CheckThis(AmazonUsername string, AmazonPassword string) error {
cookieJar, _ := cookiejar.New(nil)
client := &http.Client{
Jar: cookieJar,
}
res, err := client.Get("https://www.amazon.com/gp/sign-in.html/ref=ord_cart_unrec_signin")
checkError(err)
doc, err := goquery.NewDocumentFromResponse(res)
form := url.Values{}
doc.Find("form[name='signIn'] input").Each(func(i int, s *goquery.Selection) {
name, exist := s.Attr("name")
if exist {
value, exist := s.Attr("value")
if exist {
form.Add(name, value)
}
}
})
form.Set("email", AmazonUsername)
form.Set("password", AmazonPassword)
req, _ := http.NewRequest("POST", "https://www.amazon.com/ap/signin", bytes.NewBufferString(form.Encode()))
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
res, err = client.Do(req)
checkError(err)
defer res.Body.Close()
charResponse, _ := ioutil.ReadAll(res.Body)
ioutil.WriteFile("response.html", charResponse, 0777)
return nil
}
func main() {
CheckThis("", "")
}
答案2
得分: 0
亚马逊的索引在我的浏览器上设置了这些(以及更多)cookie:
我猜你需要将它们设置到cookiejar中以模拟浏览器行为(亚马逊期望你的请求中包含这些cookie,所以它告诉你没有启用cookie,因为缺少必需的安全值)。
我同意@JimB的观点,你应该使用亚马逊的SDK。你没有使用SDK的原因是什么呢?
最好的问候。
英文:
Amazon's index is setting this (and much more) cookies on my browser:
I suppose you need to set them into cookiejar in order to simulate a browser (amazon excepts that from your request, so is telling you have no cookies enabled because there isn't a required security value).
I agree with @JimB, you should be using Amazon SDK. Is there any reason why you are not doing this way?
Best regards.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论