英文:
Chromedp working in non-headless but doing nothing in headless
问题
我正在使用Chromedp偶尔在社交媒体上发布一些内容。当我在非无头模式下运行脚本时,它运行得很完美。
但是当我尝试在无头模式下运行时,什么都没有发生。显然出了些问题,但由于我无法“看到”出了什么问题,我感到困惑。
有人可以告诉我在无法直观地看到Chromedp在做什么时,应该如何调试吗?
也许可以解释一下为什么非无头模式可以工作,但无头模式不行,因为我不明白其中的原因。
我尝试了这段代码(进行了简化),在非无头模式下它运行得很完美。一旦我切换到无头模式,它就什么都不做了。
opts := append(chromedp.DefaultExecAllocatorOptions[:],
chromedp.Flag("headless", false),
)
actx, acancel := chromedp.NewExecAllocator(context.Background(), opts...)
defer acancel()
ctx, cancel := chromedp.NewContext(
actx,
chromedp.WithLogf(log.Printf),
)
defer cancel()
ctx, cancel = context.WithTimeout(ctx, 120*time.Second)
defer cancel()
err = chromedp.Run(ctx,
emulation.SetUserAgentOverride("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0"),
//chromedp.ResetViewport(),
chromedp.Navigate("https://example.com/"),
chromedp.Sleep(time.Second*5),
chromedp.MouseClickXY(1075, 33),
chromedp.Sleep(time.Second*5),
// 还有很多点击和按键操作
chromedp.MouseClickXY(850, 61),
chromedp.Sleep(time.Second*10),
// 结束
chromedp.Stop(),
)
if err != nil {
fmt.Println(err)
}
我添加了一行“debug”代码,现在我在控制台上看到了大量的代码(我猜是HTML),但老实说我仍然无法弄清楚出了什么问题。
英文:
I am using Chromedp to post some stuff occasionally on Social Media. When I run the script in non-headless mode it is working perfect.
When I try to run it in headless mode, nothing happens. Obviously something is going wrong, but since I cannot "see" what is going wrong I am clueless.
Can anyone tell me how one should debug something when not able to visually see what Chromedp is doing?
And maybe explain to me why non-headless is working but headless not, because I do not understand how that can be.
I tried it with this code (minimized it), and in non-headless it works perfect. As soon as I go headless is doesn't do anything.
opts := append(chromedp.DefaultExecAllocatorOptions[:],
chromedp.Flag("headless", false),
)
actx, acancel := chromedp.NewExecAllocator(context.Background(), opts...)
defer acancel()
ctx, cancel := chromedp.NewContext(
actx,
chromedp.WithLogf(log.Printf),
)
defer cancel()
ctx, cancel = context.WithTimeout(ctx, 120*time.Second)
defer cancel()
err = chromedp.Run(ctx,
emulation.SetUserAgentOverride("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0"),
//chromedp.ResetViewport(),
chromedp.Navigate("https://example.com/"),
chromedp.Sleep(time.Second*5),
chromedp.MouseClickXY(1075, 33),
chromedp.Sleep(time.Second*5),
// A LOT MORE CLICKS AND KEY PRESSES GOING ON
chromedp.MouseClickXY(850, 61),
chromedp.Sleep(time.Second*10),
// end
chromedp.Stop(),
)
if err != nil {
fmt.Println(err)
}
I added a "debug" line and now I see tons and tons of code (HTML I guess) output in my console but I still cannot figure out where it goes wrong in all honesty.
答案1
得分: 1
有人能告诉我在无法直观看到Chromedp的操作时,应该如何调试它吗?
启用调试日志
ctx, cancel := chromedp.NewContext(
actx,
chromedp.WithDebugf(log.Printf),
)
输出会很多。将输出重定向到文件中,这样你就可以仔细检查并在需要时与他人分享。
并阅读Chrome DevTools Protocol以了解日志。
截取屏幕截图
你可以使用以下函数之一来截取屏幕截图:
- chromedp.CaptureScreenshot:截取当前浏览器视口的屏幕截图。
- chromedp.FullScreenshot:截取整个页面的全屏截图。
你甚至可以开始一个屏幕录制以保存一系列图像。在这里可以看到一个例子here。
而且,也许你可以解释一下为什么非无头模式可以工作,但无头模式不行。
无头模式下视口大小不同
如果页面是响应式的,并且能够适应不同的窗口大小,那么使用硬编码位置的chromedp.MouseClickXY
可能会失败。截取屏幕截图以确认。避免直接使用chromedp.MouseClickXY
,而是使用chromedp.Click
来点击特定元素。
如果需要,你可以使用chromedp.WindowSize来设置窗口的初始大小。或者使用chromedp.EmulateViewport来更改后续视口的大小。
浏览器被检测为机器人
另一个常见原因是无头模式下的浏览器被检测为机器人,网站拒绝为该浏览器提供服务。请参阅Bot detection engines。
要了解更多信息,请查看以下一些报告的问题:
- https://github.com/chromedp/chromedp/issues/1163
- https://github.com/chromedp/chromedp/issues/877
- https://github.com/chromedp/chromedp/issues/1069
- https://github.com/chromedp/chromedp/issues/1102
问题很可能是由于使用硬编码位置的chromedp.MouseClickXY
引起的。由于问题没有提供足够的信息,我无法做更多的处理。如果你被卡住了,但不能公开分享你的用例,你可以给我发送一封电子邮件,提供具体信息,以便我进行调查。
英文:
> Can anyone tell me how one should debug something when not able to visually see what Chromedp is doing?
Enable the debug log
ctx, cancel := chromedp.NewContext(
actx,
chromedp.WithDebugf(log.Printf),
)
The output will be quite a lot. Redirect the output to a file so that you can examine it carefully and share it to others if in need.
And read the Chrome DevTools Protocol to understand the log.
Take a screenshot
You can take a screenshot with one of the following funcs:
- chromedp.CaptureScreenshot: take a screenshot of the current browser viewport.
- chromedp.FullScreenshot: take a full screenshot of the entire page.
You can even start a screencast to save a series of images. See an example here.
> And maybe explain to me why non-headless is working but headless not
The viewport has a different size in headless mode
If the page is responsive and adapt to different window sizes, then chromedp.MouseClickXY
with a hard-coded position could fail. Take a screenshot to confirm that. Avoid using chromedp.MouseClickXY
directly. Use chromedp.Click
to click a specific element.
If in need, you can use chromedp.WindowSize to set the initial size of the window. Or use chromedp.EmulateViewport to change the size of the viewport later.
The browser is detected as a bot
Another common reason is that the browser in headless mode is detected as a bot and the website refuses to serve the browser. See Bot detection engines.
To understand what it is, see some of the reported issues below:
- https://github.com/chromedp/chromedp/issues/1163
- https://github.com/chromedp/chromedp/issues/877
- https://github.com/chromedp/chromedp/issues/1069
- https://github.com/chromedp/chromedp/issues/1102
The issue is most likely caused by the usages of chromedp.MouseClickXY
with hard-coded position. Since the question does not contain enough information, I can not do much about it. If you're stuck but can not share your use case publicly, you can send me an email with concrete information so that I can look into it.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论