英文:
Python Selenium Undetected Chromedriver doesn't work with auth proxy
问题
我使用Undetected Chromedriver来绕过Cloudflare的检测。
我尝试强制它与代理一起工作,但没有成功。
第一次尝试(IP没有更改):
import undetected_chromedriver as uc
options = uc.ChromeOptions()
...
ip, port, username, password = proxy.split(':') # 代理格式为ip:port:username:password
options.add_argument(f'--proxy-server=http://{username}:{password}@{ip}:{port}')
driver = uc.Chrome(executable_path=ChromeDriverManager().install(), options=options)
driver.get('https://2ip.ru/')
第二次尝试(使用Selenium-Wire):
from seleniumwire import undetected_chromedriver as uc
...
ip, port, username, password = proxy.split(':') # 代理格式为ip:port:username:password
wire_options = {
'proxy': {
'http': f'http://{username}:{password}@{ip}:{port}',
'https': f'http://{username}:{password}@{ip}:{port}',
'no_proxy': 'localhost,127.0.0.1' # 排除的地址
}
}
driver = uc.Chrome(executable_path=ChromeDriverManager().install(), options=options, seleniumwire_options=wire_options)
driver.get('https://2ip.ru/')
浏览器尝试使用代理连接,但我需要手动输入密码和用户名。
如果您能指出我的错误或提供其他解决方案,我将不胜感激。
英文:
I use Undetected Chromedriver to bypass Cloudflare detection.
I trying force it to work with proxy, but unsuccessful.
First attempt (IP just doesn't change):
import undetected_chromedriver as uc
options = uc.ChromeOptions()
...
ip,port,username,password = proxy.split(':') #proxy in format ip:port:username:password
options.add_argument(f'--proxy-server=http://{username}:{password}@{ip}:{port}')
driver = uc.Chrome(executable_path=ChromeDriverManager().install(), options=options)
driver.get('https://2ip.ru/')
Second attempt (with Selenium-Wire):
from seleniumwire import undetected_chromedriver as uc
...
ip,port,username,password = proxy.split(':') #proxy in format ip:port:username:password
wire_options = {
'proxy': {
'http': f'http://{username}:{password}@{ip}:{port}',
'https': f'http://{username}:{password}@{ip}:{port}',
'no_proxy': 'localhost,127.0.0.1' # excludes
}
}
driver = uc.Chrome(executable_path=ChromeDriverManager().install(), options=options,seleniumwire_options=wire_options)
driver.get('https://2ip.ru/')
Browser trying to connect with proxy, but I need to manually enter password and username.
I will be thanks, if you show me my mistakes or offer another decision of problem.
答案1
得分: 0
SeleniumBase 具有一个特殊的未检测到-chromedriver模式,可让您通过方法参数(或命令行)设置代理设置。
pip install seleniumbase
,然后在填写SB()
调用中的代理设置后运行python
:
from seleniumbase import SB
with SB(uc=True, pls="none", proxy="USER:PASS@HOST:PORT") as sb:
sb.open("https://nowsecure.nl/#relax")
sb.assert_text("OH YEAH, you passed!", "h1")
sb.post_message("Selenium wasn't detected!")
它将uc
设置为True(启用未检测到-chromedriver模式),将pls
设置为"none"
(在该模式下使用不同的pageLoadStrategy
可能会冻结代理测试),并设置代理设置(在您填写所需设置后)。
英文:
SeleniumBase has a special undetected-chromedriver mode that lets you set the proxy settings via method arg (or on the command-line).
pip install seleniumbase
, then run with python
after you fill in the proxy settings on the SB()
call:
from seleniumbase import SB
with SB(uc=True, pls="none", proxy="USER:PASS@HOST:PORT") as sb:
sb.open("https://nowsecure.nl/#relax")
sb.assert_text("OH YEAH, you passed!", "h1")
sb.post_message("Selenium wasn't detected!")
It sets uc
to True (to enabled undetected-chromedriver mode), it sets pls
to "none" (a different pageLoadStrategy might freeze up proxy tests in that mode), and it sets the proxy
settings (after you fill in the one you want).
答案2
得分: 0
在互联网上搜索了一整天后,我发现Chrome不支持基本的代理身份验证。感谢[AbdeLhalimSB][1]指引我找到了[正确的解决方案][2]。但是这个解决方案以及其他类似的解决方案对我不起作用,因为我的浏览器拒绝从zip存档加载扩展程序。因此,我需要更改这段代码,以便浏览器可以从文件夹加载扩展程序。
extension.py:
import os
def proxies(username, password, endpoint, port, path):
try:
os.mkdir(path)
except FileExistsError:
pass
manifest_json = """
{
"version": "1.0.0",
"manifest_version": 2,
"name": "Proxies",
"permissions": [
"proxy",
"tabs",
"unlimitedStorage",
"storage",
"<all_urls>",
"webRequest",
"webRequestBlocking"
],
"background": {
"scripts": ["background.js"]
},
"minimum_chrome_version": "22.0.0"
}
"""
background_js = """
var config = {
mode: "fixed_servers",
rules: {
singleProxy: {
scheme: "http",
host: "%s",
port: parseInt(%s)
},
bypassList: ["localhost"]
}
};
chrome.proxy.settings.set({ value: config, scope: "regular" }, function() {});
function callbackFn(details) {
return {
authCredentials: {
username: "%s",
password: "%s"
}
};
}
chrome.webRequest.onAuthRequired.addListener(
callbackFn,
{ urls: ["<all_urls>"] },
['blocking']
);
""" % (endpoint, port, username, password)
with open(path + "\\manifest.json", 'w') as m_file:
m_file.write(manifest_json)
with open(path + "\\background.js", 'w') as b_file:
b_file.write(background_js)
main.py:
from webdriver_manager.chrome import ChromeDriverManager
import undetected_chromedriver as uc
import os
from extension import proxies
options = uc.ChromeOptions()
script_address = os.getcwd()
...
ip, port, username, password = proxy.split(':') # 代理的格式是ip:端口:用户名:密码
proxies(username, password, ip, int(port), script_address + '\\Proxies\\' + ip)
options.add_argument('--load-extension=' + script_address + '\\Proxies\\' + ip)
driver = uc.Chrome(executable_path=ChromeDriverManager().install(), options=options)
请注意,此解决方案在无头模式下可以工作,但在隐身模式下不可以。
[1]: https://stackoverflow.com/users/20112601/abdelhalimsb
[2]: https://stackoverflow.com/questions/76396403/how-can-i-integrate-proxies-into-my-selenium-web-scraper-for-true-people-search/76400335#76400335
英文:
After day of searches on the Internet, I found out that Chrome doesn't support basic proxy authentification. Thank [AbdeLhalimSB][1] for leading me to the [right solution][2]. But this and other similar solutions don't work for me because my browser refuses to load extensions from zip archive. So, I needed to change this code so that browser can load extension from folder.
extension.py:
import os
def proxies(username, password, endpoint, port,path):
try:
os.mkdir(path)
except FileExistsError:
pass
manifest_json = """
{
"version": "1.0.0",
"manifest_version": 2,
"name": "Proxies",
"permissions": [
"proxy",
"tabs",
"unlimitedStorage",
"storage",
"<all_urls>",
"webRequest",
"webRequestBlocking"
],
"background": {
"scripts": ["background.js"]
},
"minimum_chrome_version":"22.0.0"
}
"""
background_js = """
var config = {
mode: "fixed_servers",
rules: {
singleProxy: {
scheme: "http",
host: "%s",
port: parseInt(%s)
},
bypassList: ["localhost"]
}
};
chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});
function callbackFn(details) {
return {
authCredentials: {
username: "%s",
password: "%s"
}
};
}
chrome.webRequest.onAuthRequired.addListener(
callbackFn,
{urls: ["<all_urls>"]},
['blocking']
);
""" % (endpoint, port, username, password)
with open(path+"\\manifest.json", 'w') as m_file:
m_file.write(manifest_json)
with open(path+"\\background.js", 'w') as b_file:
b_file.write(background_js)
main.py:
from webdriver_manager.chrome import ChromeDriverManager
import undetected_chromedriver as uc
import os
from extension import proxies
options = uc.ChromeOptions()
script_address = os.getcwd()
...
ip,port,username,password = proxy.split(':') #proxy in format ip:port:username:password
proxies(username, password, ip, int(port),script_address+'\\Proxies\\'+ip)
options.add_argument('--load-extension='+script_address+'\\Proxies\\'+ip)
driver = uc.Chrome(executable_path=ChromeDriverManager().install(), options=options)
Note, this solution will work in headless mode, but not in incognito mode.
[1]: https://stackoverflow.com/users/20112601/abdelhalimsb
[2]: https://stackoverflow.com/questions/76396403/how-can-i-integrate-proxies-into-my-selenium-web-scraper-for-true-people-search/76400335#76400335
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论