下载需要SSL证书验证的网页在Python中

huangapple go评论105阅读模式
英文:

Downloading webpage requiring SSL certificate authentication in Python

问题

I'm here to provide translations. Here's the translated content:

我正在尝试爬取一个需要SSL证书进行身份验证的端点。当我尝试在浏览器中访问该站点时,会弹出一个窗口,我需要选择要随请求发送的证书。在下载网页时,我尝试了以下方法:

import urllib
urllib.request.urlopen("https://mywebpage.com/some/endpoint", cafile="C:/Users/Me/path/to/my/cert.p12")

但我收到了以下错误:

ssl.SSLError: [X509: NO_CERTIFICATE_OR_CRL_FOUND] no certificate or crl found (_ssl.c:4149)

我认为这可能是因为我发送的是PKCS12证书而不是PEM文件,因此我进行了以下操作:

openssl pkcs12 -in "C:/Users/Me/path/to/my/cert.p12" -out "C:/Users/Me/path/to/my/cert.pem"

我为PKCS12密码和PEM密码提供了相同的值。然后我尝试了这个:

import urllib
urllib.request.urlopen("https://mywebpage.com/some/endpoint", cafile="C:/Users/Me/path/to/my/cert.pem")

然而,这返回了以下SSL错误:

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)

当我执行 ssl.SSLContext().load_verify_locations(cafile="C:/Users/Me/path/to/my/cert.pem") 时,我没有收到错误,并且证书信息似乎已加载。

然后,我决定尝试解决这个问题,尝试搜索特定的SSL错误。我找到了这个 SO 问题。由于我使用的是Windows,我的OpenSSL版本可能与Python期望的不同,因此我尝试了以下操作:

import ssl
import certifi
from urllib import request
request.urlopen("https://mywebpage.com/some/endpoint", 
    context=ssl.create_default_context(cafile=certifi.where()))

但这返回了以下SSL错误:

ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1007)

那么,我在这里做错了什么?似乎服务器无法验证我随请求发送的SSL证书,但在我检查时,它似乎是匹配的。我该如何使此请求生效?

英文:

I'm trying to scrape an endpoint that requires an SSL certificate for authentication. When I try accessing the site in the browser, a window comes up where I select the certificate to be sent with the request. When downloading the webpage, I've tried doing:

import urllib
urllib.request.urlopen("https://mywebpage.com/some/endpoint", cafile = "C:/Users/Me/path/to/my/cert.p12")

but I got the following error:

> ssl.SSLError: [X509: NO_CERTIFICATE_OR_CRL_FOUND] no certificate or crl found (_ssl.c:4149)

I thought this might be because I was sending a PKCS12 instead of a PEM file, so I did the following:

openssl pkcs12 -in "C:/Users/Me/path/to/my/cert.p12" -out "C:/Users/Me/path/to/my/cert.pem"

I provided the same value for the PKCS12 passphrase and the PEM password. I then tried doing this:

import urllib
urllib.request.urlopen("https://mywebpage.com/some/endpoint", cafile = "C:/Users/Me/path/to/my/cert.pem")

However, this returns the following SSL error:

> ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)

When I do ssl.SSLContext().load_verify_locations(cafile = "C:/Users/Me/path/to/my/cert.pem") I get no errors and the certificate information seems to load.

I then decided to try getting around the issue and tried searching for the specific SSL error. I came across this SO question. Since I'm using Windows, my version of OpenSSL is likely configured to be different from what Python is expecting, so I tried doing the following:

import ssl
import certifi
from urllib import request
request.urlopen("https://mywebpage.com/some/endpoint", 
    context = ssl.create_default_context(cafile = certifi.where()))

but this returned the following SSL error:

> ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1007)

So, what am I doing wrong here? It appears that the server was unable to verify the SSL certificate I sent with the request but, when I check it, it appears to match. How can I make this request work?

答案1

得分: 2

> CERTIFICATE_VERIFY_FAILED] 证书验证失败:无法获取本地颁发者证书

这个错误消息与站点需要客户端证书无关。相反,它是关于客户端(您的程序)无法验证服务器证书的问题。

虽然我们不了解服务器证书的具体情况,但可能是因为将预期的客户端证书用作 cafile 参数而引起的。但 cafile 用于指定用于服务器验证的受信任根 CA,而不是客户端证书。要指定客户端证书,您需要创建一个 SSL 上下文:

from urllib.request import urlopen
import ssl
ctx = ssl.create_default_context()
ctx.load_verify_locations('ca-which-signed-the-server-certificate.pem')
ctx.load_cert_chain('my-client-cert.pem', 'my-client-key.pem')
urlopen('https://needs-mtls.example.com', context=ctx)
英文:

> CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate

This error message is unrelated to the site requiring a client certificate. Instead it is about the client (your program) not being able to validate the server certificate.

While nothing about the server certificate is known it might be caused by using the expected client certificate as cafile argument. But cafile is for specifying the trusted root CA for server validation, not the client certificate. To specify this you have to create a SSL context:

from urllib.request import urlopen
import ssl
ctx = ssl.create_default_context()
ctx.load_verify_locations('ca-which-signed-the-server-certificate.pem')
ctx.load_cert_chain('my-client-cert.pem','my-client-key.pem')
urlopen('https://needs-mtls.example.com', context=ctx)

huangapple
  • 本文由 发表于 2023年6月13日 14:39:04
  • 转载请务必保留本文链接:https://go.coder-hub.com/76462239.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定