修复mongodump 4.2+(go语言)中的SSL证书处理问题。

huangapple go评论82阅读模式
英文:

Patch SSL Certificate handling in mongodump 4.2+ (go lang)

问题

我希望将我的Mongo数据库从版本4升级到5。我在Mongo中使用SSL配置,以确保与数据库的通信是加密的。为了备份数据库,我使用mongodump命令。

在Mongo 4.2中,mongodump被重写为Go语言,导致它引入了一个关于SSL证书处理的常见Go bug。具体来说,带有中间证书的PEM文件没有完全加载。该bug不会影响Mongo服务器、客户端本身、任何版本或任何其他应用程序,只会影响mongodump命令。

该bug在这里有描述:https://jira.mongodb.org/browse/TOOLS-2598

> 由于工具的SSL/TLS代码是从Go驱动程序复制的,当前实现仅解析pem文件中的最后一个证书

> 这与Mongoshell的行为不一致,后者只加载pem文件中的第一个证书。

在这个相关的交流中:https://jira.mongodb.org/browse/TOOLS-2996,我没有看到解决方案。我已经尝试了将密钥和证书以各种排列组合传递给mongodump命令。

我一直在查看mongodump的源代码,特别是SSL加载代码。

为了方便Go开发,我创建了这个Dockerfile,以便立即提供一个用于构建此代码的工作环境,但我对Go语言不熟悉。

FROM centos:8
RUN yum -y update
RUN yum -y install git-core vim-enhanced golang krb5-devel krb5-libs snappy
RUN groupadd -r app -g 1000 && \
    useradd -r -g app -u 1000 app -d /app && \
    mkdir -p /app && \
    chown -R app:app /app
USER 1000
WORKDIR /app
ENTRYPOINT /bin/bash

在这个代码库中修复这些PEM加载bug的可行性如何?我的Mongo数据库有很多客户端,因此为了解决这个问题而轮换证书需要进行大量的计划和停机时间。将mongodump修补程序以接受现有证书是否是一个可以接受的中期权衡?

是否有人能够帮助我编写适当的修补程序?也许现在有一些Go开发人员常用的标准SSL代码?请问有人对如何推进这个问题有任何想法吗?(最好是提供修补程序!)

因为我要测试的证书的创建方式比较复杂,所以很抱歉我没有一个可重现的测试案例。

英文:

I wish to upgrade my Mongo database from version 4 to 5. I use ssl configuration with Mongo to ensure communication with the database is encrypted. To backup the database I use mongodump.

In Mongo 4.2 mongodump was rewritten in Go lang, causing it to import a common Go bug around ssl certificate handling. Specifically PEM files with intermediate certificates aren't fully loaded. The bug does not impact Mongo server or client itself, any version, or any other apps. Only mongodump is impacted.

The bug is described here: https://jira.mongodb.org/browse/TOOLS-2598

> Since tool's SSL/TLS code is copied from Go driver, the current implementation only parses the last certificate inside the pem file

> This is a discrepancy with Mongoshell behavior, which only loads the first certificate inside the pem file.

In this related exchange: https://jira.mongodb.org/browse/TOOLS-2996 I fail to see the resolution. I have tried every permutation of keys and certificates in the arguments passed to mongodump.

I've been looking at the source code for mongodump and specifically the SSL loading code.

To aid in go development, I've created this dockerfile to instantly provide a working environment for building this code but I am unfamiliar with go as a language.

FROM centos:8
RUN yum -y update
RUN yum -y install git-core vim-enhanced golang krb5-devel krb5-libs snappy
RUN groupadd -r app -g 1000 && \
    useradd -r -g app -u 1000 app -d /app && \
    mkdir -p /app && \
    chown -R app:app /app
USER 1000
WORKDIR /app
ENTRYPOINT /bin/bash

How plausible is it to fix these PEM loading bugs in this code base? My Mongo estate has many clients so rotating the certificates to solve this issue involves a high degree of planning and downtime. Patching mongodump to accept the existing certs feels like an acceptable medium term trade off.

Is anyone able to help me write the appropriate patch, perhaps there's standard ssl code that go developers use now? Does anyone have any ideas on how I can move this forward please? (Patches would be ideal!)

I apologise in advance that I have no reproducible test case here due to complexities of how the certificates I want to test were created.

答案1

得分: 0

这个补丁是我写的(基于一个被拒绝的git拉取请求,用于GODRIVER-1753),解决了这些问题。

https://jira.mongodb.org/browse/GODRIVER-1753

https://github.com/mongodb/mongo-go-driver/pull/521/files

它还修复了一个asn1尾随数据的问题。


diff --git a/README.md b/README.md
index 20f3ffe8..4b3bed1a 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,13 @@
+
+** PATCHED MONGO-TOOLS **
+
+在Mongo 4.2中,Mongo团队用Go重写了许多核心工具。
+
+这引入了一种处理PEM文件的错误。
+面对大规模Mongo部署中的停机时间和复杂性,这个修补版本的mongo解决了这些问题。
+
+
MongoDB Tools

diff --git a/common/db/db.go b/common/db/db.go
index 3e78abab..9290bb31 100644
--- a/common/db/db.go
+++ b/common/db/db.go
@@ -186,9 +186,13 @@ func addClientCertFromBytes(cfg *tls.Config, data []byte, keyPasswd string) (str
}

            if currentBlock.Type == "CERTIFICATE" {
  •   	certBlock = data[start : len(data)-len(remaining)]
    
  •   	certDecodedBlock = currentBlock.Bytes
    
  •   	start += len(certBlock)
    
  •   	tempCertBlock := data[start : len(data)-len(remaining)]
    
  •   	certBlock = append(certBlock, tempCertBlock...) //处理多个证书的情况
    
  •   	tempCertEncodedBlock := currentBlock.Bytes
    
  •   	certDecodedBlock = append(certDecodedBlock, tempCertEncodedBlock...)
    
  •   	start += len(tempCertBlock)
              } else if strings.HasSuffix(currentBlock.Type, "PRIVATE KEY") {
                      isEncrypted := x509.IsEncryptedPEMBlock(currentBlock) || strings.Contains(currentBlock.Type, "ENCRYPTED PRIVATE KEY")
                      if isEncrypted {
    

@@ -244,12 +248,17 @@ func addClientCertFromBytes(cfg *tls.Config, data []byte, keyPasswd string) (str

    // tls.X509KeyPair的文档指出,Leaf证书不会被保留。
  • crt, err := x509.ParseCertificate(certDecodedBlock)
  • if err != nil {
  •   return "", err
    
  • }
  • return crt.Subject.String(), nil
  •    l := len(certDecodedBlock)
    
  •    for {
    
  •      crt, err := x509.ParseCertificate(certDecodedBlock[0:l])
    
  •      if err == nil {
    
  •        return crt.Subject.String(), nil
    
  •      }
    
  •      l = l - 1
    
  •      if l == 0 {
    
  •        return "", err
    
  •      }
    
  •    }
    

}

//从x509证书主题创建用于x509身份验证的用户名。
diff --git a/vendor/go.mongodb.org/mongo-driver/mongo/options/clientoptions.go b/vendor/go.mongodb.org/mongo-driver/mongo/options/clientoptions.go
index d66114fa..2f3c6554 100644
--- a/vendor/go.mongodb.org/mongo-driver/mongo/options/clientoptions.go
+++ b/vendor/go.mongodb.org/mongo-driver/mongo/options/clientoptions.go
@@ -941,9 +941,13 @@ func addClientCertFromBytes(cfg *tls.Config, data []byte, keyPasswd string) (str
}

            if currentBlock.Type == "CERTIFICATE" {
  •   	certBlock = data[start : len(data)-len(remaining)]
    
  •   	certDecodedBlock = currentBlock.Bytes
    
  •   	start += len(certBlock)
    
  •   	tempCertBlock := data[start : len(data)-len(remaining)]
    
  •   	certBlock = append(certBlock, tempCertBlock...) //处理多个证书的情况
    
  •   	tempCertEncodedBlock := currentBlock.Bytes
    
  •   	certDecodedBlock = append(certDecodedBlock, tempCertEncodedBlock...)
    
  •   	start += len(tempCertBlock)
              } else if strings.HasSuffix(currentBlock.Type, "PRIVATE KEY") {
                      isEncrypted := x509.IsEncryptedPEMBlock(currentBlock) || strings.Contains(currentBlock.Type, "ENCRYPTED PRIVATE KEY")
                      if isEncrypted {
    

@@ -997,12 +1001,18 @@ func addClientCertFromBytes(cfg *tls.Config, data []byte, keyPasswd string) (str

    // tls.X509KeyPair的文档指出,Leaf证书不会被保留。
  • crt, err := x509.ParseCertificate(certDecodedBlock)
  • if err != nil {
  •   return "", err
    
  • }
  •    l := len(certDecodedBlock)
    
  •    for {
    
  •      crt, err := x509.ParseCertificate(certDecodedBlock[0:l])
    
  •      if err == nil {
    
  •        return crt.Subject.String(), nil
    
  •      }
    
  •      l = l - 1
    
  •      if l == 0 {
    
  •        return "", err
    
  •      }
    
  •    }
    
  • return x509CertSubject(crt), nil
    }

func stringSliceContains(source []string, target string) bool {

英文:

This patch I've written (based on a rejected git pull request for GODRIVER-1753) resolves the issues.

https://jira.mongodb.org/browse/GODRIVER-1753

https://github.com/mongodb/mongo-go-driver/pull/521/files

It also fixes an asn1 trailing data issue too.


diff --git a/README.md b/README.md
index 20f3ffe8..4b3bed1a 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,13 @@
+
+** PATCHED MONGO-TOOLS **
+
+In Mongo 4.2, the mongo team rewrote many of their core tools in Go.
+
+This introduced a bug with the way that PEM files are handled.
+Faced with downtime and complexity in a large Mongo estate, this
+patched version of mongo resolves the issues.
+
+
 MongoDB Tools
 ===================================
 
diff --git a/common/db/db.go b/common/db/db.go
index 3e78abab..9290bb31 100644
--- a/common/db/db.go
+++ b/common/db/db.go
@@ -186,9 +186,13 @@ func addClientCertFromBytes(cfg *tls.Config, data []byte, keyPasswd string) (str
                }
 
                if currentBlock.Type == "CERTIFICATE" {
-			certBlock = data[start : len(data)-len(remaining)]
-			certDecodedBlock = currentBlock.Bytes
-			start += len(certBlock)
+			tempCertBlock := data[start : len(data)-len(remaining)]
+			certBlock = append(certBlock, tempCertBlock...) //To handle usecase where multiple certs are present
+
+			tempCertEncodedBlock := currentBlock.Bytes
+			certDecodedBlock = append(certDecodedBlock, tempCertEncodedBlock...)
+
+			start += len(tempCertBlock)
                } else if strings.HasSuffix(currentBlock.Type, "PRIVATE KEY") {
                        isEncrypted := x509.IsEncryptedPEMBlock(currentBlock) || strings.Contains(currentBlock.Type, "ENCRYPTED PRIVATE KEY")
                        if isEncrypted {
@@ -244,12 +248,17 @@ func addClientCertFromBytes(cfg *tls.Config, data []byte, keyPasswd string) (str
 
        // The documentation for the tls.X509KeyPair indicates that the Leaf certificate is not
        // retained.
-	crt, err := x509.ParseCertificate(certDecodedBlock)
-	if err != nil {
-		return "", err
-	}
-
-	return crt.Subject.String(), nil
+        l := len(certDecodedBlock)
+        for {
+          crt, err := x509.ParseCertificate(certDecodedBlock[0:l])
+          if err == nil {
+            return crt.Subject.String(), nil
+          }
+          l = l - 1
+          if l == 0 {
+            return "", err
+          }
+        }
 }
 
 // create a username for x509 authentication from an x509 certificate subject.
diff --git a/vendor/go.mongodb.org/mongo-driver/mongo/options/clientoptions.go b/vendor/go.mongodb.org/mongo-driver/mongo/options/clientoptions.go
index d66114fa..2f3c6554 100644
--- a/vendor/go.mongodb.org/mongo-driver/mongo/options/clientoptions.go
+++ b/vendor/go.mongodb.org/mongo-driver/mongo/options/clientoptions.go
@@ -941,9 +941,13 @@ func addClientCertFromBytes(cfg *tls.Config, data []byte, keyPasswd string) (str
                }
 
                if currentBlock.Type == "CERTIFICATE" {
-			certBlock = data[start : len(data)-len(remaining)]
-			certDecodedBlock = currentBlock.Bytes
-			start += len(certBlock)
+			tempCertBlock := data[start : len(data)-len(remaining)]
+			certBlock = append(certBlock, tempCertBlock...) //To handle usecase where multiple certs are present
+
+			tempCertEncodedBlock := currentBlock.Bytes
+			certDecodedBlock = append(certDecodedBlock, tempCertEncodedBlock...)
+
+			start += len(tempCertBlock)
                } else if strings.HasSuffix(currentBlock.Type, "PRIVATE KEY") {
                        isEncrypted := x509.IsEncryptedPEMBlock(currentBlock) || strings.Contains(currentBlock.Type, "ENCRYPTED PRIVATE KEY")
                        if isEncrypted {
@@ -997,12 +1001,18 @@ func addClientCertFromBytes(cfg *tls.Config, data []byte, keyPasswd string) (str
 
        // The documentation for the tls.X509KeyPair indicates that the Leaf certificate is not
        // retained.
-	crt, err := x509.ParseCertificate(certDecodedBlock)
-	if err != nil {
-		return "", err
-	}
+        l := len(certDecodedBlock)
+        for {
+          crt, err := x509.ParseCertificate(certDecodedBlock[0:l])
+          if err == nil {
+            return crt.Subject.String(), nil
+          }
+          l = l - 1
+          if l == 0 {
+            return "", err
+          }
+        }
 
-	return x509CertSubject(crt), nil
 }
 
 func stringSliceContains(source []string, target string) bool {

huangapple
  • 本文由 发表于 2022年1月31日 17:48:42
  • 转载请务必保留本文链接:https://go.coder-hub.com/70923942.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定