电子邮件主题,不同字符集的标题解码,如ISO-2022-JP,GB-2312等。

huangapple go评论141阅读模式
英文:

Email subject, header decoding in different charset like ISO-2022-JP, GB-2312, etc

问题

我正在处理一个项目,需要处理不同字符集的电子邮件编码/解码。以下是一个用Python编写的示例代码:

from email.header import Header, decode_header, make_header
from charset import text_to_utf8

class ....
def decode_header(self, header):
    decoded_header = decode_header(header)

    if decoded_header[0][1] is None:
        return text_to_utf8(decoded_header[0][0]).decode("utf-8", "replace")
    else:
        return decoded_header[0][0].decode(decoded_header[0][1].replace("windows-", "cp"), "replace")

基本上,对于像"=?iso-2022-jp?b?GyRCRW1CQE86GyhCIDxtb21vQHRhcm8ubmUuanA=?="这样的文本,"decode_header"函数会尝试找到编码方式:"iso-2022-jp",然后使用"decode"函数将字符集解码为Unicode。

现在,在Go语言中,我可以做类似的事情:

import "mime"

dec := new(mime.WordDecoder)
text := "=?utf-8?q?=C3=89ric?= <eric@example.org>, =?utf-8?q?Ana=C3=AFs?= <anais@example.org>"
header, err := dec.DecodeHeader(text)

似乎mime.WordDecoder允许设置一个字符集解码器的"hook":

type WordDecoder struct {
   // CharsetReader, if non-nil, defines a function to generate
   // charset-conversion readers, converting from the provided
   // charset into UTF-8.
   // Charsets are always lower-case. utf-8, iso-8859-1 and us-ascii charsets
   // are handled by default.
   // One of the the CharsetReader's result values must be non-nil.
   CharsetReader func(charset string, input io.Reader) (io.Reader, error)
}

我想知道是否有任何库可以像Python中的"decode"函数一样,允许我转换任意字符集,就像上面的示例一样。我不想编写一个像mime/encodedword.go中使用的那样庞大的"switch-case"语句:

func (d *WordDecoder) convert(buf *bytes.Buffer, charset string, content []byte) error {
   switch {
   case strings.EqualFold("utf-8", charset):
      buf.Write(content)
   case strings.EqualFold("iso-8859-1", charset):
      for _, c := range content {
         buf.WriteRune(rune(c))
      }
   ...

非常感谢您的帮助。

英文:

I am working on a project which needs to deal with email encoding/decoding in different charsets. A python code for this can be shown in the below:

from email.header import Header, decode_header, make_header
from charset import text_to_utf8	

class ....
def decode_header(self, header):
    decoded_header = decode_header(header)

    if decoded_header[0][1] is None:
        return text_to_utf8(decoded_header[0][0]).decode(&quot;utf-8&quot;, &quot;replace&quot;)
    else:
        return decoded_header[0][0].decode(decoded_header[0][1].replace(&quot;windows-&quot;, &quot;cp&quot;), &quot;replace&quot;)

Basically, for the text like "=?iso-2022-jp?b?GyRCRW1CQE86GyhCIDxtb21vQHRhcm8ubmUuanA=?="; the "decode_header" function just tries to find the encoding: 'iso-2022-jp'; then it will use the "decode" function to decode the charset to unicode.

Now, in go, i can do something similar to like:

import &quot;mime&quot;

dec := new(mime.WordDecoder)
text := &quot;=?utf-8?q?=C3=89ric?= &lt;eric@example.org&gt;, =?utf-8?q?Ana=C3=AFs?= &lt;anais@example.org&gt;&quot;
header, err := dec.DecodeHeader(text)

Seems that there mime.WordDecoder allow to put a charset decoder &quot;hook&quot;: 
type WordDecoder struct {
   // CharsetReader, if non-nil, defines a function to generate
   // charset-conversion readers, converting from the provided
   // charset into UTF-8.
   // Charsets are always lower-case. utf-8, iso-8859-1 and us-ascii charsets
   // are handled by default.
   // One of the the CharsetReader&#39;s result values must be non-nil.
   CharsetReader func(charset string, input io.Reader) (io.Reader, error)
}	        

I am wondering is there any library which can allow me to convert arbitrary charset like the "decode" function in python as shown in the above example. I don't want to write a big "switch-case"like the one being used in mime/encodedword.go:

func (d *WordDecoder) convert(buf *bytes.Buffer, charset string, content []byte) error {
   switch {
   case strings.EqualFold(&quot;utf-8&quot;, charset):
      buf.Write(content)
   case strings.EqualFold(&quot;iso-8859-1&quot;, charset):
      for _, c := range content {
         buf.WriteRune(rune(c))
      }
....

Any help would be very appreciated.

Thanks.

答案1

得分: 1

谢谢。看起来包golang.org/x/net/html/charset已经提供了一个包含可用编码的映射。以下代码适用于我:

import "golang.org/x/net/html/charset"

CharsetReader := func (label string, input io.Reader) (io.Reader, error) {
    label = strings.Replace(label, "windows-", "cp", -1)
    encoding, _ := charset.Lookup(label)
    return encoding.NewDecoder().Reader(input), nil
}
dec := mime.WordDecoder{CharsetReader: CharsetReader}
text := "=?iso-2022-jp?b?GyRCRW1CQE86GyhCIDxtb21vQHRhcm8ubmUuanA=?="
header, err := dec.DecodeHeader(text)

感谢您的帮助!

英文:

Thanks. It seems that the package golang.org/x/net/html/charset already provided a map with available encoding. The following code works for me:

import &quot;golang.org/x/net/html/charset&quot;

CharsetReader := func (label string, input io.Reader) (io.Reader, error) {
	label = strings.Replace(label, &quot;windows-&quot;, &quot;cp&quot;, -1)
	encoding, _ := charset.Lookup(label)
	return encoding.NewDecoder().Reader(input), nil
}
dec := mime.WordDecoder{CharsetReader: CharsetReader}
text := &quot;=?iso-2022-jp?b?GyRCRW1CQE86GyhCIDxtb21vQHRhcm8ubmUuanA=?=&quot;
header, err := dec.DecodeHeader(text)

Thanks for your help!

答案2

得分: 0

我不确定这是否是你要找的,但有一个golang.org/x/text包,我正在使用它将Windows-1251转换为UTF-8。代码如下:

import (
	"golang.org/x/text/encoding/charmap"
	"golang.org/x/text/transform"
	"io/ioutil"
	"strings"
)

func convert(s string) string {
	sr := strings.NewReader(s)
	tr := transform.NewReader(sr, charmap.Windows1251.NewDecoder())
	buf, err := ioutil.ReadAll(tr)
	if err != nil {
		return ""
	}
	return string(buf)
}

我认为在你的情况下,如果你想避免使用"一个大的'switch-case'",你可以创建一个包含所有可用编码的映射,并做如下操作:

var encodings = map[string]transform.Transformer{
	"win-1251": charmap.Windows1251.NewDecoder(),
}

func convert(s, charset string) string {
	buf, err := ioutil.ReadAll(transform.NewReader(strings.NewReader(s), encodings[charset]))
	if err != nil {
		return ""
	}
	return string(buf)
}

以上是翻译好的内容,请确认是否满意。

英文:

I'm not sure it is what you are looking for but there is golang.org/x/text package which I'm using to convert Windows-1251 to UTF-8. Code looks like

import (
	&quot;golang.org/x/text/encoding/charmap&quot;
	&quot;golang.org/x/text/transform&quot;
	&quot;io/ioutil&quot;
	&quot;strings&quot;
)

func convert(s string) string {
	sr := strings.NewReader(s)
	tr := transform.NewReader(sr, charmap.Windows1251.NewDecoder())
	buf, err := ioutil.ReadAll(tr)
	if err != nil {
		return &quot;&quot;
	}
	return string(buf)
}

I think in your case if you want to avoid "a big 'switch-case'" you can create kind of map with full list of available encodings and just make something like:

var encodings = map[string]transform.Transformer{
	&quot;win-1251&quot;: charmap.Windows1251.NewDecoder(),
}

func convert(s, charset string) string {
	buf, err := ioutil.ReadAll(transform.NewReader(strings.NewReader(s), encodings[charset]))
	if err != nil {
		return &quot;&quot;
	}
	return string(buf)
}

huangapple
  • 本文由 发表于 2016年1月30日 10:30:23
  • 转载请务必保留本文链接:https://go.coder-hub.com/35097318.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定