英文:
Email subject, header decoding in different charset like ISO-2022-JP, GB-2312, etc
问题
我正在处理一个项目,需要处理不同字符集的电子邮件编码/解码。以下是一个用Python编写的示例代码:
from email.header import Header, decode_header, make_header
from charset import text_to_utf8
class ....
def decode_header(self, header):
decoded_header = decode_header(header)
if decoded_header[0][1] is None:
return text_to_utf8(decoded_header[0][0]).decode("utf-8", "replace")
else:
return decoded_header[0][0].decode(decoded_header[0][1].replace("windows-", "cp"), "replace")
基本上,对于像"=?iso-2022-jp?b?GyRCRW1CQE86GyhCIDxtb21vQHRhcm8ubmUuanA=?="这样的文本,"decode_header"函数会尝试找到编码方式:"iso-2022-jp",然后使用"decode"函数将字符集解码为Unicode。
现在,在Go语言中,我可以做类似的事情:
import "mime"
dec := new(mime.WordDecoder)
text := "=?utf-8?q?=C3=89ric?= <eric@example.org>, =?utf-8?q?Ana=C3=AFs?= <anais@example.org>"
header, err := dec.DecodeHeader(text)
似乎mime.WordDecoder允许设置一个字符集解码器的"hook":
type WordDecoder struct {
// CharsetReader, if non-nil, defines a function to generate
// charset-conversion readers, converting from the provided
// charset into UTF-8.
// Charsets are always lower-case. utf-8, iso-8859-1 and us-ascii charsets
// are handled by default.
// One of the the CharsetReader's result values must be non-nil.
CharsetReader func(charset string, input io.Reader) (io.Reader, error)
}
我想知道是否有任何库可以像Python中的"decode"函数一样,允许我转换任意字符集,就像上面的示例一样。我不想编写一个像mime/encodedword.go中使用的那样庞大的"switch-case"语句:
func (d *WordDecoder) convert(buf *bytes.Buffer, charset string, content []byte) error {
switch {
case strings.EqualFold("utf-8", charset):
buf.Write(content)
case strings.EqualFold("iso-8859-1", charset):
for _, c := range content {
buf.WriteRune(rune(c))
}
...
非常感谢您的帮助。
英文:
I am working on a project which needs to deal with email encoding/decoding in different charsets. A python code for this can be shown in the below:
from email.header import Header, decode_header, make_header
from charset import text_to_utf8
class ....
def decode_header(self, header):
decoded_header = decode_header(header)
if decoded_header[0][1] is None:
return text_to_utf8(decoded_header[0][0]).decode("utf-8", "replace")
else:
return decoded_header[0][0].decode(decoded_header[0][1].replace("windows-", "cp"), "replace")
Basically, for the text like "=?iso-2022-jp?b?GyRCRW1CQE86GyhCIDxtb21vQHRhcm8ubmUuanA=?="; the "decode_header" function just tries to find the encoding: 'iso-2022-jp'; then it will use the "decode" function to decode the charset to unicode.
Now, in go, i can do something similar to like:
import "mime"
dec := new(mime.WordDecoder)
text := "=?utf-8?q?=C3=89ric?= <eric@example.org>, =?utf-8?q?Ana=C3=AFs?= <anais@example.org>"
header, err := dec.DecodeHeader(text)
Seems that there mime.WordDecoder allow to put a charset decoder "hook":
type WordDecoder struct {
// CharsetReader, if non-nil, defines a function to generate
// charset-conversion readers, converting from the provided
// charset into UTF-8.
// Charsets are always lower-case. utf-8, iso-8859-1 and us-ascii charsets
// are handled by default.
// One of the the CharsetReader's result values must be non-nil.
CharsetReader func(charset string, input io.Reader) (io.Reader, error)
}
I am wondering is there any library which can allow me to convert arbitrary charset like the "decode" function in python as shown in the above example. I don't want to write a big "switch-case"like the one being used in mime/encodedword.go:
func (d *WordDecoder) convert(buf *bytes.Buffer, charset string, content []byte) error {
switch {
case strings.EqualFold("utf-8", charset):
buf.Write(content)
case strings.EqualFold("iso-8859-1", charset):
for _, c := range content {
buf.WriteRune(rune(c))
}
....
Any help would be very appreciated.
Thanks.
答案1
得分: 1
谢谢。看起来包golang.org/x/net/html/charset已经提供了一个包含可用编码的映射。以下代码适用于我:
import "golang.org/x/net/html/charset"
CharsetReader := func (label string, input io.Reader) (io.Reader, error) {
label = strings.Replace(label, "windows-", "cp", -1)
encoding, _ := charset.Lookup(label)
return encoding.NewDecoder().Reader(input), nil
}
dec := mime.WordDecoder{CharsetReader: CharsetReader}
text := "=?iso-2022-jp?b?GyRCRW1CQE86GyhCIDxtb21vQHRhcm8ubmUuanA=?="
header, err := dec.DecodeHeader(text)
感谢您的帮助!
英文:
Thanks. It seems that the package golang.org/x/net/html/charset already provided a map with available encoding. The following code works for me:
import "golang.org/x/net/html/charset"
CharsetReader := func (label string, input io.Reader) (io.Reader, error) {
label = strings.Replace(label, "windows-", "cp", -1)
encoding, _ := charset.Lookup(label)
return encoding.NewDecoder().Reader(input), nil
}
dec := mime.WordDecoder{CharsetReader: CharsetReader}
text := "=?iso-2022-jp?b?GyRCRW1CQE86GyhCIDxtb21vQHRhcm8ubmUuanA=?="
header, err := dec.DecodeHeader(text)
Thanks for your help!
答案2
得分: 0
我不确定这是否是你要找的,但有一个golang.org/x/text
包,我正在使用它将Windows-1251转换为UTF-8。代码如下:
import (
"golang.org/x/text/encoding/charmap"
"golang.org/x/text/transform"
"io/ioutil"
"strings"
)
func convert(s string) string {
sr := strings.NewReader(s)
tr := transform.NewReader(sr, charmap.Windows1251.NewDecoder())
buf, err := ioutil.ReadAll(tr)
if err != nil {
return ""
}
return string(buf)
}
我认为在你的情况下,如果你想避免使用"一个大的'switch-case'",你可以创建一个包含所有可用编码的映射,并做如下操作:
var encodings = map[string]transform.Transformer{
"win-1251": charmap.Windows1251.NewDecoder(),
}
func convert(s, charset string) string {
buf, err := ioutil.ReadAll(transform.NewReader(strings.NewReader(s), encodings[charset]))
if err != nil {
return ""
}
return string(buf)
}
以上是翻译好的内容,请确认是否满意。
英文:
I'm not sure it is what you are looking for but there is golang.org/x/text
package which I'm using to convert Windows-1251 to UTF-8. Code looks like
import (
"golang.org/x/text/encoding/charmap"
"golang.org/x/text/transform"
"io/ioutil"
"strings"
)
func convert(s string) string {
sr := strings.NewReader(s)
tr := transform.NewReader(sr, charmap.Windows1251.NewDecoder())
buf, err := ioutil.ReadAll(tr)
if err != nil {
return ""
}
return string(buf)
}
I think in your case if you want to avoid "a big 'switch-case'" you can create kind of map with full list of available encodings and just make something like:
var encodings = map[string]transform.Transformer{
"win-1251": charmap.Windows1251.NewDecoder(),
}
func convert(s, charset string) string {
buf, err := ioutil.ReadAll(transform.NewReader(strings.NewReader(s), encodings[charset]))
if err != nil {
return ""
}
return string(buf)
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论