英文:
How to pass compressed bytes through channel?
问题
我正在尝试从缓冲读取器压缩文件,并通过字节通道传递压缩字节,但结果很差 :), 这是我目前想到的,显然这不起作用...
func Compress(r io.Reader) (<-chan byte) {
c := make(chan byte)
go func(){
var wBuff bytes.Buffer
rBuff := make([]byte, 1024)
writer := zlib.NewWriter(*wBuff)
for {
n, err := r.Read(rBuff)
if err != nil && err != io.EOF { panic(err) }
if n == 0 { break }
writer.Write(rBuff) // 压缩并写入压缩数据
// 如何通过通道发送已写入的压缩字节?
// 据我了解,wBuff 最终将包含整个压缩数据?
}
writer.Close()
close(c) // 表示没有更多的数据
}()
return c
}
请谅解,因为我对Go语言非常陌生。
英文:
I'm trying to compress file from buffered reader and pass compressed bytes through byte channel, but with poor results :), here's what I came up till now, obviously this don't works...
func Compress(r io.Reader) (<-chan byte) {
c := make(chan byte)
go func(){
var wBuff bytes.Buffer
rBuff := make([]byte, 1024)
writer := zlib.NewWriter(*wBuff)
for {
n, err := r.Read(rBuff)
if err != nil && err != io.EOF { panic(err) }
if n == 0 { break }
writer.Write(rBuff) // Compress and write compressed data
// How to send written compressed bytes through channel?
// as fas as I understand wBuff will eventually contain
// whole compressed data?
}
writer.Close()
close(c) // Indicate that no more data follows
}()
return c
}
Please bear with me, as I'm very new to Go
答案1
得分: 3
我建议使用[]byte
而不是byte
。这样更高效。由于并发内存访问,可能需要通过通道发送缓冲区的副本,而不是发送[]byte
缓冲区本身。
您可以定义一个type ChanWriter chan []byte
,并让它实现io.Writer
接口。然后将ChanWriter
传递给zlib.NewWriter
。
您可以为压缩操作创建一个goroutine,然后立即从Compress
函数返回ChanWriter
的通道。如果没有goroutine,则函数没有理由返回一个通道,首选的返回类型是io.Reader
。
Compress
函数的返回类型应该更改为类似chan <- BytesWithError
的东西。在这种情况下,ChanWriter
可以定义为type ChanWriter chan BytesWithError
。
英文:
I suggest to use []byte
instead of byte
. It is more efficient. Because of concurrent memory accesses it may be necessary to send a copy of the buffer through the channel rather than sending the []byte
buffer itself.
You can define a type ChanWriter chan []byte
and let it implement the io.Writer
interface. Then pass the ChanWriter
to zlib.NewWriter
.
You can create a goroutine for doing the compression and then immediately return the ChanWriter
's channel from your Compress
function. If there is no goroutine then there is no reason for the function to return a channel and the preferred return type is io.Reader
.
The return type of the Compress
function should be changed into something like chan <-BytesWithError
. In this case ChanWriter
can be defined as type ChanWriter chan BytesWithError
.
答案2
得分: 2
通过一个通道逐个发送字节并不是特别高效的。另一种更有用的方法是返回一个实现了io.Reader
接口的对象,通过从原始的io.Reader
读取一个块并在返回之前压缩其输出来实现Read()
方法。
英文:
Sending bytes one by one down a channel is not going to be particularly efficient. Another approach that may be more useful would be to return an object implementing the io.Reader
interface, implementing the Read()
method by reading a block from a original io.Reader
and compressing its output before returning it.
答案3
得分: 1
你的writer.Write(rBuff)
语句总是写入len(rBuff)
个字节,即使n != len(rBuff)
。
writer.Write(rBuff[:n])
此外,你的Read
循环是
for {
n, err := r.Read(rBuff)
if err != nil && err != io.EOF {
panic(err)
}
if n == 0 {
break
}
writer.Write(rBuff[:n])
// ...
}
等同于
for {
n, err := r.Read(rBuff)
if err != nil && err != io.EOF {
panic(err)
}
// !(err != nil && err != io.EOF)
// !(err != nil) || !(err != io.EOF)
// err == nil || err == io.EOF
if err == nil || err == io.EOF {
if n == 0 {
break
}
}
writer.Write(rBuff[:n])
// ...
}
循环在err == nil && n == 0
时提前退出。
相反,写成
for {
n, err := r.Read(rBuf)
if err != nil {
if err != io.EOF {
panic(err)
}
if n == 0 {
break
}
}
writer.Write(rBuf[:n])
// ...
}
英文:
Your writer.Write(rBuff)
statement always writes len(rBuff)
bytes, even when n != len(rBuff)
.
writer.Write(rBuff[:n])
Also, your Read
loop is
for {
n, err := r.Read(rBuff)
if err != nil && err != io.EOF {
panic(err)
}
if n == 0 {
break
}
writer.Write(rBuff[:n])
// ...
}
which is equivalent to
for {
n, err := r.Read(rBuff)
if err != nil && err != io.EOF {
panic(err)
}
// !(err != nil && err != io.EOF)
// !(err != nil) || !(err != io.EOF)
// err == nil || err == io.EOF
if err == nil || err == io.EOF {
if n == 0 {
break
}
}
writer.Write(rBuff[:n])
// ...
}
The loop exits prematurely if err == nil && if n == 0
.
Instead, write
for {
n, err := r.Read(rBuf)
if err != nil {
if err != io.EOF {
panic(err)
}
if n == 0 {
break
}
}
writer.Write(rBuf[:n])
// ...
}
答案4
得分: 0
好的,我找到了一个可行的解决方案:(请随时指出可以改进的地方,或者我做错了什么?)
func Compress(r io.Reader) (<-chan byte) {
c := make(chan byte)
go func(){
var wBuff bytes.Buffer
rBuff := make([]byte, 1024)
writer := zlib.NewWriter(&wBuff)
for {
n, err := r.Read(rBuff)
if err != nil {
if err != io.EOF {
panic(err)
}
if n == 0 {
break
}
}
writer.Write(rBuff[:n])
for _, v := range wBuff.Bytes() {
c <- v
}
wBuff.Truncate(0)
}
writer.Close()
for _, v := range wBuff.Bytes() {
c <- v
}
close(c) // 表示没有更多的数据跟随
}()
return c
}
英文:
Ok, I've found working solution: (Feel free to indicate where it can be improved, or maybe I'm doing something wrong?)
func Compress(r io.Reader) (<-chan byte) {
c := make(chan byte)
go func(){
var wBuff bytes.Buffer
rBuff := make([]byte, 1024)
writer := zlib.NewWriter(&wBuff)
for {
n, err := r.Read(rBuff)
if err != nil {
if err != io.EOF {
panic(err)
}
if n == 0 {
break
}
}
writer.Write(rBuff[:n])
for _, v := range wBuff.Bytes() {
c <- v
}
wBuff.Truncate(0)
}
writer.Close()
for _, v := range wBuff.Bytes() {
c <- v
}
close(c) // Indicate that no more data follows
}()
return c
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论