通过TCP压缩和传输文件(使用Golang)

huangapple go评论68阅读模式
英文:

Compress and transfer file via TCP (Golang)

问题

我写了一个简单的示例代码,它可以工作,但是接收到的文件大小没有被压缩。

我的客户端(用于连接服务器和发送文件):

// 连接服务器
conn, err := net.Dial("tcp", serverAddr)
CheckError(err)
defer conn.Close()

in, err := os.Open(srcFile)
if err != nil {
    log.Fatal(err)
}
pr, pw := io.Pipe()
gw, err := gzip.NewWriterLevel(pw, 7)
CheckError(err)
go func() {
    n, err := io.Copy(gw, in)
    gw.Close()
    pw.Close()
    log.Printf("copied    %v %v", n, err)
}()
// 可能在下面会出错?
_, err = io.Copy(conn, pr)

请帮忙,如何正确使用管道(pipe)和拷贝(copy)函数?

英文:

I write simple example code, it worked, but size of file that recived is not compressed

My client (for connect to server and send file):

	// connect to server
	conn, err := net.Dial("tcp", serverAddr)
	CheckError(err)
	defer conn.Close()

	in, err := os.Open(srcFile)
	if err != nil {
		log.Fatal(err)
	}
	pr, pw := io.Pipe()
	gw, err := gzip.NewWriterLevel(pw, 7)
	CheckError(err)
	go func() {
		n, err := io.Copy(gw, in)
		gw.Close()
		pw.Close()
		log.Printf("copied    %v %v", n, err)
	}()
        //maybe error some next?
	_, err = io.Copy(conn, pr)

Please, help, how right to use pipe with copy

答案1

得分: 3

如我在评论中已经说过的那样,你的代码是有效的。我创建了一个小例子来测试或查看是否可以解决你的问题。所以我猜你可以关闭这个问题。

package main

import (
	"compress/gzip"
	"io"
	"log"
	"net"
	"os"
)

func main() {
	// 在一个随机端口上创建监听器。
	listener, err := net.Listen("tcp", "127.0.0.1:")
	if err != nil {
		log.Fatal(err)
	}
	log.Println("Server listening on: " + listener.Addr().String())
	done := make(chan struct{})
	go func() {
		defer func() { done <- struct{}{} }()
		for {
			conn, err := listener.Accept()
			if err != nil {
				log.Println(err)
				return
			}
			go func(c net.Conn) {
				defer func() {
					c.Close()
					done <- struct{}{}
				}()
				buf := make([]byte, 1024)
				for {
					n, err := c.Read(buf)
					if err != nil {
						if err != io.EOF {
							log.Println(err)
						}

						return
					}
					log.Printf("received: %q", buf[:n])
					log.Printf("bytes: %d", n)

				}

			}(conn)
		}
	}()

	conn, err := net.Dial("tcp", listener.Addr().String())
	if err != nil {
		log.Fatal(err)
	}
	log.Println("Connected to server.")

	file, err := os.Open("./file.txt")
	if err != nil {
		log.Fatal(err)
	}

	pr, pw := io.Pipe()
	w, err := gzip.NewWriterLevel(pw, 7)
	if err != nil {
		log.Fatal(err)
	}
	go func() {
		n, err := io.Copy(w, file)
		if err != nil {
			log.Fatal(err)
		}
		w.Close()
		pw.Close()
		log.Printf("copied to piped writer via the compressed writer: %d", n)
	}()

	n, err := io.Copy(conn, pr)
	if err != nil {
		log.Fatal(err)
	}
	log.Printf("copied to connection: %d", n)

	conn.Close()
	<-done
	listener.Close()
	<-done
}

使用一个简单的文本文件作为输入,其中包含许多重复的字符,以便进行压缩,该程序的输出为:文件大小为153字节,发送/接收了46字节。

2022/04/04 11:23:58 Server listening on: 127.0.0.1:58250
2022/04/04 11:23:58 Connected to server.
2022/04/04 11:23:58 received: "\x1f\x8b\b\x00\x00\x00\x00\x00\x00\xff"
2022/04/04 11:23:58 bytes: 10
2022/04/04 11:23:58 copied to piped writer via the compressed writer: 153
2022/04/04 11:23:58 copied to connection: 46
2022/04/04 11:23:58 received: "*I-.I,NI,N\xc1\x01\x8aS\x8a\x13i\bx\xb9pX    \r\b\x00\x00\xff\xff\xc7\xfe\xa6c\x99\x00\x00\x00"
2022/04/04 11:23:58 bytes: 36
2022/04/04 11:23:58 accept tcp 127.0.0.1:58250: use of closed network connection
英文:

As I already said in the comment, your code works. I created a little example to test or see if I can solve your problem. So I guess you can close this question.

package main
import (
&quot;compress/gzip&quot;
&quot;io&quot;
&quot;log&quot;
&quot;net&quot;
&quot;os&quot;
)
func main() {
// Create a listener on a random port.
listener, err := net.Listen(&quot;tcp&quot;, &quot;127.0.0.1:&quot;)
if err != nil {
log.Fatal(err)
}
log.Println(&quot;Server listening on: &quot; + listener.Addr().String())
done := make(chan struct{})
go func() {
defer func() { done &lt;- struct{}{} }()
for {
conn, err := listener.Accept()
if err != nil {
log.Println(err)
return
}
go func(c net.Conn) {
defer func() {
c.Close()
done &lt;- struct{}{}
}()
buf := make([]byte, 1024)
for {
n, err := c.Read(buf)
if err != nil {
if err != io.EOF {
log.Println(err)
}
return
}
log.Printf(&quot;received: %q&quot;, buf[:n])
log.Printf(&quot;bytes: %d&quot;, n)
}
}(conn)
}
}()
conn, err := net.Dial(&quot;tcp&quot;, listener.Addr().String())
if err != nil {
log.Fatal(err)
}
log.Println(&quot;Connected to server.&quot;)
file, err := os.Open(&quot;./file.txt&quot;)
if err != nil {
log.Fatal(err)
}
pr, pw := io.Pipe()
w, err := gzip.NewWriterLevel(pw, 7)
if err != nil {
log.Fatal(err)
}
go func() {
n, err := io.Copy(w, file)
if err != nil {
log.Fatal(err)
}
w.Close()
pw.Close()
log.Printf(&quot;copied to piped writer via the compressed writer: %d&quot;, n)
}()
n, err := io.Copy(conn, pr)
if err != nil {
log.Fatal(err)
}
log.Printf(&quot;copied to connection: %d&quot;, n)
conn.Close()
&lt;-done
listener.Close()
&lt;-done
}

The output of that program with a simple text file with many repeated characters in it, to have something to compress: The file is 153 bytes and I send/received 46 bytes

2022/04/04 11:23:58 Server listening on: 127.0.0.1:58250
2022/04/04 11:23:58 Connected to server.
2022/04/04 11:23:58 received: &quot;\x1f\x8b\b\x00\x00\x00\x00\x00\x00\xff&quot;
2022/04/04 11:23:58 bytes: 10
2022/04/04 11:23:58 copied to piped writer via the compressed writer: 153
2022/04/04 11:23:58 copied to connection: 46
2022/04/04 11:23:58 received: &quot;*I-.I,NI,N\xc1\x01\x8aS\x8a\x13i\bx\xb9pX    \r\b\x00\x00\xff\xff\xc7\xfe\xa6c\x99\x00\x00\x00&quot;
2022/04/04 11:23:58 bytes: 36
2022/04/04 11:23:58 accept tcp 127.0.0.1:58250: use of closed network connection

huangapple
  • 本文由 发表于 2022年4月2日 09:22:54
  • 转载请务必保留本文链接:https://go.coder-hub.com/71714183.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定