Golang的TCP套接字读取最终会返回EOF。

huangapple go评论76阅读模式
英文:

Golang tcp socket read gives EOF eventually

问题

我有问题从套接字中读取数据。有一个运行着大量呼叫(每分钟10-60个呼叫)的星号实例,并且我正在尝试读取和处理与这些呼叫相关的CDR事件(连接到AMI)。

这是我正在使用的库(不是我的,但由于错误而被推送到分支)https://github.com/warik/gami

它非常简单,主要的操作在gami.go的readDispatcher中进行。

buf := make([]byte, _READ_BUF)    // 读取缓冲区
    for {
	    rc, err := (*a.conn).Read(buf)

所以,有一个TCPConn(a.conn)和大小为1024的缓冲区,我从套接字中读取消息。到目前为止还好,但是最终,不时地(这个时间可能从10分钟到5小时不等,与通过套接字传输的数据量无关)读取操作会失败,并出现io.EOF错误。我尝试立即重新连接和重新登录,但这也是不可能的-连接超时,所以我被迫等待大约40-60秒,而这个时间对我来说非常关键,因为延迟导致我丢失了很多数据。我进行了谷歌搜索,阅读了源代码并尝试了很多方法-没有任何结果。最奇怪的是,用Python或PHP打开的简单套接字不会失败。

  • 是否可能是因为我的机器或星号服务器上缺少表示套接字的文件描述符而导致的问题?
  • 是否可能是星号配置的问题(因为我有另一个星号,这个问题不会复现,但是在最后一个星号上的呼叫时间也较短)?
  • 是否可能是我处理套接字连接或Go的方式的问题?

go版本go1.2.1 linux/amd64

星号1.8

英文:

I have problem reading from socket. There is asterisk instance running with plenty of calls (10-60 in a minute) and I'm trying to read and process CDR events related to those calls (connected to AMI).

Here is library which I'm using (not mine, but was pushed to fork because of bugs) https://github.com/warik/gami

Its pretty straightforward, main action goes in gami.go - readDispatcher.

buf := make([]byte, _READ_BUF)    // read buffer
    for {
	    rc, err := (*a.conn).Read(buf)

So, there is TCPConn (a.conn) and buffer with size 1024 to which I'm reading messages from socket. So far so good, but eventually, from time to time (this time may vary from 10 minutes to 5 hours independently of data amount which comes through socket) Read operation fails with io.EOF error. I was trying to reconnect and relogin immediately, but its also impossible - connection times out, so i was pushed to wait for about 40-60sec, and this time is very crucial to me, I'm losing a lot of data because of delay. I was googling, reading sources and trying a lot of stuff - nothing. The most strange thing, that simple socket opened in python or php does not fail.

  • Is it possible that problem because of lack of file descriptors to represent socket on mine machine or on asterisk server?
  • Is it possible that problem in asterisk configuration (because i have another asterisk on which this problem doesn't reproduce, but also, i have time less calls on last one)?
  • Is it possible that problem in my way to deal with socket connection or with Go in general?

go version go1.2.1 linux/amd64

asterisk 1.8

答案1

得分: 1

更新到最新的Asterisk版本。当AMI发送大量数据时,可能会出现类似的错误。

要检查问题,您需要通过AMI命令发送类似于“COMMAND sip show peers”(或任何其他长输出命令),并查看结果。

英文:

Update to latest asterisk. There was bug like that when AMI send alot of data.

For check issue, you have send via ami command like "COMMAND sip show peers"(or any other long output command) and see result.

答案2

得分: -1

好的,问题出在操作系统套接字缓冲区溢出上。显然有太多数据需要处理。

因此,有三种可能的解决方法:

  • 增加套接字缓冲区容量
  • 以某种方式提高从套接字读取数据的速度
  • 降低数据量或频率

问题在于gami默认读取asterisk的所有数据。我在实际读取操作之后对它们进行了过滤。根据AMI监听应用程序在性能较差的PC上运行的情况,显然它无法在缓冲区容量耗尽之前读取所有数据。但是可以通过向AMI发送"Events"操作并指定所需的"EventMask"来仅接收特定事件。

因此,我的决定是这样做,并为不同的事件类型创建不同的连接。

英文:

Ok, problem was in OS socket buffer overflow. As appeared there were to much data to handle.

So, were are three possible ways to fix this:

  • increase socket buffer volume
  • increase somehow speed of process which reeds data from socket
  • lower data volume or frequency

The thing that gami is by default reading all data from asterisk. And i was reading all of them and filter them after actual read operation. According that AMI listening application were running on pretty poor PC it appeared that it simply cannot read all the data before buffer capacity will be exposed.But its possible to receive only particular events, by sending "Events" action to AMI and specifying desired "EventMask".
So, my decision was to do that. And create different connections for different events type.

答案3

得分: -2

我也遇到了读取数据的问题。我不得不为此创建一个单独的线程(go routine),并通过通道发送结果。然后我突然明白了。读取本身会阻塞客户端线程,因为它等待结束(EOF),但是给定的线程除了等待读取之外还必须做其他事情,所以读取必须在单独的线程(routine)中启动,这是一个示例。它通过通道发送读取结果。

go func(mch chan string, conn net.Conn, ctx context.Context) { 
    reader := bufio.NewReader(conn)
    for {
        s, err := reader.ReadString('#')
        if err != nil {
            mch <- "[read stop] " + fmt.Sprint(err)
            break
        } else {
            mch <- s
        }
        select {
        case <-ctx.Done():
            return
        default:
            za.SleepM(3)
        }
    }
}(mch, conn, ctx)
英文:

I also had a problem with read data. I had to make a separate thread for it (go routine) and send the result via chanel. Then it hit me like a flash.
The read itself blocks the client rutin , because it waits for the end (EOF), but the given thread must also do something other than just waiting for reading, so reading must be started in a separate thread (routine), and this is an example. It sends the reading result through the channel.

go func(mch chan string, conn net.Conn, ctx context.Context) { 
		reader := bufio.NewReader(conn)
		for {
			s, err := reader.ReadString(&#39;#&#39;)
			if err != nil {
				mch &lt;- &quot;[read stop] &quot; + fmt.Sprint(err)
				break
			} else {
				mch &lt;- s
			}
			select {
			case &lt;-ctx.Done():
				return
			default:
				za.SleepM(3)
			}
		}
	}(mch, conn, ctx)

huangapple
  • 本文由 发表于 2014年9月15日 22:55:09
  • 转载请务必保留本文链接:https://go.coder-hub.com/25850933.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定