构建系统日志消息时出现乱序输出。

huangapple go评论83阅读模式
英文:

Scrambled output when constructing a syslog message

问题

我正在编写一个程序,可以将已经记录在文件中的标准syslog消息转换为包括PRIheader等的syslog消息。

我已经能够从存储在我的kafka测试集群中的syslog消息中实现这一点,它的可视化效果如下:

2017-03-09T15:22:00.642769+00:00 ldaptestserver slapd[392]: slap_global_control: unrecognized control: 1.3.6.1.4.1.42.2.27.8.5.1

通过在消息前添加优先级并初始化到syslog端口的TCP连接,并将消息作为[]byte切片写入套接字,我能够实现这一点。在从kafka订阅后发送到syslog之前,最终的消息如下所示:

<13>2017-03-09T15:22:00.642769+00:00 ldaptestserver slapd[392]: slap_global_control: unrecognized control: 1.3.6.1.4.1.42.2.27.8.5.1

其中只有<13>被添加到整个日志中。

现在,我尝试打开并读取/var/log/syslog,并构造了与上述类似的消息:

<13>2017-03-08T12:29:02.231335+00:00 ldaptestserver slapd[392]: slap_global_control: unrecognized control: 1.3.6.1.4.1.42.2.27.8.5.1 // 来自kafka的原始消息,可以正常工作
<13>2017-00-01T16:18:04.000000+00:00 ldaptestserver slapd[392]: slap_global_control: unrecognized control: 1.3.6.1.4.1.42.2.27.8.5.1 // 手工构造的消息

但是,当我将[]byte消息写入syslog连接套接字时,它产生了一个乱序的输出。

(请注意,IP是本地主机IP,而不是消息中的主机名。在使用类似的kafka消息的情况下,它以原始主机名完美记录。)

有人可以帮助我理解出了什么问题,以及我应该怎么做才能得到正确的输出吗?

我在使用golang编写这个程序。

-- Scott.

英文:

I was writing a program which could convert a standard syslog message that is already logged in the file, for eg:

Mar  9 15:51:36 ldaptestserver slapd[392]: slap_global_control: unrecognized control: 1.3.6.1.4.1.42.2.27.8.5.1

to a syslog message including the PRI, header etc.

I was able to do this from a syslog message stored in my kafka test cluster, which visually looked like

2017-03-09T15:22:00.642769+00:00 ldaptestserver slapd[392]: slap_global_control: unrecognized control: 1.3.6.1.4.1.42.2.27.8.5.1

by prepending the priority and initiating a tcp connection to the syslog port, and writing the message as a []byte slice to the socket. The final message before sending it to syslog, after subscribing from kafka looked like

<13>2017-03-09T15:22:00.642769+00:00 ldaptestserver slapd[392]: slap_global_control: unrecognized control: 1.3.6.1.4.1.42.2.27.8.5.1

where only <13> is prepended to the entire log.

Now, instead of kafka, I try to open and read the /var/log/syslog, and constructed the same message that looked like the above,

<13>2017-03-08T12:29:02.231335+00:00 ldaptestserver slapd[392]: slap_global_control: unrecognized control: 1.3.6.1.4.1.42.2.27.8.5.1 // Original one from kafka that worked
<13>2017-00-01T16:18:04.000000+00:00 ldaptestserver slapd[392]: slap_global_control: unrecognized control: 1.3.6.1.4.1.42.2.27.8.5.1 // The message that is hand crafted.

But when I write the []byte message to the syslog connection socket, it produces a scrambled output.

构建系统日志消息时出现乱序输出。

(Notice that the IP is the localhost IP, instead of the hostname in the message. In the case where the similar kafka message was used, it logged perfectly with the original hostname.)

Can someone help me understand where is it messed up, and what should I do to get it without being scrambled?

I was using golang to program this.

-- Scott.

答案1

得分: 4

尝试将从kafka中提取的工作日志以[]byte切片的形式直接转储,而不是将其打印为string()。这将给你一串数字,你可以将其转换为对应的ascii字符,然后查看是否有像回车换行这样的分隔符,用于分隔头部的每个部分和消息的行尾。如果你在打印时将其转换为string(),这些分隔符将不可见。很有可能,syslog服务器的解析器期望使用行终止符来区分头部部分或消息的结尾,但它找不到,因此会不断附加后续的消息,直到解析器的缓冲区用尽。

英文:

Instead of printing the working log extracted from kafka as string(), try to dump the []byte slice itself as it is. This will give you a bunch of numbers, which you can translate to its corresponding ascii characters, and you can see if there are any separators like carriage return or line feed that is separating the end of each section in the header and end of line of the message. This wouldn't be visible if you are casting it to string() when printing. Chances are that the syslog server's parser is expecting a line terminator to distinguish between header sections or end of a message, which it is failing to find, and keeps on appending subsequent messages until the parser's buffer runs out.

huangapple
  • 本文由 发表于 2017年3月9日 23:52:49
  • 转载请务必保留本文链接:https://go.coder-hub.com/42699788.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定