如果不使用数据库,数据库会挂起。

huangapple go评论91阅读模式
英文:

Database hangs if not used

问题

我有一个正在启动的网络应用程序。在启动时运行正常,但如果我离开它一段时间(比如一个小时),然后再发送请求,查询就会挂起。我考虑每次查询后关闭连接,然后重新打开一个新的连接,但文档明确指出:“关闭数据库是很少见的,因为数据库句柄应该是长期存在的,并在许多goroutine之间共享。”我做错了什么?

package main

import (
  "database/sql"
  "log"
  "net/http"
  _ "github.com/lib/pq"
)

var Db *sql.DB

func main() {
  var err error
  Db, err = sql.Open("postgres", "user=me password=openupitsme host=my.host.not.yours dbname=mydb sslmode=require")
  if err != nil {
    log.Fatal("Cannot connect to db: ", err)
  }
  http.HandleFunc("/page", myHandler)
  http.ListenAndServe(":8080", nil)
}

func myHandler(w http.ResponseWriter, r *http.Request) {
  log.Println("Handling Request....", r)
  query := `SELECT pk FROM mytable LIMIT 1`
  rows, err := Db.Query(query)
  if err != nil {
    log.Println(err)
  }
  defer rows.Close()
  for rows.Next() {
    var pk int64
    if err := rows.Scan(&pk); err != nil {
      log.Println(err)
    }
    log.Println(pk)
  }
  log.Println("Request Served...")
}

编辑 #1:
我的PostgreSQL日志显示:

2015-07-08 18:10:01 EDT [7710-1] user@here LOG:  could not receive data from client: Connection reset by peer
2015-07-08 18:20:01 EDT [7756-1] user@here LOG:  could not receive data from client: Connection reset by peer
英文:

I have a web application I am starting. Works fine upon startup but if I leave it (for say, an hour) and hit it with another request the query hangs. I thought about closing it after each query then opening up a new connection but the docs explicitly say "It is rare to Close a DB, as the DB handle is meant to be long-lived and shared between many goroutines.". What am I doing wrong?

package main

import (
  "database/sql"
  "log"
  "net/http"
  _ "github.com/lib/pq"
)

var Db *sql.DB

func main() {
  var err error
  Db, err = sql.Open("postgres", "user=me password=openupitsme host=my.host.not.yours dbname=mydb sslmode=require")
  if err != nil {
    log.Fatal("Cannot connect to db: ", err)
  }
  http.HandleFunc("/page", myHandler)
  http.ListenAndServe(":8080", nil)
}

func myHandler(w http.ResponseWriter, r *http.Request) {
  log.Println("Handling Request....", r)
  query := `SELECT pk FROM mytable LIMIT 1`
  rows, err := Db.Query(query)
  if err != nil {
    log.Println(err)
  }
  defer rows.Close()
  for rows.Next() {
    var pk int64
    if err := rows.Scan(&pk); err != nil {
      log.Println(err)
    }
    log.Println(pk)
  }
  log.Println("Request Served...")
}

EDIT #1:
My postgres log shows:

2015-07-08 18:10:01 EDT [7710-1] user@here LOG:  could not receive data from client: Connection reset by peer
2015-07-08 18:20:01 EDT [7756-1] user@here LOG:  could not receive data from client: Connection reset by peer

答案1

得分: 4

我遇到过类似的问题。在我们的情况下,问题是由位于客户机和数据库之间的连接跟踪防火墙引起的。

这种防火墙会跟踪TCP级别的连接,并且为了限制资源使用,它们会超时那些在它们看来长时间没有活动的连接。在这种情况下,我们观察到的症状与你的情况非常相似:在客户端,连接似乎卡住了,而在服务器端,你可以看到"connection reset by peer"的错误。

防止这种情况发生的一种方法是确保启用了TCP Keepalives,并且保持活动间隔小于引起连接问题的防火墙、路由器等的超时时间。这可以通过libpq连接参数keepaliveskeepalives_idlekeepalives_intervalkeepalives_count来控制,你可以在连接字符串中设置这些参数。请参阅手册以了解这些参数的描述。

  • keepalive确定是否启用了keepalive功能。它的默认值为1(启用),所以你可能不需要指定它。
  • keepalives_idle确定在发送keepalive之前的空闲时间。如果你没有指定它,它将默认为操作系统的默认值。

在Linux系统中,你可以通过查看/proc/sys/net/ipv4/tcp_keepalive_time来查看默认值。在我的服务器上,它设置为7200秒,这对你来说太长了,因为你的观察是连接在大约1小时后断开。

你可以尝试将它设置为2500秒。

Linux文档项目提供了一个有用的TCP Keepalive HOWTO文档,详细描述了它们的工作原理。

请注意,并非所有操作系统都支持TCP keepalive。如果你无法启用keepalive,还有一些其他选项可以考虑:

  1. 如果可以控制,重新配置丢弃连接的防火墙/路由器,使其不会对PostgreSQL客户端连接进行丢弃。

  2. 在应用程序级别,你可以尝试发送一些流量来保持数据库句柄的活动状态,例如每小时发送一个SELECT 1;语句。如果你的编程环境提供连接缓存(从评论中了解到它确实提供),那么这可能有些棘手。

英文:

I have experienced similar issues. In our case, the problem was caused by a connection tracking firewall located between the client machine and the database.

Such firewalls keep track of TCP level connections, and in order to limit resource usage, then will time out connections which to them appear inactive for an extended period. The symptoms we observed in this case were very similar to yours: at the client end, the connection appears to be hanging, while at the server end you can see connection reset by peer.

One way to prevent this is to ensure that TCP Keepalives are enabled, and that the keepalive interval is less than the timeout of the firewalls, routers, etc which are causing your connection issue. This is controlled by the libpq connection parameters keepalives, keepalives_idle, keepalives_interval and keepalives_count which you can set in the connection string. See the manual for a description of these parameters.

  • keepalive determines if the keepalive function is enabled or not. It defaults to 1 (enabled) so you probably do not need to specify this.

  • keepalives_idle determines the amount of idle time before it will send a keepalive. If you do not specify this, it will default to the default for the operating system.

    In a Linux system you can see the default by examining /proc/sys/net/ipv4/tcp_keepalive_time - in my server it is set to 7200 seconds, which would be too long in your case, since your observation is that the connection is dropped after ~1 hour.

    You could try setting it to, say, 2500 seconds.

The Linux Documentation Project provides a useful TCP Keepalive HOWTO document that describes how they work in some detail.

Note that not all operating systems support TCP keepalives. If you are unable to enable keepalives here are some other options you might like to consider:

  1. If it is in your control, reconfigure the firewall/router which is dropping the connection so that it will not do so for Postgresql client connections

  2. At an application level, you might be able to send some traffic that will keep the DB handles active - for example sending a statement such as SELECT 1; every hour or so. If your programming environment provides connection caching (from the comments I gather it does) then this might be tricky.

huangapple
  • 本文由 发表于 2015年7月7日 11:14:19
  • 转载请务必保留本文链接:https://go.coder-hub.com/31259154.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定