英文:
Database hangs if not used
问题
我有一个正在启动的网络应用程序。在启动时运行正常,但如果我离开它一段时间(比如一个小时),然后再发送请求,查询就会挂起。我考虑每次查询后关闭连接,然后重新打开一个新的连接,但文档明确指出:“关闭数据库是很少见的,因为数据库句柄应该是长期存在的,并在许多goroutine之间共享。”我做错了什么?
package main
import (
"database/sql"
"log"
"net/http"
_ "github.com/lib/pq"
)
var Db *sql.DB
func main() {
var err error
Db, err = sql.Open("postgres", "user=me password=openupitsme host=my.host.not.yours dbname=mydb sslmode=require")
if err != nil {
log.Fatal("Cannot connect to db: ", err)
}
http.HandleFunc("/page", myHandler)
http.ListenAndServe(":8080", nil)
}
func myHandler(w http.ResponseWriter, r *http.Request) {
log.Println("Handling Request....", r)
query := `SELECT pk FROM mytable LIMIT 1`
rows, err := Db.Query(query)
if err != nil {
log.Println(err)
}
defer rows.Close()
for rows.Next() {
var pk int64
if err := rows.Scan(&pk); err != nil {
log.Println(err)
}
log.Println(pk)
}
log.Println("Request Served...")
}
编辑 #1:
我的PostgreSQL日志显示:
2015-07-08 18:10:01 EDT [7710-1] user@here LOG: could not receive data from client: Connection reset by peer
2015-07-08 18:20:01 EDT [7756-1] user@here LOG: could not receive data from client: Connection reset by peer
英文:
I have a web application I am starting. Works fine upon startup but if I leave it (for say, an hour) and hit it with another request the query hangs. I thought about closing it after each query then opening up a new connection but the docs explicitly say "It is rare to Close a DB, as the DB handle is meant to be long-lived and shared between many goroutines.". What am I doing wrong?
package main
import (
"database/sql"
"log"
"net/http"
_ "github.com/lib/pq"
)
var Db *sql.DB
func main() {
var err error
Db, err = sql.Open("postgres", "user=me password=openupitsme host=my.host.not.yours dbname=mydb sslmode=require")
if err != nil {
log.Fatal("Cannot connect to db: ", err)
}
http.HandleFunc("/page", myHandler)
http.ListenAndServe(":8080", nil)
}
func myHandler(w http.ResponseWriter, r *http.Request) {
log.Println("Handling Request....", r)
query := `SELECT pk FROM mytable LIMIT 1`
rows, err := Db.Query(query)
if err != nil {
log.Println(err)
}
defer rows.Close()
for rows.Next() {
var pk int64
if err := rows.Scan(&pk); err != nil {
log.Println(err)
}
log.Println(pk)
}
log.Println("Request Served...")
}
EDIT #1:
My postgres log shows:
2015-07-08 18:10:01 EDT [7710-1] user@here LOG: could not receive data from client: Connection reset by peer
2015-07-08 18:20:01 EDT [7756-1] user@here LOG: could not receive data from client: Connection reset by peer
答案1
得分: 4
我遇到过类似的问题。在我们的情况下,问题是由位于客户机和数据库之间的连接跟踪防火墙引起的。
这种防火墙会跟踪TCP级别的连接,并且为了限制资源使用,它们会超时那些在它们看来长时间没有活动的连接。在这种情况下,我们观察到的症状与你的情况非常相似:在客户端,连接似乎卡住了,而在服务器端,你可以看到"connection reset by peer"的错误。
防止这种情况发生的一种方法是确保启用了TCP Keepalives,并且保持活动间隔小于引起连接问题的防火墙、路由器等的超时时间。这可以通过libpq连接参数keepalives
、keepalives_idle
、keepalives_interval
和keepalives_count
来控制,你可以在连接字符串中设置这些参数。请参阅手册以了解这些参数的描述。
keepalive
确定是否启用了keepalive功能。它的默认值为1
(启用),所以你可能不需要指定它。keepalives_idle
确定在发送keepalive之前的空闲时间。如果你没有指定它,它将默认为操作系统的默认值。
在Linux系统中,你可以通过查看/proc/sys/net/ipv4/tcp_keepalive_time
来查看默认值。在我的服务器上,它设置为7200秒,这对你来说太长了,因为你的观察是连接在大约1小时后断开。
你可以尝试将它设置为2500秒。
Linux文档项目提供了一个有用的TCP Keepalive HOWTO文档,详细描述了它们的工作原理。
请注意,并非所有操作系统都支持TCP keepalive。如果你无法启用keepalive,还有一些其他选项可以考虑:
-
如果可以控制,重新配置丢弃连接的防火墙/路由器,使其不会对PostgreSQL客户端连接进行丢弃。
-
在应用程序级别,你可以尝试发送一些流量来保持数据库句柄的活动状态,例如每小时发送一个
SELECT 1;
语句。如果你的编程环境提供连接缓存(从评论中了解到它确实提供),那么这可能有些棘手。
英文:
I have experienced similar issues. In our case, the problem was caused by a connection tracking firewall located between the client machine and the database.
Such firewalls keep track of TCP level connections, and in order to limit resource usage, then will time out connections which to them appear inactive for an extended period. The symptoms we observed in this case were very similar to yours: at the client end, the connection appears to be hanging, while at the server end you can see connection reset by peer
.
One way to prevent this is to ensure that TCP Keepalives are enabled, and that the keepalive interval is less than the timeout of the firewalls, routers, etc which are causing your connection issue. This is controlled by the libpq connection parameters keepalives
, keepalives_idle
, keepalives_interval
and keepalives_count
which you can set in the connection string. See the manual for a description of these parameters.
-
keepalive
determines if the keepalive function is enabled or not. It defaults to1
(enabled) so you probably do not need to specify this. -
keepalives_idle
determines the amount of idle time before it will send a keepalive. If you do not specify this, it will default to the default for the operating system.In a Linux system you can see the default by examining
/proc/sys/net/ipv4/tcp_keepalive_time
- in my server it is set to 7200 seconds, which would be too long in your case, since your observation is that the connection is dropped after ~1 hour.You could try setting it to, say, 2500 seconds.
The Linux Documentation Project provides a useful TCP Keepalive HOWTO document that describes how they work in some detail.
Note that not all operating systems support TCP keepalives. If you are unable to enable keepalives here are some other options you might like to consider:
-
If it is in your control, reconfigure the firewall/router which is dropping the connection so that it will not do so for Postgresql client connections
-
At an application level, you might be able to send some traffic that will keep the DB handles active - for example sending a statement such as
SELECT 1;
every hour or so. If your programming environment provides connection caching (from the comments I gather it does) then this might be tricky.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论