英文:
Request Context Deadline Exceeded
问题
我正在Go语言中处理请求上下文。我正在处理用于更新用户账户的端点。当我连续发送多个请求(POSTMAN)到同一个端点时,它返回错误context deadline exceeded
。我不确定为什么只有在连续发送相同请求到同一个端点时才会出现这个错误。context deadline exceeded
错误出现在第9个请求左右,并在1秒后再次出现。之后,它还会阻塞对其他端点的所有请求。
routes.go [每个请求都会有一个请求上下文 - requestMiddleware]
func Routes(app repository.DatabaseRepo) http.Handler {
r := chi.NewRouter()
// 在这里创建请求上下文并设置请求ID
r.Use(appMiddleware.RequestMiddleware)
updateUserH := admin.New(app)
r.Route("/admin", func(r chi.Router) {
r.Patch("/update-user", utils.MakeHTTPHandler(updateUserH.UpdateUser))
})
return r
}
requestMiddleware.go [创建带有请求ID的上下文]
type RequestIDKey string
const RequestID RequestIDKey = "request_id"
func RequestMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
requestIDUuid := uuid.New().String()
ctx := context.WithValue(req.Context(), RequestID, requestIDUuid)
next.ServeHTTP(w, req.WithContext(ctx))
})
}
UpdateUser.go [使用请求上下文并设置超时]
func (app application) UpdateUser(w http.ResponseWriter, req *http.Request) error {
....
ctx := req.Context()
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
utilsApp := utils.Application{DB: app.DB}
err := utils.InjectUG(utilsApp, ctx, w.Header().Get("username"), "Admin")
if err != nil {
return err
}
return utils.WriteJSON(w, http.StatusOK, utils.ApiSuccess{Success: "[Admin] Successfully updated '" + updateUser.Username + "' user!", Status: http.StatusOK})
}
更新
utils.InjectUG
函数调用了这个数据库查询。这个查询在触发多个传入请求时导致整个上下文超过了截止时间。当我注释掉utils.InjectUG
时,它可以正常工作。我需要这个数据库查询来检查用户是否属于用户组以访问特定资源。
type Application struct {
DB repository.DatabaseRepo
}
func InjectUG(app Application, ctx context.Context, username string, userGroups ...string) error {
return app.CheckUserGroup(ctx, username, userGroups...)
}
// 确定用户是否被分配了该用户组
func (app Application) CheckUserGroup(ctx context.Context, username string, userGroups ...string) error {
isAuthorizedUser, err := app.DB.GetUserGroupsByUsername(ctx, username, userGroups...)
if err != nil {
return ApiError{Err: "Internal Server Error", Status: http.StatusInternalServerError}
}
if isAuthorizedUser {
return nil
}
return ApiError{Err: "Access Denied: User does not have permission to access this resource", Status: http.StatusForbidden}
}
var SQL_GET_USERGROUPS_BY_USERNAME = `SELECT ug.user_group FROM user_groups ug
LEFT JOIN user_group_mapping ugm
ON ugm.user_group_id = ug.user_group_id
WHERE ugm.user_id = (SELECT user_id FROM users WHERE username = $1);`
func (m *PostgresDBRepo) GetUserGroupsByUsername(ctx context.Context, username string, userGroups ...string) (bool, error) {
// 在这个查询上失败:context deadline exceeded
rows, err := m.DB.Query(ctx, SQL_GET_USERGROUPS_BY_USERNAME, username)
if err != nil {
log.Println("Query failed at GetUserGroupsByUsername:", err)
return false, err
}
var userGroup string
if rows != nil {
for rows.Next() {
if err = rows.Scan(&userGroup); err != nil {
return false, err
}
if utils.Contains(userGroups, userGroup) {
return true, nil
}
}
}
return true, nil
}
英文:
I am working on request contexts in Go. I am working on this endpoint for updating user accounts. When I send multiple incoming requests (POSTMAN) to the same endpoint consecutively, it returns the error context deadline exceeded
. I am not sure why it only happens when I send the same request to the same endpoint continuously. The context deadline exceeded
error appears around the 9th request and will appear 1 second later. After which, it blocks all requests to other endpoints as well.
routes.go [Every request will have a request Context - requestMiddleware]
func Routes(app repository.DatabaseRepo) http.Handler {
r := chi.NewRouter()
// Creates request context here with requestID
r.Use(appMiddleware.RequestMiddleware)
updateUserH := admin.New(app)
r.Route("/admin", func(r chi.Router) {
r.Patch("/update-user", utils.MakeHTTPHandler(updateUserH.UpdateUser))
})
return r
}
requestMiddleware.go [Creates a context with requestID]
type RequestIDKey string
const RequestID RequestIDKey = "request_id"
func RequestMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
requestIDUuid := uuid.New().String()
ctx := context.WithValue(req.Context(), RequestID, requestIDUuid)
next.ServeHTTP(w, req.WithContext(ctx))
})
}
UpdateUser.go [Uses the request context and sets a timeout]
func (app application) UpdateUser(w http.ResponseWriter, req *http.Request) error {
....
ctx := req.Context()
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
utilsApp := utils.Application{DB: app.DB}
err := utils.InjectUG(utilsApp, ctx, w.Header().Get("username"), "Admin")
if err != nil {
return err
}
return utils.WriteJSON(w, http.StatusOK, utils.ApiSuccess{Success: "[Admin] Successfully updated '" + updateUser.Username + "' user!", Status: http.StatusOK})
}
UPDATE
utils.InjectUG
function is calling this database query. This query is causing the entire context to exceed the deadline upon firing multiple incoming requests. When I commented out utils.InjectUG
, it works perfectly. I need this database query to check if a user belongs to a user group to access particular resources.
type Application struct {
DB repository.DatabaseRepo
}
func InjectUG(app Application, ctx context.Context, username string, userGroups ...string) error {
return app.CheckUserGroup(ctx, username, userGroups...)
}
// Determines if a user has been assigned that usergroup
func (app Application) CheckUserGroup(ctx context.Context, username string, userGroups ...string) error {
isAuthorizedUser, err := app.DB.GetUserGroupsByUsername(ctx, username, userGroups...)
if err != nil {
return ApiError{Err: "Internal Server Error", Status: http.StatusInternalServerError}
}
if isAuthorizedUser {
return nil
}
return ApiError{Err: "Access Denied: User does not have permission to access this resource", Status: http.StatusForbidden}
}
var SQL_GET_USERGROUPS_BY_USERNAME = `SELECT ug.user_group FROM user_groups ug
LEFT JOIN user_group_mapping ugm
ON ugm.user_group_id = ug.user_group_id
WHERE ugm.user_id = (SELECT user_id FROM users WHERE username = $1);`
func (m *PostgresDBRepo) GetUserGroupsByUsername(ctx context.Context, username string, userGroups ...string) (bool, error) {
// IT FAILS ON THIS QUERY: context deadline exceeded
rows, err := m.DB.Query(ctx, SQL_GET_USERGROUPS_BY_USERNAME, username)
if err != nil {
log.Println("Query failed at GetUserGroupsByUsername:", err)
return false, err
}
var userGroup string
if rows != nil {
for rows.Next() {
if err = rows.Scan(&userGroup); err != nil {
return false, err
}
if utils.Contains(userGroups, userGroup) {
return true, nil
}
}
}
return true, nil
}
答案1
得分: 1
[我无法找到重复的问题,这很奇怪]
rows
应该显式关闭。
修复方法是在错误检查后添加 defer rows.Close()
:
rows, err := m.DB.Query(ctx, SQL_GET_USERGROUPS_BY_USERNAME, username)
if err != nil {
log.Println("Query failed at GetUserGroupsByUsername:", err)
return false, err
}
+ defer rows.Close()
参考资料:
-
循环遍历所有行也会隐式关闭它,但最好使用 defer 来确保无论如何都关闭 rows。
-
https://pkg.go.dev/github.com/jackc/pgx/v5#Conn.Query
必须在再次使用连接之前关闭返回的 Rows。
这里有一个简单的演示,重现了这个问题:
package main
import (
"context"
"log"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"golang.org/x/exp/slices"
)
func GetUserGroupsByUsername(ctx context.Context, db *pgxpool.Pool, userGroups ...string) (bool, error) {
rows, err := db.Query(ctx, "values ('g1'),('g2')")
if err != nil {
log.Println("Query failed at GetUserGroupsByUsername:", err)
return false, err
}
// uncomment the next line to address the issue.
// defer rows.Close()
var userGroup string
for rows.Next() {
if err = rows.Scan(&userGroup); err != nil {
return false, err
}
if slices.Contains(userGroups, userGroup) {
return true, nil
}
}
log.Println("error:", rows.Err())
return true, nil
}
func main() {
connString := "postgres://username:password@localhost:5432/dbname"
pool, err := pgxpool.New(context.Background(), connString)
if err != nil {
log.Fatalf("Unable to connect to database: %v\n", err)
}
defer pool.Close()
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
for i := 0; i < 10; i++ {
log.Println(GetUserGroupsByUsername(ctx, pool, "g1"))
}
}
输出结果:
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:34 Query failed at GetUserGroupsByUsername: context deadline exceeded
2023/05/08 09:10:34 false context deadline exceeded
2023/05/08 09:10:34 Query failed at GetUserGroupsByUsername: context deadline exceeded
2023/05/08 09:10:34 false context deadline exceeded
^Csignal: interrupt
英文:
[It's weird that I can not find a duplicate question]
The rows
should be closed explicitly.
The fix is to add defer rows.Close()
after the error check:
rows, err := m.DB.Query(ctx, SQL_GET_USERGROUPS_BY_USERNAME, username)
if err != nil {
log.Println("Query failed at GetUserGroupsByUsername:", err)
return false, err
}
+ defer rows.Close()
References:
-
> Looping all the way through the rows also closes it implicitly, but it is better to use defer to make sure rows is closed no matter what.
-
https://pkg.go.dev/github.com/jackc/pgx/v5#Conn.Query
> The returned Rows must be closed before the connection can be used again.
Here is a simple demo that reproduces the issue:
package main
import (
"context"
"log"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"golang.org/x/exp/slices"
)
func GetUserGroupsByUsername(ctx context.Context, db *pgxpool.Pool, userGroups ...string) (bool, error) {
rows, err := db.Query(ctx, "values ('g1'),('g2')")
if err != nil {
log.Println("Query failed at GetUserGroupsByUsername:", err)
return false, err
}
// uncomment the next line to address the issue.
// defer rows.Close()
var userGroup string
for rows.Next() {
if err = rows.Scan(&userGroup); err != nil {
return false, err
}
if slices.Contains(userGroups, userGroup) {
return true, nil
}
}
log.Println("error:", rows.Err())
return true, nil
}
func main() {
connString := "postgres://username:password@localhost:5432/dbname"
pool, err := pgxpool.New(context.Background(), connString)
if err != nil {
log.Fatalf("Unable to connect to database: %v\n", err)
}
defer pool.Close()
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
for i := 0; i < 10; i++ {
log.Println(GetUserGroupsByUsername(ctx, pool, "g1"))
}
}
The output:
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:33 true <nil>
2023/05/08 09:10:34 Query failed at GetUserGroupsByUsername: context deadline exceeded
2023/05/08 09:10:34 false context deadline exceeded
2023/05/08 09:10:34 Query failed at GetUserGroupsByUsername: context deadline exceeded
2023/05/08 09:10:34 false context deadline exceeded
^Csignal: interrupt
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论