英文:
Gorm query lasts over a minute before successfully committing
问题
我正在使用控制器-服务-存储库架构编写一个简单的 Golang 应用程序。
我的存储库可以访问数据库，并负责在数据库中插入记录。
在我使用 Postman 发出请求后，它持续了一分钟以上，并成功地将记录插入到数据库中。我还尝试直接将控制器访问数据库，结果也是一样的。
我甚至尝试编写一个简单的处理程序，建立与数据库的连接并执行查询，这个处理程序运行得很好，但如果我使用这种架构，性能就会变慢。
如果有人知道可能出现的问题，请告诉我。
这是我的主函数以及控制器、服务和存储库的代码：
主函数：
func main() {
r := setupRouter()
pgInstance := db2.ConnectToDatabase()
repo := repository.NewRepository(pgInstance)
service := service2.NewService(repo)
c := controller2.NewController(service)
routes.InitializeRoutes(r, c)
// Listen and Server in 0.0.0.0:8080
r.Run(":8080")
}
控制器：
type Controller struct {
service service.Service
}
func NewController(service service.Service) Controller {
return Controller{
service: service,
}
}
服务：
type Service struct {
repository repository.Repository
}
func NewService(repo repository.Repository) Service {
return Service{
repository: repo,
}
}
存储库：
type Repository struct {
db *gorm.DB
}
func NewRepository(db *gorm.DB) Repository {
return Repository{
db: db,
}
}
func (r Repository) Db() *gorm.DB {
return r.db
}
英文:
I am writing a simple golang app with controller-service-repository architecture.
My repository has access to database and is in charge of inserting records in database.
After I make a request in Postman, it lasts over a minute and it successfully inserts a record in database. I also tried giving the controller direct access to database and same thing occurs.
I even tried writing a simple handler that establishes connection to database and performs a query and that works just fine, but if I use this architecture something is slowing down the performance.
If anyone has an idea what might be a problem, please let me know.
Here is the code of my main function as well as the controller, service and repository :
main :
func main() {
r := setupRouter()
pgInstance := db2.ConnectToDatabase()
repo := repository.NewRepository(pgInstance)
service := service2.NewService(repo)
c := controller2.NewController(service)
routes.InitializeRoutes(r, c)
// Listen and Server in 0.0.0.0:8080
r.Run(":8080")
}
controller :
type Controller struct {
service service.Service
}
func NewController(service service.Service) Controller {
return Controller{
service: service,
}
}
service :
type Service struct {
repository repository.Repository
}
func NewService(repo repository.Repository) Service {
return Service{
repository: repo,
}
}
repository :
type Repository struct {
db *gorm.DB
}
func NewRepository(db *gorm.DB) Repository {
return Repository{
db: db,
}
}
func (r Repository) Db() *gorm.DB {
return r.db
}
答案1
得分: 1
经过几周的调查，我找到了导致问题的原因。
我有一个BeforeSave钩子，用于加密用户密码。我将性能成本设置得太高了。
对于那些和我犯同样错误的人来说，成本设置在10左右就可以了（20是不可行的）。
英文:
After weeks of digging I found the issue what is causing the problem.
I have a BeforeSave hook that encrypted users password.I set performance cost too high.
For those who did same mistake as me, cost around 10 is just fine (20 is a no go).
答案2
得分: 0
以下是相同的文件，其中添加了一些内容以更好地测量和调试任何性能问题：
main.go
> 有些东西正在减慢性能。
然后，我会启动并启用Go的内置分析器。这有助于了解应用程序花费时间的位置，并可用于识别瓶颈。
您还可以添加一个LoggerMiddleware来记录传入的HTTP请求，以监视传入请求的性能。
确保数据库连接池已配置，以确保池具有足够的连接来处理您的请求。设置最大打开连接数、空闲连接数和连接生命周期有助于有效管理资源。
还要激活GORM的LogMode，以便在控制台上打印SQL查询以进行调试。这有助于检查GORM生成并执行的实际SQL查询。
package main
import (
"github.com/gin-gonic/gin"
"gorm.io/gorm"
"log"
"net/http"
_ "net/http/pprof"
"time"
)
func main() {
// 分析：启用Go的内置分析器，查看应用程序花费时间的位置。
go func() {
log.Println(http.ListenAndServe("localhost:6060", nil))
}()
// 设置路由器
r := gin.Default()
// 用于记录请求的中间件：此中间件记录传入的HTTP请求，用于调试和监视性能。
r.Use(LoggerMiddleware())
// 数据库连接：使用GORM建立与数据库的连接。
db, err := gorm.Open(...)
if err != nil {
log.Fatal(err)
}
// 配置连接池：GORM使用连接池来管理与数据库的连接。此配置确保池具有足够的连接来处理您的请求。
sqlDB, err := db.DB()
if err != nil {
log.Fatal(err)
}
sqlDB.SetMaxOpenConns(10)
sqlDB.SetMaxIdleConns(5)
sqlDB.SetConnMaxLifetime(time.Minute * 5)
// 查询优化：启用GORM的LogMode，以便在控制台上打印SQL查询以进行调试。这有助于检查GORM生成并执行的实际SQL查询。
db = db.Debug()
// 初始化存储库、服务、控制器
repo := NewRepository(db)
service := NewService(repo)
controller := NewController(service)
// 初始化路由
InitializeRoutes(r, controller)
// 监听并在0.0.0.0:8080上提供服务
r.Run(":8080")
}
// LoggerMiddleware是用于记录HTTP请求的中间件函数。
func LoggerMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
startTime := time.Now()
c.Next()
log.Println("Request:", c.Request.Method, c.Request.URL, "Duration:", time.Since(startTime))
}
}
repository.go
repository.go文件包含应用程序的存储库层。存储库负责与数据库交互。它抽象了数据库访问并包含CRUD操作。
您可以通过更新CreateRecord函数来添加事务管理：这样可以确保在记录创建过程中出现错误时回滚事务。正确管理事务对于避免争用和潜在的性能问题很重要。
package main
import (
"gorm.io/gorm"
"log"
)
type Repository struct {
db *gorm.DB
}
// NewRepository使用数据库连接初始化新的存储库。
func NewRepository(db *gorm.DB) Repository {
return Repository{
db: db,
}
}
// Db返回数据库连接。
func (r Repository) Db() *gorm.DB {
return r.db
}
// CreateRecord将新记录插入数据库。
// 事务管理：此函数使用具有回滚功能的事务。
// 长时间运行的事务可能会导致争用并降低性能，因此正确管理它们很重要。
func (r Repository) CreateRecord(record interface{}) {
tx := r.db.Begin()
if err := tx.Create(&record).Error; err != nil {
tx.Rollback()
log.Println("Error inserting record:", err)
return
}
tx.Commit()
}
service.go
service.go文件包含应用程序的服务层，即业务逻辑，并可与存储库进行数据库访问或进行外部API调用：添加超时。
ExternalAPICall函数是使用超时进行外部API调用的示例。设置超时很重要，以避免无限等待，这可能会阻塞资源并对性能产生负面影响。
package main
import (
"net/http"
"time"
"log"
)
type Service struct {
repository Repository
}
// NewService使用给定的存储库初始化新的服务。
func NewService(repo Repository) Service {
return Service{
repository: repo,
}
}
// ExternalAPICall使用超时进行外部API调用。
// 外部API调用：如果您的服务层正在调用外部API或服务，则设置超时很重要，以避免无限等待。
// 否则，这可能会阻塞资源并降低应用程序的性能。
func (s Service) ExternalAPICall() {
client := &http.Client{
Timeout: time.Second * 10,
}
resp, err := client.Get("http://example.com")
if err != nil {
log.Println("Error fetching data:", err)
return
}
defer resp.Body.Close()
// 处理响应
}
controller.go
controller.go文件包含应用程序的控制器层，负责处理HTTP请求和响应。它可以将业务逻辑委托给服务层，并负责构造HTTP响应。
虽然没有特定的与性能相关的更改，但请确保在此处正确处理错误和构造响应，以确保应用程序的高效运行。
package main
type Controller struct {
service Service
}
// NewController使用给定的服务初始化新的控制器。
func NewController(service Service) Controller {
return Controller{
service: service,
}
}
// 在此处定义处理程序函数。在这里处理HTTP请求和响应。
// 控制器负责处理HTTP请求，并可以将业务逻辑委托给服务层。
// 应在此处管理适当的错误处理和响应构造。
<details>
<summary>英文:</summary>
Here are the same files, with some additions to better measure and debug any performance issue:
### main.go
> Something is slowing down the performance.
Then I would start and enable [Go's built-in profiler](https://go.dev/blog/pprof). This helps in understanding where the application is spending time and can be used to identify bottlenecks.
You can also add a `LoggerMiddleware` is added to log the incoming HTTP requests, for monitoring the performance of incoming requests.
Make sure the [database connection pool](https://gorm.io/docs/generic_interface.html#Connection-Pool) is configured to ensure that the pool has enough connections to handle your requests. Setting maximum open connections, idle connections, and connection lifetime helps in managing resources efficiently.
Activate also [GORM's LogMode](https://gorm.io/docs/logger.html), enabled to print SQL queries to the console for debugging purposes. This helps in examining the actual SQL queries being generated and executed against the database.
```go
package main
import (
"github.com/gin-gonic/gin"
"gorm.io/gorm"
"log"
"net/http"
_ "net/http/pprof"
"time"
)
func main() {
// Profiling: Enabling Go's built-in profiler to see where the time is being spent in your application.
go func() {
log.Println(http.ListenAndServe("localhost:6060", nil))
}()
// Setting up the router
r := gin.Default()
// Middleware for logging requests: This middleware logs the incoming HTTP requests,
// for debugging and monitoring performance.
r.Use(LoggerMiddleware())
// Database Connection: Establish connection to the database using GORM.
db, err := gorm.Open(...)
if err != nil {
log.Fatal(err)
}
// Configure connection pool: GORM uses a connection pool to manage connections to the database.
// This configuration ensures that the pool has enough connections to handle your requests.
sqlDB, err := db.DB()
if err != nil {
log.Fatal(err)
}
sqlDB.SetMaxOpenConns(10)
sqlDB.SetMaxIdleConns(5)
sqlDB.SetConnMaxLifetime(time.Minute * 5)
// Query Optimization: Enabling GORM's LogMode to print SQL queries to the console for debugging purposes.
// This is useful to check the actual SQL queries being generated by GORM and executed against the database.
db = db.Debug()
// Initialize repository, service, controller
repo := NewRepository(db)
service := NewService(repo)
controller := NewController(service)
// Initialize routes
InitializeRoutes(r, controller)
// Listen and serve on 0.0.0.0:8080
r.Run(":8080")
}
// LoggerMiddleware is a middleware function for logging HTTP requests.
func LoggerMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
startTime := time.Now()
c.Next()
log.Println("Request:", c.Request.Method, c.Request.URL, "Duration:", time.Since(startTime))
}
}
repository.go
The repository.go file contains the repository layer of the application. The repository is responsible for interacting with the database. It abstracts the database access and contains the CRUD operations.
You can add transaction management by updating the CreateRecord function: this ensures that if there is an error during the creation of the record, the transaction is rolled back. Proper management of transactions is important to avoid contention and potential performance issues.
package main
import (
"gorm.io/gorm"
"log"
)
type Repository struct {
db *gorm.DB
}
// NewRepository initializes a new repository with a database connection.
func NewRepository(db *gorm.DB) Repository {
return Repository{
db: db,
}
}
// Db returns the database connection.
func (r Repository) Db() *gorm.DB {
return r.db
}
// CreateRecord inserts a new record into the database.
// Transaction Management: This function uses a transaction with rollback in case of an error.
// Long-running transactions can cause contention and slow down the performance, so it is good to manage them properly.
func (r Repository) CreateRecord(record interface{}) {
tx := r.db.Begin()
if err := tx.Create(&record).Error; err != nil {
tx.Rollback()
log.Println("Error inserting record:", err)
return
}
tx.Commit()
}
service.go
The service.go file contains the service layer of the application, that is the business logic and can interact with the repository for database access or make external API calls: add a timeout.
The ExternalAPICall function is an example of making an external API call with a timeout. Setting a timeout is important to avoid waiting indefinitely, which can block resources and negatively affect performance.
package main
import (
"net/http"
"time"
"log"
)
type Service struct {
repository Repository
}
// NewService initializes a new service with the given repository.
func NewService(repo Repository) Service {
return Service{
repository: repo,
}
}
// ExternalAPICall makes an external API call with a timeout.
// External API Calls: If your service layer is making calls
// to external APIs or services, it s important to set timeouts
// to avoid waiting indefinitely.
// This could otherwise block resources and slow down the performance of your application.
func (s Service) ExternalAPICall() {
client := &http.Client{
Timeout: time.Second * 10,
}
resp, err := client.Get("http://example.com")
if err != nil {
log.Println("Error fetching data:", err)
return
}
defer resp.Body.Close()
// Process response
}
controller.go
The controller.go file contains the controller layer of the application, responsible for handling HTTP requests and responses. It can delegate business logic to the service layer and is responsible for constructing the HTTP responses.
While there are no specific performance-related changes, make sure that proper error handling and response construction should be managed here to ensure the efficient operation of the application.
package main
type Controller struct {
service Service
}
// NewController initializes a new controller with the given service.
func NewController(service Service) Controller {
return Controller{
service: service,
}
}
// Define your handler functions here. This is where you will handle HTTP requests and responses.
// The controller is responsible for handling HTTP requests and can delegate business logic to the service layer.
// Proper error handling and response construction should be managed here.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论