When to Open/Close Connection with Node-Postgres and Serverless

huangapple go评论73阅读模式
英文:

When to Open/Close Connection with Node-Postgres and Serverless

问题

I understand that you want the code part to be translated, and you have a question about managing pgPool in a Serverless Framework API while migrating from AWS Aurora PG to Heroku PG. Managing the connection and queries in a serverless environment can be tricky. Here's a translation of your issue:

我明白您想要将代码部分翻译,并且您在将AWS Aurora PG迁移到Heroku PG时对在Serverless Framework API中管理pgPool有疑问。在无服务器环境中管理连接和查询可能有些棘手。以下是您问题的翻译:

在 Serverless 框架中管理数据库连接池 pgPool 以及查询可能会涉及一些挑战。您的代码似乎在每次查询之前都需要等待 await setupPgPool 并且在结束时调用 await pgPool.end()。这可能导致问题,因为每次都要等待连接池的设置可能会降低性能。您还遇到了 Cannot use a pool after calling end on the pool 的错误。

最佳做法通常取决于您的应用程序需求,但在 Serverless 环境中,重用连接池通常是有效的。您可以考虑在处理程序内部设置 pgPool 并将其作为参数传递给您的模型,以避免每次查询都要等待连接池的设置。另外,确保只在最后一次查询后调用 pgPool.end(),以避免关闭连接池太早。

最后,建议您检查 AWS Lambda 和 Heroku Postgres 文档,以确保遵循最佳做法和性能优化,特别是在 Serverless 环境中管理数据库连接。

英文:

I'm migrating from AWS Aurora PG to Heroku PG and I'm currently refactoring my queries. Using the Node-Postgres library in my Serverless Framework API, I now have to manage opening and closing the connection, something I didn't have to do with the AWS-SDK.

I am following this article to setup my queries. Taken from the article:

let pgPool;

const setupPgPool = () => {
  // pgConfig is the host/user/password/database object
  pgPool = new pg.Pool(pgConfig);
};

module.exports.hello = async () => {
  if (!pgPool) {
    // "Cold start". Get Heroku Postgres creds and create connection pool.
    await setupPgPool();
  }
  // Else, backing container "warm". Use existing connection pool.

  try {
    const result = await pgPool.query('SELECT now()');

    // Response body must be JSON.
    return {
      statusCode: 200,
      body: JSON.stringify({
        output: {
          currTimePg: result.rows[0].now,
        },
      }),
    };
  } catch (e) {
    // Return error message in response body for easy debugging.
    // INSECURE - CHANGE FOR PROD
    return {
      statusCode: 500,
      body: JSON.stringify({
        error: e.message,
      }),
    };
  }
};

I'm trying to solve the issue of where to put the pgPool. My folder structure is like so:

src/
  controllers/
    auth/
      createUser.js
  models/
    authModel.js
    userModel.js

To dumb down my create createUser handler let's say it looks like this:

module.exports = (App) => {
  App.controllers.createAuth = async (event, context, cb) => {
    const body = JSON.parse(event.body)
    const { email, password } = body

    try {

      const hash = createHash(password)
        
      // Auth is a class that contains all my auth table read/write functions to the db
      const Auth = new App.Auth()
      const authRecordId = await Auth.createAuth(email, hash)

      // User is a class that contains all my user table read/write functions to the db
      const User = new App.User()
      await User.createUser(authRecordId)

      const token = createToken(authRecordId)

      return cb(null, utils.res(200, { token }))

    } catch (error) {
      return cb(null, utils.res(500, { message: 'There was an error. Please try again' } ))
    }
  }
}

Inside my authModel I have:

let pgPool

const setupPgPool = () => {
  // pgConfig is the host/user/password/database object
  pgPool = new pg.Pool(pgConfig)
}

Auth.prototype.createAuth = async (email, hash) => {
  const sql = `
    INSERT INTO auth (email, hash)
    VALUES (lower($1), $2)
    RETURNING id;
  `
  const values = [ email, hash ]

  try {
    if (!pgPool) {
      await setupPgPool()
    }

    const { rows } = await pgPool.query(sql, values)
    await pgPool.end()
    return rows[0].id
  } catch (err) {
    throw err
  }
}

The userModal query looks very similar (setup pgPool, run query, end connection, return data). The issue is pgPool never exists and I always need to await it before running the query. I also need to run await pgPool.end() otherwise my queries will hang. Is this the best practice, as the article suggests, in a Serverless Framework API?

Lastly, should I instead open the connection my handler and pass pgPool as an argument to my models? That way if I have to make more than one query to my database I don't have to await setupPgPool every time I want to read/write from the database?

EDIT: I was refactoring my first handler as I was writing this. After implementing what I used in the post, I am now running into this error: Cannot use a pool after calling end on the pool. Seems to be because I am closing the connection.

EDIT 2: Commented out the await pgPool.end() lines and now my calls hang again.. not really sure what to do..

答案1

得分: 0

我通过以下方式解决了我的问题:

  • 我开始使用"serverless-postgres"而不是"pg-node"
  • 但我认为这是真正的原因,我停止使用回调参数

我简直不知道为什么这对其他人有效,但对我无效。

  1. 启动了一个全新的sls项目,并让它返回"hello"
  2. 安装了serverless-postgres,连接到我的数据库,但仍然返回"hello",没有任何错误
  3. 运行这段代码:
await client.connect();
const result = await client.query(`SELECT 1+1 AS result`);
await client.clean();
return {
  body: JSON.stringify({ message: result.rows[0] }),
  statusCode: 200
}
  1. 尝试在我的项目上运行该代码,但没有成功。
  2. 比较了我的处理程序与这个新项目的处理程序,意识到它们只是返回,而不使用第三个参数,callback

不确定Serverless何时放弃了callback并直接返回(我想我一直没有跟上最新动态),但包装在callback中会导致问题。简单地返回并删除回调函数修复了一切。

如果你也遇到问题,我建议你按照我所做的方式操作。先使外部版本的基本功能正常运行,然后尝试逐渐将其集成到你的应用程序中。

英文:

I solved my issue by the following:

  • I started using "serverless-postgres" instead of "pg-node"
  • But, I think this is the real reason, I STOPPED USING THE CALLBACK PARAMETER

I was going insane wondering how this was working for everyone else but me.

  1. Started an entire new sls project and got it to return "hello"
  2. Installed serverless-postgres, connected to my database but still return hello without any errors
  3. Run this code:
  await client.connect();
  const result = await client.query(`SELECT 1+1 AS result`);
  await client.clean();
  return {
    body: JSON.stringify({ message: result.rows[0] }),
    statusCode: 200
  }
  1. Tried to run that code on my project. Didn't work.
  2. Compared my handler to this new projects handler and realized they just return instead of using the third parameter, callback.

Not sure when Serverless did away with callback and just returned (I guess I haven't been keeping up with the latest) but wrapping with callback was causing issues. Simply returning and removing callback fixed everything.

If you are having issues too I recommend you follow what I did. Get an external version of the basics working and try to slowly integrate it into your app.

huangapple
  • 本文由 发表于 2023年4月6日 21:40:05
  • 转载请务必保留本文链接:https://go.coder-hub.com/75950215.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定