如何使用Node.JS检测PostgreSQL数据库中的更改。

huangapple go评论69阅读模式
英文:

How to detect changes in a PostgreSQL database with Node.JS

问题

目前我正在使用 node-schedule 中的计时器来检查数据库中的更改,每隔1分钟进行查询,然后与已注册的信息进行比较,以查看自上次检查以来是否有更改。

我想要做的是减少不必要的查询量,能够实时获取数据,这很有意义,因为我正在使用这个计时器来提供有关用户、类别和文章数量的统计信息。

我在我的数据库中使用 Knex 作为查询构建器。

这是我的计时器文件(我正在使用 MongoDB 存储这些信息):

const schedule = require('node-schedule')

module.exports = app => {
    schedule.scheduleJob('*/1 * * * *', async function () {
        const usersCount = await app.db('users').count('id').first()
        const categoriesCount = await app.db('categories').count('id').first()
        const articlesCount = await app.db('articles').count('id').first()

        const { Stat } = app.api.stat

        const lastStat = await Stat.findOne({}, {},
            { sort: { 'createdAt': -1 } })

        const stat = new Stat({
            users: usersCount.count,
            categories: categoriesCount.count,
            articles: articlesCount.count,
            createdAt: new Date()
        })

        const hasChanged = !lastStat
            || stat.users !== lastStat.users
            || stat.categories !== lastStat.categories
            || stat.articles !== lastStat.articles;

        if (hasChanged) {
            stat.save().then(() => console.log('[STATS] Estatísticas atualizadas!'))
        }
    })
}

我尝试了这个:

const config = require('../knexfile')
const knex = require('knex')(config)

knex.migrate.latest([config])

knex.raw(`
    CREATE or REPLACE FUNCTION public.notify_trigger() RETURNS trigger  $$
    DECLARE
        BEGIN
            PERFORM pg_notify( CAST('update_notification' AS text), row_to_json(NEW)::text );
            RETURN new;
        END;
    $$ LANGUAGE plpgsql;
    
    CREATE TRIGGER update_notification
    AFTER INSERT ON categories
    FOR EACH ROW
    EXECUTE FUNCTION public.notify_trigger();
`)

const { Client } = require('pg')

const client = new Client({
    connectionString: 'postgres://*********:*********@localhost/**********'
})

client.connect((err, client, done) => {
    if (err) {
        const msg = 'Error in connecting database: ' + err
        console.log('\x1b[41m\x1b[37m ' + msg + ' \x1b[0m')
    } else {
        const msg = 'Success in connecting database'
        console.log('\x1b[42m\x1b[37m ' + msg + ' \x1b[0m')
        client.on('notification', (msg) => {
            console.log(msg.payload)
        })
        const query = client.query('LISTEN update_notification')
    }
})

module.exports = knex
英文:

Currently I'm using a timer with node-schedule to check for changes in the database, making queries every 1 minute and then comparing with the information already registered to see if there were changes since the last check.

With what I want to do I want to decrease the amount of unnecessary queries and be able to bring the data in real time, and this is interesting because I am using this timer to provide statistics on the amount of users, categories and articles.

I am using Knex as query-builder in my db.

This is my file with the timer (I am using MongoDB to store this information):

const schedule = require('node-schedule')

module.exports = app => {
    schedule.scheduleJob('*/1 * * * *', async function () {
        const usersCount = await app.db('users').count('id').first()
        const categoriesCount = await app.db('categories').count('id').first()
        const articlesCount = await app.db('articles').count('id').first()

        const { Stat } = app.api.stat

        const lastStat = await Stat.findOne({}, {},
            { sort: { 'createdAt': -1 } })
        
        const stat = new Stat({
            users: usersCount.count,
            categories: categoriesCount.count,
            articles: articlesCount.count,
            createdAt: new Date()
        })

        const hasChanged = !lastStat
            || stat.users !== lastStat.users
            || stat.categories !== lastStat.categories
            || stat.articles !== lastStat.articles;
        
        if (hasChanged) {
            stat.save().then(() => console.log('[STATS] Estatísticas atualizadas!'))
        }
    })
}

I tried this:

const config = require('../knexfile')
const knex = require('knex')(config)

knex.migrate.latest([config])

knex.raw(`
    CREATE or REPLACE FUNCTION public.notify_trigger() RETURNS trigger  $$
    DECLARE
        BEGIN
            PERFORM pg_notify( CAST('update_notification' AS text), row_to_json(NEW)::text );
            RETURN new;
        END;
    $$ LANGUAGE plpgsql;
    
    CREATE TRIGGER update_notification
    AFTER INSERT ON categories
    FOR EACH ROW
    EXECUTE FUNCTION public.notify_trigger();
`)

const { Client } = require('pg')

// The file contains the correct information
const client = new Client({
    // user: '**********',
    // host: 'localhost',
    // database: '*********',
    // password: '******************',
    // port: 5432,
    connectionString: 'postgres://*********:*********@localhost/**********'
})

client.connect((err, client, done) => {
    if (err) {
        const msg = 'Error in connecting database: ' + err
        console.log('\x1b[41m\x1b[37m ' + msg + ' \x1b[0m')
    } else {
        const msg = 'Success in connecting database'
        console.log('\x1b[42m\x1b[37m ' + msg + ' \x1b[0m')
        client.on('notification', (msg) => {
            console.log(msg.payload)
        })
        const query = client.query('LISTEN update_notification')
    }
})

module.exports = knex

答案1

得分: 2

Postgres支持 LISTENNOTIFY 命令,可用于通知其他应用程序数据库中的更改。这些可用于避免轮询。

socket.io 的 postgres-adapter 支持通过 Websockets 进行通知。

这个 视频 也可能会有帮助。

英文:

Postgres supports LISTEN and NOTIFY commands that can be used to inform other applications of changes in the db. These can be used to avoid polling.

socket.io's postgres-adapter supports notifications over websockets.

This video may also be helpful.

答案2

得分: 0

你可以将统计数据存储在MongoDB中,并使用 watch 监视更改。

https://www.mongodb.com/docs/drivers/node/current/usage-examples/changeStream/

PS:我建议这样做,因为你提到你正在使用MongoDB来存储统计数据。

英文:

You can store the statistics in mongodb, and use watch to monitor changes.

https://www.mongodb.com/docs/drivers/node/current/usage-examples/changeStream/

PS: suggesting this because you mentioned you are using mongodb to store the statistics.

huangapple
  • 本文由 发表于 2023年5月24日 21:38:24
  • 转载请务必保留本文链接:https://go.coder-hub.com/76324168.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定