英文:
Postgresql UPSERT while encrypting column by python encryption
问题
I want to call my encryption_fnc
in DO UPDATE SET
section, while it updates, I also want to do encryption.
我想在“DO UPDATE SET”部分调用我的encryption_fnc
,在更新时也要进行加密。
英文:
I am using BULK UPSERT to insert record into new table while data of column is being encrypting by python lib CryptoDome.
here is what I done
def bulk_upsert_query(data, table_name=None):
values_list = []
add = values_list.append
for i in data:
encrypt_user_id = encryption_fnc(str(i.user_id))
add(f"({i.id},'{encrypt_user_id}','{i.name}')")
upsert_sql = r"""insert into upsert_test_table
(id,encrypt_user_id,name)
values
""" + "\n, ".join(values_list) + """ ON CONFLICT ON CONSTRAINT
"upsert_test_table_pkey"
DO UPDATE SET
name=excluded.name,
encrypt_user_id=excluded.user_id"""
return upsert_sql
data = data from my old table that have user_id as int(ex: 124,345,786)
in upsert_test_table
table id column is primary key column.
this bulk_upsert_query
fnc will encrypt int user_id and append to values_list then create a upsert query.
As per insertion it works as I expected, but when it comes for update if conflict with id column then as you can see I set encrypt_user_id=excluded.user_id
for user_id column,
it update existing encrypted user_id with int user_id(from old table), because I did not put any encryption fnc here,
Update: user_id column is changeable
So what I want is I want to call my encryption_fnc
in DO UPDATE SET
section,
while it update I also want to do encryption.
Can any one tell me who can I achieve this?
Note: I can do encryption in database site by using pgcrypto
but my requirement is do encryption in python side not database side.
答案1
得分: 1
以下是翻译好的部分:
根据评论:
在插入负载中没有user_id
列,因此excluded.user_id
为null
。
ON CONFLICT...DO UPDATE SET encrypt_user_id=excluded.user_id
插入的是null
而不是旧的、未加密的user_id
:演示
CREATE TABLE upsert_test_table
(id SMALLSERIAL,
encrypt_user_id TEXT,
name TEXT,
user_id INT,
CONSTRAINT "upsert_test_table_pkey" PRIMARY KEY (id));
INSERT INTO upsert_test_table VALUES
(1,'enc1','Bob',11);
INSERT INTO upsert_test_table (id,encrypt_user_id,name) VALUES
(1,'enc101','Bob2'),--这里有冲突
(2,'enc2','Ted')
ON CONFLICT ON CONSTRAINT "upsert_test_table_pkey"
DO UPDATE SET
name = excluded.name, --名字将被传入的数据覆盖
encrypt_user_id = excluded.user_id;--null,因为user_id不在传入的数据中
SELECT * FROM upsert_test_table;
-- id | encrypt_user_id | name | user_id
------+-----------------+------+---------
-- 1 | | Bob2 | 11
-- 2 | enc2 | Ted |
从文档中:
> ON CONFLICT DO UPDATE
中的 SET
和 WHERE
子句可以使用表的名称(或别名)访问现有行,并使用特殊的 excluded
表访问插入建议的行。
问题在于excluded
被误解,而被误用为冲突时表中已存在的数据,实际上它是即将插入的数据,作为该插入的有效负载。
只需纠正这个误解,并将新的/即将插入的值称为 excluded
,将旧的/已存在的称为目标表的名称和列即可。
upsert_sql = r"""insert into upsert_test_table
(id,encrypt_user_id,name)
values
""" + "\n, ".join(values_list) + """ ON CONFLICT ON CONSTRAINT
"upsert_test_table_pkey"
DO UPDATE SET
name = upsert_test_table.name, --保留旧的名字
encrypt_user_id = excluded.encrypt_user_id --用新的覆盖
"""
英文:
As per the comment:
there is no user_id
column in that insert payload, so excluded.user_id
is null
.
ON CONFLICT...DO UPDATE SET encrypt_user_id=excluded.user_id
is inserting a null
rather than the old, unencrypted user_id
: demo
CREATE TABLE upsert_test_table
(id SMALLSERIAL,
encrypt_user_id TEXT,
name TEXT,
user_id INT,
CONSTRAINT "upsert_test_table_pkey" PRIMARY KEY (id));
INSERT INTO upsert_test_table VALUES
(1,'enc1','Bob',11);
INSERT INTO upsert_test_table (id,encrypt_user_id,name) VALUES
(1,'enc101','Bob2'),--conflict here
(2,'enc2','Ted')
ON CONFLICT ON CONSTRAINT "upsert_test_table_pkey"
DO UPDATE SET
name = excluded.name, --name will be overwritten by incoming data
encrypt_user_id = excluded.user_id;--null because user_id not in incoming data
SELECT * FROM upsert_test_table;
-- id | encrypt_user_id | name | user_id
------+-----------------+------+---------
-- 1 | | Bob2 | 11
-- 2 | enc2 | Ted |
From the documentation:
> The SET
and WHERE
clauses in ON CONFLICT DO UPDATE
have access to the existing row using the table's name (or an alias), and to the row proposed for insertion using the special excluded
table.
The issue here was that excluded
was misunderstood and instead used as the preexisting data found already in the table upon conflict, while it's actually the data that was coming in, as the payload of that insert.
It should be enough to correct that one misinterpretation and refer to new/incoming/inserted value as excluded
, old/existing by the target table name and column:
upsert_sql = r"""insert into upsert_test_table
(id,encrypt_user_id,name)
values
""" + "\n, ".join(values_list) + """ ON CONFLICT ON CONSTRAINT
"upsert_test_table_pkey"
DO UPDATE SET
name = upsert_test_table.name, --keep the old name
encrypt_user_id = excluded.encrypt_user_id --overwrite with new
"""
答案2
得分: -1
I apologize for any misunderstanding earlier. Here is the translated portion of the text:
抱歉,我刚才理解错了,我以为excluded
表是原始(输入)表,但当我用excluded.user_id
替换为excluded.encrypt_user_id
时,它起作用了,尽管我不知道excluded
表是如何工作的。
upsert_sql = r""""insert into upsert_test_table
(id,encrypt_user_id,name)
values
""" + "\n, ".join(values_list) + """ ON CONFLICT ON CONSTRAINT
"upsert_test_table_pkey"
DO UPDATE SET
name=excluded.name,
encrypt_user_id=excluded.encrypt_user_id""""
英文:
Solved:
Sorry I was stupid, I thought excluded
table is original(input payload) table, but when I replaced excluded.user_id
with excluded.encrypt_user_id
and it works, although have no idea how excluded table works.
upsert_sql = r"""insert into upsert_test_table
(id,encrypt_user_id,name)
values
""" + "\n, ".join(values_list) + """ ON CONFLICT ON CONSTRAINT
"upsert_test_table_pkey"
DO UPDATE SET
name=excluded.name,
encrypt_user_id=excluded.encrypt_user_id"""
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论