英文:
Adding progress monitor while inserting data to a Oracle db using Python
问题
我想在我的Python脚本中添加一个进度条。我正在运行以下代码行。我正在读取一个名为'MRSTY.RRF'的文件,其大小为0.195 MB。我想知道已处理了多少文件。
insert_sty = "insert into MRSTY (CUI,TUI,STN,STY,ATUI,CVF) values (:0,:1,:2,:3,:4,:5)"
records=[]
with open("../umls_files/umls-2023AA-metathesaurus-full/2023AA/META/MRSTY.RRF" , 'r') as f:
for line in f:
line = line.strip()
records.append(line.split("|"))
for sublist in records: #remove '' at the end of every element
if sublist:
sublist.pop()
for i in records:
try:
cur.execute(insert_sty,i)
print ("record inserted")
except Exception as e:
print (i)
print("Error: ",str(e))
conn.commit()
如何实现这一目标?
英文:
I would like to add a progress bar in my python script. I am running following lines of code. I am reading a file 'MRSTY.RRF' which is 0.195 MB. I would like to how much file has been processed.
insert_sty = "insert into MRSTY (CUI,TUI,STN,STY,ATUI,CVF) values (:0,:1,:2,:3,:4,:5)"
records=[]
with open("../umls_files/umls-2023AA-metathesaurus-full/2023AA/META/MRSTY.RRF" , 'r') as f:
for line in f:
line = line.strip()
records.append(line.split("|"))
for sublist in records: #remove '' at the end of every element
if sublist:
sublist.pop()
for i in records:
try:
cur.execute(insert_sty,i)
print ("record inserted")
except Exception as e:
print (i)
print("Error: ",str(e))
conn.commit()
How can I achieve this?
答案1
得分: 2
你可以使用 tqdm
来实现这一点,它可以在长时间计算过程中显示进度条,非常简单。
我已经修改了你的代码来加入它。
from tqdm import tqdm
insert_sty = "insert into MRSTY (CUI,TUI,STN,STY,ATUI,CVF) values (:0,:1,:2,:3,:4,:5)"
records=[]
file_path = "../umls_files/umls-2023AA-metathesaurus-full/2023AA/META/MRSTY.RRF"
num_lines = sum(1 for line in open(file_path))
with open(file_path, 'r') as f:
for line in tqdm(f, total=num_lines, desc="Processing file"):
line = line.strip()
records.append(line.split("|"))
for sublist in records:
if sublist:
sublist.pop()
for i in tqdm(records, desc="Inserting records"):
try:
cur.execute(insert_sty,i)
print ("record inserted")
except Exception as e:
print (i)
print("Error: ",str(e))
conn.commit()
英文:
You could do that with tqdm
, it display a progress bar during long computations, and is very simple.
I have modified your code to add it.
from tqdm import tqdm
insert_sty = "insert into MRSTY (CUI,TUI,STN,STY,ATUI,CVF) values (:0,:1,:2,:3,:4,:5)"
records=[]
file_path = "../umls_files/umls-2023AA-metathesaurus-full/2023AA/META/MRSTY.RRF"
num_lines = sum(1 for line in open(file_path))
with open(file_path, 'r') as f:
for line in tqdm(f, total=num_lines, desc="Processing file"):
line = line.strip()
records.append(line.split("|"))
for sublist in records:
if sublist:
sublist.pop()
for i in tqdm(records, desc="Inserting records"):
try:
cur.execute(insert_sty,i)
print ("record inserted")
except Exception as e:
print (i)
print("Error: ",str(e))
conn.commit()
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论