在使用Python将数据插入Oracle数据库时添加进度监视器。

huangapple
go评论95阅读模式
英文:

Adding progress monitor while inserting data to a Oracle db using Python

问题

我想在我的Python脚本中添加一个进度条。我正在运行以下代码行。我正在读取一个名为'MRSTY.RRF'的文件，其大小为0.195 MB。我想知道已处理了多少文件。

  1. insert_sty = "insert into MRSTY (CUI,TUI,STN,STY,ATUI,CVF) values (:0,:1,:2,:3,:4,:5)"
  2. records=[]
  4. with open("../umls_files/umls-2023AA-metathesaurus-full/2023AA/META/MRSTY.RRF" , 'r') as f:
  5.     for line in f:
  6.         line = line.strip()
  7.         records.append(line.split("|"))
  8.     
  9.     for sublist in records:   #remove '' at the end of every element
  10.         if sublist:
  11.             sublist.pop()
  12.     
  14. for i in records:
  15.     try:
  16.         cur.execute(insert_sty,i)
  17.         print ("record inserted")
  18.         
  19.     except Exception as e:
  20.         print (i)
  21.         print("Error: ",str(e))
  22.         
  23. conn.commit()

如何实现这一目标？

英文:

I would like to add a progress bar in my python script. I am running following lines of code. I am reading a file 'MRSTY.RRF' which is 0.195 MB. I would like to how much file has been processed.

  1. insert_sty = "insert into MRSTY (CUI,TUI,STN,STY,ATUI,CVF) values (:0,:1,:2,:3,:4,:5)"
  2. records=[]
  4. with open("../umls_files/umls-2023AA-metathesaurus-full/2023AA/META/MRSTY.RRF" , 'r') as f:
  5.     for line in f:
  6.         line = line.strip()
  7.         records.append(line.split("|"))
  8.     
  9.     for sublist in records:   #remove '' at the end of every element
  10.         if sublist:
  11.             sublist.pop()
  12.     
  14. for i in records:
  15.     try:
  16.         cur.execute(insert_sty,i)
  17.         print ("record inserted")
  18.         
  19.     except Exception as e:
  20.         print (i)
  21.         print("Error: ",str(e))
  22.         
  23. conn.commit()

How can I achieve this?

答案1

得分: 2

你可以使用 tqdm 来实现这一点，它可以在长时间计算过程中显示进度条，非常简单。

我已经修改了你的代码来加入它。

  1. from tqdm import tqdm
  3. insert_sty = "insert into MRSTY (CUI,TUI,STN,STY,ATUI,CVF) values (:0,:1,:2,:3,:4,:5)"
  4. records=[]
  6. file_path = "../umls_files/umls-2023AA-metathesaurus-full/2023AA/META/MRSTY.RRF"
  7. num_lines = sum(1 for line in open(file_path))
  9. with open(file_path, 'r') as f:
  10.     for line in tqdm(f, total=num_lines, desc="Processing file"):
  11.         line = line.strip()
  12.         records.append(line.split("|"))
  14.     for sublist in records:  
  15.         if sublist:
  16.             sublist.pop()
  18. for i in tqdm(records, desc="Inserting records"):
  19.     try:
  20.         cur.execute(insert_sty,i)
  21.         print ("record inserted")       
  22.     except Exception as e:
  23.         print (i)
  24.         print("Error: ",str(e))
  26. conn.commit()
英文:

You could do that with tqdm, it display a progress bar during long computations, and is very simple.

I have modified your code to add it.

  1. from tqdm import tqdm
  3. insert_sty = "insert into MRSTY (CUI,TUI,STN,STY,ATUI,CVF) values (:0,:1,:2,:3,:4,:5)"
  4. records=[]
  6. file_path = "../umls_files/umls-2023AA-metathesaurus-full/2023AA/META/MRSTY.RRF"
  7. num_lines = sum(1 for line in open(file_path))
  9. with open(file_path, 'r') as f:
  10.     for line in tqdm(f, total=num_lines, desc="Processing file"):
  11.         line = line.strip()
  12.         records.append(line.split("|"))
  14.     for sublist in records:  
  15.         if sublist:
  16.             sublist.pop()
  17.     
  18. for i in tqdm(records, desc="Inserting records"):
  19.     try:
  20.         cur.execute(insert_sty,i)
  21.         print ("record inserted")       
  22.     except Exception as e:
  23.         print (i)
  24.         print("Error: ",str(e))
  25.         
  26. conn.commit()

huangapple
  • 本文由 发表于 2023年5月29日 04:08:05
  • 转载请务必保留本文链接：https://go.coder-hub.com/76353403.html
  • oracle
  • python
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定