英文:
Extracting dictionary values into .txt files
问题
from tqdm.auto import tqdmtext_data = []file_count = 0for sample in tqdm(new_dict):# Remove newline characters from each sample as we need to use them exclusively as separatorssample_text = sample['text'].replace('\n', ' ')text_data.append(sample_text)if len(text_data) == 5_000:# Once we reach the 5K mark, save to a filewith open(f'file_path/oscar_data/oscar_{file_count}.txt', 'w', encoding='utf-8') as fp:fp.write('\n'.join(text_data))text_data = []file_count += 1
英文:
I am looking to create .txt files from a dictionary, extracting text into new lines of each txt file - dictionary structure looks like:
{'id': 0,'text': 'Mtendere Village was inspired by the vision'}
I am using this code:
from tqdm.auto import tqdm #loading bartext_data = []file_count = 0for sample in tqdm(new_dict):# remove newline characters from each sample as we need to use exclusively as seperatorssample = sample['text'].replace('\n', '\s')text_data.append(sample)if len(text_data) == 5_000:# once we hit the 5K mark, save to filewith open('file_path\oscar_data\oscar_%s.txt' %file_count, 'w', encoding='utf-8') as fp:fp.write('\n'.join(text_data))text_data = []file_count += 1
However this gives me an error;
---> 12 sample = sample['text'].replace('\n', '\s')TypeError: 'int' object is not subscriptable
Although I understand what the error is telling me, I'm not sure how to correct it...
答案1
得分: 0
我认为您正在尝试将一个字典列表传递给循环，但实际上传递了一个字典。
from tqdm.auto import tqdm #加载进度条new_dict = [{'id': 0,'text': 'Mtendere Village was inspired by the vision'}]text_data = []file_count = 0for sample in tqdm(new_dict):# 从每个样本中移除换行符，因为我们需要将其专门用作分隔符sample = sample['text'].replace('\n', ' ')text_data.append(sample)if len(text_data) == 5000:# 一旦达到5K标记，将其保存到文件with open('file_path\\oscar_data\\oscar_%s.txt' %file_count, 'w', encoding='utf-8') as fp:fp.write('\n'.join(text_data))text_data = []file_count += 1
我已将new_dict更新为一个字典列表，并修复了这个问题。
英文:
I think you're trying to pass a list of dictionaries to the loop, but actually passed a dictionary.
from tqdm.auto import tqdm #loading barnew_dict = [{'id': 0,'text': 'Mtendere Village was inspired by the vision'}]text_data = []file_count = 0for sample in tqdm(new_dict):# remove newline characters from each sample as we need to use exclusively as seperatorssample = sample['text'].replace('\n', '\s')text_data.append(sample)if len(text_data) == 5000:# Once we hit the 5K mark, save it to filewith open('file_path\oscar_data\oscar_%s.txt' %file_count, 'w', encoding='utf-8') as fp:fp.write('\n'.join(text_data))text_data = []file_count += 1
I have updated new_dict to a list of dictionaries and it fixed the issue.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论