英文:
getting a error when running GPTNeoXForCausalLM from Transformers Library: NameError: name 'init_empty_weights' is not defined
问题
I'm trying to run OpenAssistant's pythia-12b model but I'm getting the following error:
NameError: name 'init_empty_weights' is not defined
I have Accelerate installed, and I'm running Transformers version 4.25.1
from transformers import AutoModelForCausalLM, GPTNeoXForCausalLM, AutoTokenizer
Model = "OpenAssistant/oasst-sft-1-pythia-12b"
tokenizer = AutoTokenizer.from_pretrained(Model, cache_dir='models_hf')
model = GPTNeoXForCausalLM.from_pretrained(Model, device_map="auto", load_in_8bit=True, cache_dir='models_hf', low_cpu_mem_usage=True)
message = "Hello, How are you?"
inp = "<|prompter|>"+message+"<|endoftext|><|assistant|>"
data = tokenizer([inp], return_tensors="pt")
data = {k: v.to(model.device) for k, v in data.items() if k in ("input_ids", "attention_mask")}
outputs = model.generate(**data)
print(tokenizer.decode(outputs[0]))
英文:
I'm trying to run OpenAssistant's pythia-12b model but I'm getting the following error:
NameError: name 'init_empty_weights' is not defined
I have Accelerate installed, and I'm running Transformers version 4.25.1
from transformers import AutoModelForCausalLM, GPTNeoXForCausalLM, AutoTokenizer
Model = "OpenAssistant/oasst-sft-1-pythia-12b"
tokenizer = AutoTokenizer.from_pretrained(Model, cache_dir='models_hf')
model = GPTNeoXForCausalLM.from_pretrained(Model, device_map="auto", load_in_8bit=True, cache_dir='models_hf',low_cpu_mem_usage=True)
message = "Hello, How are you?"
inp = "<|prompter|>"+message+"<|endoftext|><|assistant|>"
data = tokenizer([inp], return_tensors="pt")
data = {k: v.to(model.device) for k, v in data.items() if k in ("input_ids", "attention_mask")}
outputs = model.generate(**data)
print(tokenizer.decode(outputs[0]))
答案1
得分: 2
在我的情况下，安装加速包并重新启动运行时就足够了。相同的问题也在这里讨论过（链接：https://github.com/NielsRogge/Transformers-Tutorials/issues/131）。
英文:
In my case, installing accelerate package and re-starting the runtime was enough.
The same issue was also discussed here.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论