英文:
ImportError: cannot import name 'CustomLLM' from 'llama_index.llms'
问题
我在使用 llama_index 时遇到了困难。我想加载一个自定义的LLM来使用它。幸运的是，他们在文档中提供了我需要的示例，但不幸的是，它不起作用！
在他们的示例中，有以下导入部分：
from llama_index.llms import CustomLLM, CompletionResponse, LLMMetadata
但当我运行它时，我会得到以下错误：
ImportError: 无法从 'llama_index.llms' 导入名称 'CustomLLM'
我的 llama_index 版本是0.7.1（当前最新版本）。你知道有没有任何解决办法让我在 llama_index 中使用自定义数据集？
P.S. 如果需要完整的代码，这里是：
import torchfrom transformers import pipelinefrom typing import Optional, List, Mapping, Anyfrom llama_index import (ServiceContext,SimpleDirectoryReader,LangchainEmbedding,ListIndex)from llama_index.llms import CustomLLM, CompletionResponse, LLMMetadata# 设置上下文窗口大小context_window = 2048# 设置输出令牌的数量num_output = 256# 在LLM类之外存储管道/模型以避免内存问题model_name = "facebook/opt-iml-max-30b"pipeline = pipeline("text-generation", model=model_name, device="cuda:0", model_kwargs={"torch_dtype":torch.bfloat16})class OurLLM(CustomLLM):@propertydef metadata(self) -> LLMMetadata:"""获取LLM元数据。"""return LLMMetadata(context_window=context_window, num_output=num_output)def complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:prompt_length = len(prompt)response = pipeline(prompt, max_new_tokens=num_output)[0]["generated_text"]# 仅返回新生成的令牌text = response[prompt_length:]return CompletionResponse(text=text)def stream_complete(self, prompt: str, **kwargs: Any) -> CompletionResponseGen:raise NotImplementedError()# 定义我们的LLMllm = OurLLM()service_context = ServiceContext.from_defaults(llm=llm,context_window=context_window,num_output=num_output)# 加载您的数据documents = SimpleDirectoryReader('./data').load_data()index = ListIndex.from_documents(documents, service_context=service_context)# 查询并打印响应query_engine = index.as_query_engine()response = query_engine.query("<query_text>")print(response)
英文:
I'm having difficulties to work with llama_index. I want to load a custom LLM to use it. Fortunately, they have the exact example for my need on their documentation, unfortunately, it does not work!
They have these imports in their example:
from llama_index.llms import CustomLLM, CompletionResponse, LLMMetadata
And when I run it I'll get this error:
ImportError: cannot import name 'CustomLLM' from 'llama_index.llms'
My llama_index version is 0.7.1 (the last current version). Do you know any workaround for me to use a custom dataset in llama_index?
P.S. If their full code is needed here it is:
import torchfrom transformers import pipelinefrom typing import Optional, List, Mapping, Anyfrom llama_index import (ServiceContext,SimpleDirectoryReader,LangchainEmbedding,ListIndex)from llama_index.llms import CustomLLM, CompletionResponse, LLMMetadata# set context window sizecontext_window = 2048# set number of output tokensnum_output = 256# store the pipeline/model outisde of the LLM class to avoid memory issuesmodel_name = "facebook/opt-iml-max-30b"pipeline = pipeline("text-generation", model=model_name, device="cuda:0", model_kwargs={"torch_dtype":torch.bfloat16})class OurLLM(CustomLLM):@propertydef metadata(self) -> LLMMetadata:"""Get LLM metadata."""return LLMMetadata(context_window=context_window, num_output=num_output)def complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:prompt_length = len(prompt)response = pipeline(prompt, max_new_tokens=num_output)[0]["generated_text"]# only return newly generated tokenstext = response[prompt_length:]return CompletionResponse(text=text)def stream_complete(self, prompt: str, **kwargs: Any) -> CompletionResponseGen:raise NotImplementedError()# define our LLMllm = OurLLM()service_context = ServiceContext.from_defaults(llm=llm,context_window=context_window,num_output=num_output)# Load the your datadocuments = SimpleDirectoryReader('./data').load_data()index = ListIndex.from_documents(documents, service_context=service_context)# Query and print responsequery_engine = index.as_query_engine()response = query_engine.query("<query_text>")print(response)
答案1
得分: 1
需要更改您的导入库。
将
from llama_index.llms import CustomLLM, CompletionResponse, LLMMetadata
更改为
from llama_index.llms.custom import CustomLLMfrom llama_index.llms.base import CompletionResponse, LLMMetadata
英文:
You need to change your import library
Change
from llama_index.llms import CustomLLM, CompletionResponse, LLMMetadata
To this
from llama_index.llms.custom import CustomLLMfrom llama_index.llms.base import CompletionResponse, LLMMetadata
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论