Llama_index问题出现在HTTP请求之后。

huangapple
go评论104阅读模式
英文:

Llama_index issue behind HTTP request

问题

我遇到了一个问题，使用Llama_Index来使用先前为自定义内容ChatGPT查询生成的索引。

我使用以下代码生成了索引：

  1. from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTSimpleVectorIndex, LLMPredictor, PromptHelper, ServiceContext
  2. from langchain import OpenAI
  3. import sys
  4. import os
  6. def construct_index():
  7.     # 设置最大输入大小
  8.     max_input_size = 40960
  9.     # 设置输出令牌的数量
  10.     num_outputs = 20000
  11.     # 设置最大块重叠
  12.     max_chunk_overlap = 20
  13.     # 设置块大小限制
  14.     chunk_size_limit = 600
  16.     # 定义提示助手
  17.     prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)
  19.     # 定义LLM
  20.     llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="text-davinci-003", max_tokens=num_outputs))
  22.     documents = SimpleDirectoryReader("./data").load_data()
  24.     service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
  25.     index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)
  27.     index.save_to_disk('./index.json')
  29.     return index

要使用查询，我使用以下代码：

  1. def ask_ai(query):
  2.     index = GPTSimpleVectorIndex.load_from_disk('./index.json')
  3.     response = index.query(query)
  5.     return response.response

这是一段常用的代码，实际上在虚拟环境中运行时（py ./index.py）是有效的，只需添加一行来默认调用construct_index或ask_ai方法。

然而，当我尝试将其放在HTTP请求后面，例如在AWS Lambda或Flask API中，ask_ai 在load_from_disk方法中失败，两种方法都出现相同的错误：

  1. ERROR:app:Exception on /api/query [POST]
  2. Traceback (most recent call last):
  3.   File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\flask\app.py", line 2190, in wsgi_app
  4.     response = self.full_dispatch_request()
  5.   ...
  6. KeyError: 'docs'

对于Flask的实现，我从这里获取了代码：

https://github.com/bbachi/python-flask-restapi

只需在AppService中留下一个方法（ask_ai），然后从控制器中的一个路由中调用它。AWS容器化Lambda的Docker文件如下：

  1. FROM public.ecr.aws/lambda/python:3.10
  3. COPY requirements.txt ./
  4. RUN pip3 install -r requirements.txt
  5. COPY myfunction.py ./
  6. COPY index.json ./
  8. CMD ["myfunction.lambda_handler"]

在这两种情况下，index.json 包含上面提供的代码生成的索引，而对于Lambda，myfunction.py 包含 lambda_handler 方法后面的 ask_ai 方法实现。

是否有人遇到过这样的问题？非常感谢提前提供的任何帮助！

英文:

I'm having an issue using Llama_Index to use an index previously generated for custom content ChatGPT queries.

I generated the index with the following code:

  1. from llama_index import SimpleDirectoryReader, GPTListIndex, readers, GPTSimpleVectorIndex, LLMPredictor, PromptHelper, ServiceContext
  2. from langchain import OpenAI
  3. import sys
  4. import os
  6. def construct_index():
  7.     # set maximum input size
  8.     max_input_size = 40960
  9.     # set number of output tokens
  10.     num_outputs = 20000
  11.     # set maximum chunk overlap
  12.     max_chunk_overlap = 20
  13.     # set chunk size limit
  14.     chunk_size_limit = 600
  16.     # define prompt helper
  17.     prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)
  19.     # define LLM
  20.     llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="text-davinci-003", max_tokens=num_outputs))
  22.     documents = SimpleDirectoryReader("./data").load_data()
  24.     service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
  25.     index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)
  27.     index.save_to_disk('./index.json')
  29.     return index

To use query something the code I use is the following:

  1. def ask_ai(query):
  2.     index = GPTSimpleVectorIndex.load_from_disk('./index.json')
  3.     response = index.query(query)
  5.     return response.response

This is a popular code that in fact works when I run it in a virtual environment (py ./index.py) adding a line to call by default the construct_index or ask_ai methods.

However, when I try to put it behind an HTTP request, like in an AWS Lambda or Flask API, the ask_ai fails in the load_from_disk method for both approaches with the same error:

  1. ERROR:app:Exception on /api/query [POST]
  2. Traceback (most recent call last):
  3.   File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\flask\app.py", line 2190, in wsgi_app
  4.     response = self.full_dispatch_request()
  5.   File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\flask\app.py", line 1486, in full_dispatch_request
  6.     rv = self.handle_user_exception(e)
  7.   File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\flask\app.py", line 1484, in full_dispatch_request
  8.     rv = self.dispatch_request()
  9.   File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\flask\app.py", line 1469, in dispatch_request
  10.     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  11.   File "C:\pmi\python-flask-restapi\app.py", line 17, in qyery_chatgpt
  12.     return appService.ask_ai(query)
  13.   File "C:\pmi\python-flask-restapi\app_service.py", line 10, in ask_ai
  14.     index = GPTSimpleVectorIndex.load_from_disk('index.json')
  15.   File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\llama_index\indices\base.py", line 352, in load_from_disk
  16.     return cls.load_from_string(file_contents, **kwargs)
  17.   File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\llama_index\indices\base.py", line 328, in load_from_string
  18.     return cls.load_from_dict(result_dict, **kwargs)
  19.   File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\llama_index\indices\vector_store\base.py", line 260, in load_from_dict
  20.     return super().load_from_dict(result_dict, **config_dict, **kwargs)
  21.   File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\llama_index\indices\base.py", line 304, in load_from_dict
  22.     docstore = DocumentStore.load_from_dict(result_dict[DOCSTORE_KEY])
  23.   File "C:\pmi\python-flask-restapi\flaskapi\lib\site-packages\llama_index\docstore.py", line 72, in load_from_dict
  24.     for doc_id, doc_dict in docs_dict["docs"].items():
  25. KeyError: 'docs'

For the implementation of Flask I took it from here:

https://github.com/bbachi/python-flask-restapi

Just by leaving a single method in the AppService (the ask_ai) and invoking it from a single route in the controller. The docker file for the AWS container-based lambda is:

  1. FROM public.ecr.aws/lambda/python:3.10
  3. COPY requirements.txt ./
  4. RUN pip3 install -r requirements.txt
  5. COPY myfunction.py ./
  6. COPY index.json ./
  8. CMD ["myfunction.lambda_handler"]

In both cases the index.json holds the index generated with the above-provided code and for the lambda the myfunction.py contains the ask_ai method implementation behind the lambda_handler method.

Has someone faced such issue? any help is much appreciated in advance!

答案1

得分: 0

我使用Chat GPT来检查这个问题，它帮助我找到了原因，实际上这是我的一个大错误，因为我愚蠢地使用了与我后来用来加载的Llama_Index不同版本的索引，我没有注意到这一点。

如果有人遇到相同的问题，请尽量使用版本0.5.6。

英文:

I used Chat GPT to examine this issue and it led me to find out the cause, actually it was a big mistake on my side because I dumbly generated the index with a different version of Llama_Index than the one I was later using to load it, I haven't noticed that.

If someone's facing the same issue try to stick to version 0.5.6.

huangapple
  • 本文由 发表于 2023年5月28日 03:17:14
  • 转载请务必保留本文链接：https://go.coder-hub.com/76348628.html
  • flask
  • lambda
  • llama-index
  • openai-api
  • python
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定