英文:
FAISS IndexHNSW Search throws an error on Linux but works on Windows (when installing from pip)
问题
我使用faiss在Windows上训练了一个HSMW Index。我希望将其部署到Azure函数应用程序,因此我只是使用pip
安装了faiss-cpu==1.7.3
,而不是建议的使用conda
安装。
搜索最相似向量的代码是
distances, indices = index.search(vector, k)
在Windows上,Python 3.10.4,faiss-cpu==1.7.3上运行正常。
然而,在Azure函数应用程序和Ubuntu上,它抛出以下错误。
File "/home/vboxuser/Documents/VectorDB/src/match_vector.py", line 30, in get_matching_vectors
distances, indices = index.search(vector, k)
File "/home/vboxuser/Documents/VectorDB/.venv/lib/python3.10/site-packages/faiss/__init__.py", line 322, in replacement_search
self.search_c(n, swig_ptr(x), k, swig_ptr(D), swig_ptr(I))
File "/home/vboxuser/Documents/VectorDB/.venv/lib/python3.10/site-packages/faiss/swigfaiss.py", line 5436, in search
return _swigfaiss.IndexHNSW_search(self, n, x, k, distances, labels)
TypeError: in method 'IndexHNSW_search', argument 3 of type 'float const *'
我想了解是什么原因导致了这个错误,以及是否有一种解决方法,让我能够在Azure函数应用程序中运行它。
谢谢
英文:
I trained a HSMW Index using faiss on Windows. I wanted to deploy this to an Azure Function App, hence I just pip
to install faiss-cpu==1.7.3
rather than the recommended install with conda
.
The code that searches for the most similar vector is
distances, indices = index.search(vector, k)
works without issue on Windows, Python 3.10.4, faiss-cpu==1.7.3
However, in the Azure Function App and also on Ubuntu, it throws the following error.
File "/home/vboxuser/Documents/VectorDB/src/match_vector.py", line 30, in get_matching_vectors
distances, indices = index.search(vector, k)
File "/home/vboxuser/Documents/VectorDB/.venv/lib/python3.10/site-packages/faiss/__init__.py", line 322, in replacement_search
self.search_c(n, swig_ptr(x), k, swig_ptr(D), swig_ptr(I))
File "/home/vboxuser/Documents/VectorDB/.venv/lib/python3.10/site-packages/faiss/swigfaiss.py", line 5436, in search
return _swigfaiss.IndexHNSW_search(self, n, x, k, distances, labels)
TypeError: in method 'IndexHNSW_search', argument 3 of type 'float const *'
I would love to understand what causes this and whether there is a work around so that I can run it in the Azure Function App.
Thanks
答案1
得分: 1
根据这个 GitHub 问题的解决方案,在这里查看:TypeError: in method 'IndexFlat_add', argument 3 of type 'float const *' · Issue #461 · facebookresearch/faiss · GitHub 确保生成的向量类型或数组是 numpy.float32。
Code 1:-
import numpy as np
d = 128
n = 10000
xb = np.random.rand(n, d).astype(np.float32)
vector = [0.5] * d
vector = np.array(vector).astype(np.float32)
distances = np.linalg.norm(xb - vector, axis=1)
k = 5
indices = np.argsort(distances)[:k]
print("Distances:", distances[indices])
print("Indices:", indices)
Output:-
Code 2:-
import numpy as np
d = 128
n = 10000
x = np.random.rand(n, d).astype(np.float32)
print(x)
Output:-
英文:
> According to the solution in this github issue, Refer here:- TypeError: in method 'IndexFlat_add', argument 3 of type
> 'float const *' · Issue #461 · facebookresearch/faiss ·
> GitHub Make
> sure the generated vector types or array is of numpy.float32
Code 1:-
import numpy as np
d = 128
n = 10000
xb = np.random.rand(n, d).astype(np.float32)
vector = [0.5] * d
vector = np.array(vector).astype(np.float32)
distances = np.linalg.norm(xb - vector, axis=1)
k = 5
indices = np.argsort(distances)[:k]
print("Distances:", distances[indices])
print("Indices:", indices)
Output:-
Code 2:-
import numpy as np
d = 128
n = 10000
x = np.random.rand(n, d).astype(np.float32)
print (x)
Output:-
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论