FAISS IndexHNSW Search throws an error on Linux but works on Windows (when installing from pip)

huangapple go评论77阅读模式
英文:

FAISS IndexHNSW Search throws an error on Linux but works on Windows (when installing from pip)

问题

我使用faiss在Windows上训练了一个HSMW Index。我希望将其部署到Azure函数应用程序,因此我只是使用pip安装了faiss-cpu==1.7.3,而不是建议的使用conda安装。

搜索最相似向量的代码是

distances, indices = index.search(vector, k)

在Windows上,Python 3.10.4,faiss-cpu==1.7.3上运行正常。

然而,在Azure函数应用程序和Ubuntu上,它抛出以下错误。

  File "/home/vboxuser/Documents/VectorDB/src/match_vector.py", line 30, in get_matching_vectors
    distances, indices = index.search(vector, k)
  File "/home/vboxuser/Documents/VectorDB/.venv/lib/python3.10/site-packages/faiss/__init__.py", line 322, in replacement_search
    self.search_c(n, swig_ptr(x), k, swig_ptr(D), swig_ptr(I))
  File "/home/vboxuser/Documents/VectorDB/.venv/lib/python3.10/site-packages/faiss/swigfaiss.py", line 5436, in search
    return _swigfaiss.IndexHNSW_search(self, n, x, k, distances, labels)
TypeError: in method 'IndexHNSW_search', argument 3 of type 'float const *'

我想了解是什么原因导致了这个错误,以及是否有一种解决方法,让我能够在Azure函数应用程序中运行它。

谢谢

英文:

I trained a HSMW Index using faiss on Windows. I wanted to deploy this to an Azure Function App, hence I just pip to install faiss-cpu==1.7.3 rather than the recommended install with conda.

The code that searches for the most similar vector is

distances, indices = index.search(vector, k)

works without issue on Windows, Python 3.10.4, faiss-cpu==1.7.3

However, in the Azure Function App and also on Ubuntu, it throws the following error.

  File "/home/vboxuser/Documents/VectorDB/src/match_vector.py", line 30, in get_matching_vectors
    distances, indices = index.search(vector, k)
  File "/home/vboxuser/Documents/VectorDB/.venv/lib/python3.10/site-packages/faiss/__init__.py", line 322, in replacement_search
    self.search_c(n, swig_ptr(x), k, swig_ptr(D), swig_ptr(I))
  File "/home/vboxuser/Documents/VectorDB/.venv/lib/python3.10/site-packages/faiss/swigfaiss.py", line 5436, in search
    return _swigfaiss.IndexHNSW_search(self, n, x, k, distances, labels)
TypeError: in method 'IndexHNSW_search', argument 3 of type 'float const *'

I would love to understand what causes this and whether there is a work around so that I can run it in the Azure Function App.

Thanks

答案1

得分: 1

根据这个 GitHub 问题的解决方案,在这里查看:TypeError: in method 'IndexFlat_add', argument 3 of type 'float const *' · Issue #461 · facebookresearch/faiss · GitHub 确保生成的向量类型或数组是 numpy.float32

Code 1:-

import numpy as np
d = 128
n = 10000
xb = np.random.rand(n, d).astype(np.float32)

vector = [0.5] * d
vector = np.array(vector).astype(np.float32)

distances = np.linalg.norm(xb - vector, axis=1)

k = 5
indices = np.argsort(distances)[:k]

print("Distances:", distances[indices])
print("Indices:", indices)

Output:-

FAISS IndexHNSW Search throws an error on Linux but works on Windows (when installing from pip)

Code 2:-

import numpy as np

d = 128
n = 10000

x = np.random.rand(n, d).astype(np.float32)

print(x)

Output:-

FAISS IndexHNSW Search throws an error on Linux but works on Windows (when installing from pip)

英文:

> According to the solution in this github issue, Refer here:- TypeError: in method 'IndexFlat_add', argument 3 of type
> 'float const *' · Issue #461 · facebookresearch/faiss ·
> GitHub
Make
> sure the generated vector types or array is of numpy.float32

Code 1:-

import numpy as np
d = 128
n = 10000
xb = np.random.rand(n, d).astype(np.float32)

vector = [0.5] * d
vector = np.array(vector).astype(np.float32)

distances = np.linalg.norm(xb - vector, axis=1)

k = 5
indices = np.argsort(distances)[:k]

print("Distances:", distances[indices])
print("Indices:", indices)

Output:-

FAISS IndexHNSW Search throws an error on Linux but works on Windows (when installing from pip)

Code 2:-

import numpy as np

d = 128

n = 10000

x = np.random.rand(n, d).astype(np.float32)

print (x)

Output:-

FAISS IndexHNSW Search throws an error on Linux but works on Windows (when installing from pip)

huangapple
  • 本文由 发表于 2023年4月17日 20:33:30
  • 转载请务必保留本文链接:https://go.coder-hub.com/76035200.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定