英文:
langchain emedding with huggingface can't pass the access token
问题
我正在使用llama_index和langchain创建文档索引。
对于嵌入，我想使用<https://huggingface.co/bert-base-multilingual-cased>。
使用以下代码创建嵌入没有问题：
from langchain.embeddings.huggingface import HuggingFaceEmbeddings
from llama_index import LangchainEmbedding, ServiceContext
embed_model = LangchainEmbedding(
HuggingFaceEmbeddings(model_name="bert-base-multilingual-cased")
)
service_context = ServiceContext.from_defaults(embed_model=embed_model)
我的问题是，我不知道如何在请求模型时使用我的HF访问令牌。我想使用访问令牌是因为1）HF文档建议使用它，2）我可能会使用HF Pro计划以获得更高的API速率限制。
英文:
I am creating an index of documents with llama_index and langchain.
For the embedding I want to use <https://huggingface.co/bert-base-multilingual-cased>.
I had no problem creating the embedding using the following code:
from langchain.embeddings.huggingface import HuggingFaceEmbeddings
from llama_index import LangchainEmbedding, ServiceContext
embed_model = LangchainEmbedding(
HuggingFaceEmbeddings(model_name="bert-base-multilingual-cased")
)
service_context = ServiceContext.from_defaults(embed_model=embed_model)
my problem is that I don't understand how to use my HF access token when requesting the model. I want to use the access token because 1) it is suggested by the HF documentation and 2) I will probably use the HF Pro Plan in order to have an higher api rate limit.
答案1
得分: 0
我认为你不能在langchain.embeddings.HuggingfaceEmbeddings中使用授权令牌，但如果你需要使用授权令牌，你可以使用Hugging Face Hub。
from langchain.embeddings import HuggingFaceHubEmbeddings
embeddings = HuggingFaceHubEmbeddings(repo_id='path/to/repo',
huggingfacehub_api_token='API_TOKEN')
英文:
I think you can't use authorization tokens in langchain.embeddings.HuggingfaceEmbeddings but you can surely use hugging face hub if you need to use the authorization tokens.
from langchain.embeddings import HuggingFaceHubEmbeddings
embeddings = HuggingFaceHubEmbeddings(repo_id='path/to/repo',
huggingfacehub_api_token='API_TOKEN')
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论