问题

I am trying to use databricks connect.

我正在尝试使用Databricks Connect。

I have installed databricks-connect version 9.1.39 in a virtual environment within my python project.

我已在我的Python项目中的虚拟环境中安装了Databricks Connect 版本9.1.39。

I have selected the python3.8 file in the virtual environment as the interpreter of the vscode project. However, when trying to run a file that starts with

我已将虚拟环境中的python3.8文件选择为VSCode项目的解释器。然而，当尝试运行一个以以下内容开头的文件时

from databricks.connect import DatabricksSession

from databricks.connect import DatabricksSession

I always get a

我总是收到以下错误

ModuleNotFoundError: No module named 'databricks'

ModuleNotFoundError: 找不到名为'databricks'的模块

Just to make sure this was not due to the module databricks-connect not being in the actual environment when I run it, I actually opened a python shell within the venv and run the same line (from databricks.connect ...) and got the same error.

只是为了确保这不是因为在运行时模块databricks-connect不在实际环境中，我实际上在venv中打开了一个Python shell，并运行了相同的命令（from databricks.connect ...），并收到了相同的错误。

Why is this happening? Is it due to datarbicks.connect not being related to databricks-connect?

为什么会发生这种情况？是因为databricks.connect与databricks-connect无关吗？

Thanks in advance.

提前感谢您。

答案1

得分: 2

DatabricksSession仅存在于专为Databricks Runtime 13或更高版本设计的Databricks Connect V2中。如果您使用的是DBR 9.1，那么您需要按照DBR 11.3及更低版本的说明进行操作 - 在这种情况下，您需要使用databricks-connect configure命令配置连接详细信息，然后使用普通的Spark Session创建：

from pyspark.sql.session import SparkSession

spark = SparkSession.builder.getOrCreate()

from pyspark.sql.session import SparkSession

spark = SparkSession.builder.getOrCreate()