英文:
AWS EMR network connection
问题
我正在尝试将一个包安装到EMR集群中。每次我都收到以下错误：
警告：在连接中断后进行重试（Retry(total=4, connect=None, read=None, redirect=None, status=None)）后，连接失败 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f5af4e9dad0>: 无法建立新连接：[Errno 101] 网络不可达')'：/simple/fastparquet/
我应该怎么做？
我尝试通过ping google.com，但问题仍然存在。
英文:
I am trying to install a package into EMR cluster. Every time I get the following error
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f5af4e9dad0>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/fastparquet/
What should i do?
I tried to ping into google.com but same problem persisits.
答案1
得分: 1
在Jupyter笔记本环境中使用PySpark进行交互时，请考虑使用EMR Studio（而不是EMR笔记本）。它需要一些基础的设置工作。
之后，您可以在笔记本级别安装库。
列出当前的库：
sc.list_packages()
安装Celery（例如）库：
sc.install_pypi_package("celery")
参考链接：https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-installing-libraries-and-kernels.html#emr-managed-notebooks-work-with-libraries
英文:
To interact using PySpark in a Jupyter notebook environment, consider using EMR Studio (not EMR notebook). It requires some foundation setup work.
Afterwards, you can install libraries on the notebook level.
Listing current libraries:
sc.list_packages()
Installing the Celery (for example) library
sc.install_pypi_package("celery")
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论