英文:
Where is my kedro output when using the databricks extension in vscode
问题
我正在使用kedro与databricks扩展一起在vscode中访问Azure上的databricks服务器。
一切都运行得很顺利,但在本地执行文件时我看不到任何输出。
我收到的唯一输出是:
31/03/2023, 15:09:32 - 将代码同步到/Repos/<>@<>.ide ...
31/03/2023, 15:09:32 - 运行kedro-hello-world.py ...
[03/31/23 13:09:37] INFO Kedro项目dp-kedro session.py:355
31/03/2023, 15:09:43 - 完成(耗时13914毫秒)
如果我查看日志,我看到以下内容:
{
"level": "debug",
"logger": "SDK",
"loggingFunction": "CommandExecutionService.commandStatus",
"message": "cluster",
"operationId": "id",
"operationName": "CommandExecutionService.commandStatus",
"request": {"method": "GET"},
"response": {
"id": "id",
"results": {
"data": "[03/31/23 12:41:43] INFO Kedro项目dp-kedro session.py:355",
"resultType": "text",
},
"status": "Finished",
},
"timestamp": 1680266509470,
}
所以似乎其余的消息没有返回到vscode?直接在databricks上执行文件可以正常工作,我可以看到Kedro的所有输出。
英文:
I am using kedro together with the databricks extension for vscode to access databricks server on Azure.
Everything works pretty well but I don't see any output when executing the file locally.
The only output I receive is:
31/03/2023, 15:09:32 - Synchronizing code to /Repos/<>@<>.ide ...
31/03/2023, 15:09:32 - Running kedro-hello-world.py ...
[03/31/23 13:09:37] INFO Kedro project dp-kedro session.py:355
31/03/2023, 15:09:43 - Done (took 13914ms)
If i look at the logs i see the following:
{
"level": "debug",
"logger": "SDK",
"loggingFunction": "CommandExecutionService.commandStatus",
"message": "cluster",
"operationId": "id",
"operationName": "CommandExecutionService.commandStatus",
"request": {"method": "GET"},
"response": {
"id": "id",
"results": {
"data": "[03/31/23 12:41:43] INFO Kedro project dp-kedro session.py:355",
"resultType": "text",
},
"status": "Finished",
},
"timestamp": 1680266509470,
}
So it seems that the rest of the messages arent coming back to vscode?
Executing the file directly on databricks works and I can see all outputs of Kedro.
答案1
得分: 3
<strike>
所以在Kedro中配置日志记录是在[logging.yml][1]文件中完成的,该文件指向Python的`logging.dictConfig`方法 1:1
我还没有机会自己使用新的Databricks扩展,也许这里的[详细模式][2]会有用吗?
</strike>
--------------------
在下面的主题中发现 -
>我终于找到了,因为我正在通过vscode -> databricks -> kedro进行工作,日志记录(使用的是rich)出现了问题,从配置中删除rich并将其替换为控制台就可以解决...所以最初的提示是查看日志记录起作用的:) 谢谢
请将您的`logging.yml`设置为普通控制台记录并删除`RichHandler`
```diff
root:
- handlers: [rich, info_file_handler, error_file_handler]
+ handlers: [console, info_file_handler, error_file_handler]
<details>
<summary>英文:</summary>
<strike>
So logging configuration in Kedro is done within the [logging.yml][1] file which points to the Python `logging.dictConfig` method 1:1
I haven't had a chance to use the new databricks extension myself, perhaps the [verbose mode][2] here is useful?
</strike>
--------------------
As discovered in the thread below -
>I've finally found it, because I am working through vscode -> databricks -> kedro, the logging (which is using rich) is messed up, removing rich from the config and replace it by console works... so the initial tip of looking into logging works :) thx
Please set your `logging.yml` to plain console logging and remove the `RichHandler`
```diff
root:
- handlers: [rich, info_file_handler, error_file_handler]
+ handlers: [console, info_file_handler, error_file_handler]
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论