在使用Visual Studio Code中的Databricks扩展时,我的Kedro输出在哪里?

huangapple go评论59阅读模式
英文:

Where is my kedro output when using the databricks extension in vscode

问题

我正在使用kedro与databricks扩展一起在vscode中访问Azure上的databricks服务器。
一切都运行得很顺利,但在本地执行文件时我看不到任何输出。
我收到的唯一输出是:

31/03/2023, 15:09:32 - 将代码同步到/Repos/<>@<>.ide ...
31/03/2023, 15:09:32 - 运行kedro-hello-world.py ...

[03/31/23 13:09:37] INFO     Kedro项目dp-kedro                                                  session.py:355
31/03/2023, 15:09:43 - 完成(耗时13914毫秒)

如果我查看日志,我看到以下内容:

{
    "level": "debug",
    "logger": "SDK",
    "loggingFunction": "CommandExecutionService.commandStatus",
    "message": "cluster",
    "operationId": "id",
    "operationName": "CommandExecutionService.commandStatus",
    "request": {"method": "GET"},
    "response": {
        "id": "id",
        "results": {
            "data": "[03/31/23 12:41:43] INFO     Kedro项目dp-kedro                                                  session.py:355",
            "resultType": "text",
        },
        "status": "Finished",
    },
    "timestamp": 1680266509470,
}

所以似乎其余的消息没有返回到vscode?直接在databricks上执行文件可以正常工作,我可以看到Kedro的所有输出。

英文:

I am using kedro together with the databricks extension for vscode to access databricks server on Azure.
Everything works pretty well but I don't see any output when executing the file locally.
The only output I receive is:

31/03/2023, 15:09:32 - Synchronizing code to /Repos/<>@<>.ide ...
31/03/2023, 15:09:32 - Running kedro-hello-world.py ...

[03/31/23 13:09:37] INFO     Kedro project dp-kedro                                                  session.py:355
31/03/2023, 15:09:43 - Done (took 13914ms)

If i look at the logs i see the following:

{
    "level": "debug",
    "logger": "SDK",
    "loggingFunction": "CommandExecutionService.commandStatus",
    "message": "cluster",
    "operationId": "id",
    "operationName": "CommandExecutionService.commandStatus",
    "request": {"method": "GET"},
    "response": {
        "id": "id",
        "results": {
            "data": "[03/31/23 12:41:43] INFO     Kedro project dp-kedro                                                  session.py:355",
            "resultType": "text",
        },
        "status": "Finished",
    },
    "timestamp": 1680266509470,
}

So it seems that the rest of the messages arent coming back to vscode?
Executing the file directly on databricks works and I can see all outputs of Kedro.

答案1

得分: 3

<strike>
所以在Kedro中配置日志记录是在[logging.yml][1]文件中完成的,该文件指向Python的`logging.dictConfig`方法 1:1

我还没有机会自己使用新的Databricks扩展,也许这里的[详细模式][2]会有用吗?

</strike>

--------------------

在下面的主题中发现 -

>我终于找到了,因为我正在通过vscode -> databricks -> kedro进行工作,日志记录(使用的是rich)出现了问题,从配置中删除rich并将其替换为控制台就可以解决...所以最初的提示是查看日志记录起作用的:) 谢谢

请将您的`logging.yml`设置为普通控制台记录并删除`RichHandler`

```diff
 root:
-  handlers: [rich, info_file_handler, error_file_handler]
+  handlers: [console, info_file_handler, error_file_handler]

<details>
<summary>英文:</summary>

&lt;strike&gt;
So logging configuration in Kedro is done within the [logging.yml][1] file which points to the Python `logging.dictConfig` method 1:1

I haven&#39;t had a chance to use the new databricks extension myself, perhaps the [verbose mode][2] here is useful?

&lt;/strike&gt;


--------------------
As discovered in the thread below - 


&gt;I&#39;ve finally found it, because I am working through vscode -&gt; databricks -&gt; kedro, the logging (which is using rich) is messed up, removing rich from the config and replace it by console works... so the initial tip of looking into logging works :) thx


Please set your `logging.yml` to plain console logging and remove the `RichHandler` 


```diff
 root:
-  handlers: [rich, info_file_handler, error_file_handler]
+  handlers: [console, info_file_handler, error_file_handler]

huangapple
  • 本文由 发表于 2023年3月31日 21:18:24
  • 转载请务必保留本文链接:https://go.coder-hub.com/75899005.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定