Changing the WandB logging directory

huangapple go评论110阅读模式
英文:

Changing the WandB logging directory

问题

我正在使用WandB进行大量实验,它运行在一个共享服务器上,需要将日志等内容放入特定的目录。WandB喜欢存储大量信息(由我定义),因此文件夹迅速增长,一天多达5GB。

是否有办法指定WandB存储日志的文件夹位置?我查阅了他们的文档,但没有找到任何提到在init或其他地方指定路径的参数。

或者,我考虑将WandB文件夹的物理位置从我的脚本路径移动到正确的位置,并创建一个符号链接到我的脚本位置。您认为这种方法可行吗?

英文:

I'm doing a lot of experiments with WandB, and it is on a shared server where logs and whatnot need to be placed in specific directories. WandB likes to store a lot of information (as it should, defined by me) so the folder grows rapidly, up to 5GB in a single day.

Is there any way to specify to WandB where to put this folder with logs?.. I searched their docs, but didn't find anything that mentions a parameter in init or otherwise to specify the path.

Alternatively, I thought of moving the physical location of the WandB folder from my script path to the proper place, and creating a symlink to my script location. Do you think that would work?

答案1

得分: 4

以下是要翻译的内容:

因为Google没有正确索引WandB文档,而且文档中没有充分提到这一点,所以为了将来可能错过的人更容易找到这些信息,我发表这个答案。


默认的WandB日志目录设置为./wandb,这是wandb.initdir配置参数的默认值,这意味着你可以在wandb.init期间设置dir="你的路径"

此外,尝试为多个无关的运行创建一个全局的wandb目录并将所有运行的WandB日志符号链接到一个目录是不明智的,因为WandB处理工件的方式会导致问题。

英文:

Posting this answer because Google didn't properly index the WandB docs, and it isn't mentioned there sufficiently, so this becomes easier to find for those in the future who may have missed it.


The default WandB logging directory is set to ./wandb which is the default parameter for the dir config parameter of wandb.init which means you can set dir="your path" during wandb.init;

Additionally, it is unwise to attempt to create a global wandb directory for multiple unrelated runs and symlink all of the runs WandB logs to the one directory due to how WandB handles artifacts.

答案2

得分: 1

Wandb托管在服务器上创建名为wandb的卷。所有日志都存储在该文件夹中。

jetson@nano:~$ docker volume inspect wandb
[
   {
       "CreatedAt": "2023-10-01T15:26:21+02:00",
       "Driver": "local",
       "Labels": null,
       "Mountpoint": "/var/lib/docker/volumes/wandb/_data",
       "Name": "wandb",
       "Options": null,
       "Scope": "local"
   }
]

您可以更改其Mountpoint,或在"docker run wandb/local"命令中设置另一个卷。

英文:

Wandb hosted on server creates volume with name wandb. All logs store into that folder.

jetson@nano:~$ docker volume inspect wandb
[
   {
       "CreatedAt": "2023-10-01T15:26:21+02:00",
       "Driver": "local",
       "Labels": null,
       "Mountpoint": "/var/lib/docker/volumes/wandb/_data",
       "Name": "wandb",
       "Options": null,
       "Scope": "local"
   }
]

You can change it's Mountpoint, or set another volume at "docker run wandb/local" commnad.

huangapple
  • 本文由 发表于 2023年3月8日 16:53:21
  • 转载请务必保留本文链接:https://go.coder-hub.com/75671008.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定