英文:
Changing the WandB logging directory
问题
我正在使用WandB进行大量实验,它运行在一个共享服务器上,需要将日志等内容放入特定的目录。WandB喜欢存储大量信息(由我定义),因此文件夹迅速增长,一天多达5GB。
是否有办法指定WandB存储日志的文件夹位置?我查阅了他们的文档,但没有找到任何提到在init
或其他地方指定路径的参数。
或者,我考虑将WandB文件夹的物理位置从我的脚本路径移动到正确的位置,并创建一个符号链接到我的脚本位置。您认为这种方法可行吗?
英文:
I'm doing a lot of experiments with WandB, and it is on a shared server where logs and whatnot need to be placed in specific directories. WandB likes to store a lot of information (as it should, defined by me) so the folder grows rapidly, up to 5GB in a single day.
Is there any way to specify to WandB where to put this folder with logs?.. I searched their docs, but didn't find anything that mentions a parameter in init
or otherwise to specify the path.
Alternatively, I thought of moving the physical location of the WandB folder from my script path to the proper place, and creating a symlink to my script location. Do you think that would work?
答案1
得分: 4
以下是要翻译的内容:
因为Google没有正确索引WandB文档,而且文档中没有充分提到这一点,所以为了将来可能错过的人更容易找到这些信息,我发表这个答案。
默认的WandB日志目录设置为./wandb
,这是wandb.init
的dir
配置参数的默认值,这意味着你可以在wandb.init
期间设置dir="你的路径"
;
此外,尝试为多个无关的运行创建一个全局的wandb
目录并将所有运行的WandB日志符号链接到一个目录是不明智的,因为WandB处理工件的方式会导致问题。
英文:
Posting this answer because Google didn't properly index the WandB docs, and it isn't mentioned there sufficiently, so this becomes easier to find for those in the future who may have missed it.
The default WandB logging directory is set to ./wandb
which is the default parameter for the dir
config parameter of wandb.init
which means you can set dir="your path"
during wandb.init
;
Additionally, it is unwise to attempt to create a global wandb
directory for multiple unrelated runs and symlink all of the runs WandB logs to the one directory due to how WandB handles artifacts.
答案2
得分: 1
Wandb托管在服务器上创建名为wandb的卷。所有日志都存储在该文件夹中。
jetson@nano:~$ docker volume inspect wandb
[
{
"CreatedAt": "2023-10-01T15:26:21+02:00",
"Driver": "local",
"Labels": null,
"Mountpoint": "/var/lib/docker/volumes/wandb/_data",
"Name": "wandb",
"Options": null,
"Scope": "local"
}
]
您可以更改其Mountpoint,或在"docker run wandb/local"命令中设置另一个卷。
英文:
Wandb hosted on server creates volume with name wandb. All logs store into that folder.
jetson@nano:~$ docker volume inspect wandb
[
{
"CreatedAt": "2023-10-01T15:26:21+02:00",
"Driver": "local",
"Labels": null,
"Mountpoint": "/var/lib/docker/volumes/wandb/_data",
"Name": "wandb",
"Options": null,
"Scope": "local"
}
]
You can change it's Mountpoint, or set another volume at "docker run wandb/local" commnad.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论