遇到了Slurm的问题。错误信息:”没有这样的文件或目录”

huangapple go评论109阅读模式
英文:

Having issue with slurm. error: "no such file or directory"

问题

我正在尝试使用sbatch <script.sh>命令运行一个slurm脚本。然而,尽管我多次检查了我的路径变量,但仍然出现了文件未找到的错误。此外,我认为这与我的go环境有关,因为我还遇到了“无法导入绝对路径”的错误。我不确定问题出在哪里。我已经附上了我的slurm配置文件以及下面的错误输出。

#!/bin/bash
#SBATCH --partition production
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --time=5:00:00
#SBATCH --mem=2GB
#SBATCH --job-name=myTest
#SBATCH --mail-type=END
#SBATCH --mail-user=atd341@nyu.edu
#SBATCH --output=slurm_%j.out

module purge
module load go/1.17
##RUNDIR=${SCRATCH}/run-${SLURM_JOB_ID/.*}
##mkdir -p ${RUNDIR}
DATADIR=${SCRATCH}/inmap_sandbox
cd $SLURM_WORK_DIR
source $DATADIR/setup.sh
go run $DATADIR/

以下是输出结果:

/var/spool/slurmd/job16296/slurm_script: line 19: /inmap_sandbox/setup.sh: 没有那个文件或目录
import "/inmap_sandbox": 无法导入绝对路径

我已经尝试检查我的路径变量,并确保我遵循了正确的路径。作为参考,我的目录结构是/scratch/inmap_sandbox。我正在尝试在/scratch目录中运行sbatch文件。

英文:

I'm trying to run a slurm script using sbatch <script.sh>. However, despite checking my path variable multiple times, i get a file not found error. Moreover I think this has to do with my go environment but I also get a "cannot import absolute path" error. I'm not sure what the issue is. I have attached my slurm configuration file as well as the error output below

#!/bin/bash
#SBATCH --partition production
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --time=5:00:00
#SBATCH --mem=2GB
#SBATCH --job-name=myTest
#SBATCH --mail-type=END
#SBATCH --mail-user=atd341@nyu.edu
#SBATCH --output=slurm_%j.out

module purge
module load go/1.17
##RUNDIR=${SCRATCH}/run-${SLURM_JOB_ID/.*}
##mkdir -p ${RUNDIR}
DATADIR=${SCRATCH}/inmap_sandbox
cd $SLURM_WORK_DIR
source $DATADIR/setup.sh
go run $DATADIR/

Here is the output:

>/var/spool/slurmd/job16296/slurm_script: line 19: /inmap_sandbox/setup.sh: No such file or directory
import "/inmap_sandbox": cannot import absolute path

I have tried checking my path variable and making sure I'm following the correct path. For reference by directory structure is /scratch/inmap_sandbox. I'm trying to run the sbatch file in the /scratch directory

答案1

得分: 1

很明显,${SCRATCH}变量在运行脚本的环境中可能没有设置。尝试将其显式设置为/scratch

解决了这个问题后,请注意,如果此批处理脚本在与您交互式使用的前端节点分开的计算节点上运行,则它们可能不会同时挂载相同的${SCRATCH}文件系统(或者可能在不同的位置挂载)。

请查阅系统文档,了解前端节点和计算节点之间共享的文件系统。您甚至可能需要传递SLURM能力选项来请求某些共享文件系统。如果没有文档,比较前端节点和批处理脚本内部相同命令的输出可能会有所帮助。具体来说,在批处理脚本的早期添加一个单独的行,包含mount命令,并将其生成的输出与前端节点上相同命令的输出进行比较。

英文:

Offhand it appears the ${SCRATCH} variable might not be set inside the environment running the script. Try explicitly setting that to /scratch?

Once you get past that problem, note that if this batch script is running on a compute node that is separate from the frontend node you are using interactively, then they might not both mount the same ${SCRATCH} file system (or possibly mount it in different places).

Consult the system documentation to find out which file systems are shared between the frontend and the compute nodes. You might even need to pass SLURM capability options to request certain shared filesystems. In the absence of documentation, comparing the output of mount on the frontend and from within the batch script might be helpful. More specifically, add the mount command on a line by itself early in your batch script, and compare the output it generates to the output of the same command on the frontend.

huangapple
  • 本文由 发表于 2022年12月16日 05:53:31
  • 转载请务必保留本文链接:https://go.coder-hub.com/74817990.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定