createWriteStream在由于磁盘空间不足而未完全写入文件时不会触发错误事件。

huangapple go评论77阅读模式
英文:

createWriteStream do not fire an error event when the file is not fully written due to disk space

问题

以下是您要翻译的部分:

这是我在使用createWriteStream时遇到的问题:

  1. 假设我有一个100MB的文件,我想将它写入磁盘上的另一个文件中。
  2. 可用磁盘空间只有50MB。

我的代码如下:

const fs = require('fs');
const w = fs.createWriteStream('/path/in/other/partition/destination.txt');
const r = fs.createReadStream("/source.txt")

r.on('data', function(chunk) {
  w.write(chunk);
});

w.on('close', function () {
   console.log('closed1');
});
w.on('error', (e) => console.log('error'));

现在我期望在没有足够空间时会收到错误,但实际上我得到的是一个部分创建的50MB大小的文件(在开始写入之前的可用磁盘空间),而没有任何错误。

  • 对于这种行为是否有解释?
  • 我如何强制createWriteStream在这种情况下触发一个错误事件,而不需要事先知道文件大小?

Node.js版本:v18.16.0

英文:

Here's the problem i'm facing with createWriteStream:

  1. Let's say I have a 100mb file, and i want to write it in another file on the disk
  2. the disk available size is 50mb

my code:

    const fs = require('fs');
    const w = fs.createWriteStream('/path/in/other/partition/destination.txt');
    const r = fs.createReadStream("/source.txt")

    r.on('data', function(chunk) {
    w.write(chunk);
    });

    w.on('close', function () {
       console.log('closed1');
    });
    w.on('error', (e) => console.log('error'));

Now i expect to get an error when there's no more space available, but what I got instead is a partially created file of 50mb size (available disk size before start writing) without any error.

  • Is there any explanation for this behavior ?
  • how can I force createWriteStream to fire an error event in this case knowing that i do not know the file size beforehand?

node version: v18.16.0

答案1

得分: 1

First off, disk full is not considered an error in the standard nodejs file write operations. You pass in a number of bytes to write and get back how many bytes were written. If the two values are not the same (all bytes were not written), then one assumes that's probably a disk full issue, but it doesn't trigger an error at the lowest level of writing.

And, nodejs streams don't seem to check to see if all bytes were written and don't seem to make it easy for the code using a stream to check for that either. This seems like a giant hole in the implementation to me. Streams are wonderful to use when errors don't happen and are sometimes a nightmare when errors do happen because they don't communicate them back cleanly in some cases. I've taken to using them less and less and just do manual reads and writes where I have total control. That might be advised here.

Looking at the file writeStream code, it does not appear to ever check if all bytes are written and that's the only way to detect a disk becoming full on a write. So, it appears to me that the stream doesn't check for disk full and doesn't make it an error. It's possible there's some other way it might notice, but I'm not seeing it myself in the writeStream code.

It's also possible that the stream is filling up its buffer and w.write() is returning false and you're not checking for that which would tell you that the stream is backed up. You could then wait for the drain event and set a timer and if you hit the timeout you set, then no drain event is coming and something is definitely wrong, perhaps a disk full.

And, you could also pass a callback to w.write() and see if you get an error there, but I don't see anywhere for that callback to communicate the error either.

英文:

First off, disk full is not considered an error in the standard nodejs file write operations. You pass in a number of bytes to write and get back how many bytes were written. If the two values are not the same (all bytes were not written), then one assumes that's probably a disk full issue, but it doesn't trigger an error at the lowest level of writing.

And, nodejs streams don't seem to check to see if all bytes were written and don't seem to make it easy for the code using a stream to check for that either. This seems like a giant hole in the implementation to me. Streams are wonderful to use when errors don't happen and are sometimes a nightmare when errors do happen because they don't communicate them back cleanly in some cases. I've taken to using them less and less and just do manual reads and writes where I have total control. That might be advised here.

Looking at the file writeStream code, it does not appear to ever check if all bytes are written and that's the only way to detect a disk becoming full on a write. So, it appears to me that the stream doesn't check for disk full and doesn't make it an error. It's possible there's some other way it might notice, but I'm not seeing it myself in the writeStream code.

It's also possible that the stream is filling up its buffer and w.write() is returning false and you're not checking for that which would tell you that the stream is backed up. You could then wait for the drain event and set a timer and if you hit the timeout you set, then no drain event is coming and something is definitely wrong, perhaps a disk full.

And, you could also pass a callback to w.write() and see if you get an error there, but I don't see anywhere for that callback to communicate the error either.

huangapple
  • 本文由 发表于 2023年6月13日 00:52:14
  • 转载请务必保留本文链接:https://go.coder-hub.com/76458767.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定