让一个消耗 std::io::Write 的函数写入内存

huangapple go评论48阅读模式
英文:

Making a std::io::Write-consuming function write into memory

问题

我正在尝试使用一个 crate,它接受实现了 std::io::Write 的任何东西的所有权来进行写入,但我想将它写入的数据保存在内存中以供进一步处理,而不是直接写入磁盘。我正在尝试使用的 crate 是 ciborium(如果相关的话)。

我离成功最接近的方法是将一个 &mut [u8] 作为写入器传递,就像这个通用问题的示例中所示:

use std::io::Write;

fn write_stuff<W: Write>(mut w: W) {
    w.write_all(b"Hello
use std::io::Write;

fn write_stuff<W: Write>(mut w: W) {
    w.write_all(b"Hello\0Rust!").unwrap();
}

fn main() {
    let mut v = [0u8; 20];
    
    write_stuff(&mut v[..]);
    
    dbg!(v);
}
Rust!").unwrap();
} fn main() { let mut v = [0u8; 20]; write_stuff(&mut v[..]); dbg!(v); }

这个方法在我能够看到已经写入的字节方面是有效的,但有一个相当痛苦的缺陷,我无法知道实际上写入了多少数据到切片中。对于我的情况来说,这是一个严重的问题,因为正在序列化的数据很可能包含 0 字节(以及其他任何值)。

我尝试过的任何会跟踪已写入字节数的方法(例如 Vec<u8>Cursor 等)都无法告诉我作为调用者有多少字节已写入,因为我调用的函数会获取该值的所有权,因此调用者之后的任何操作都会违反所有权规则。

目前,我能想到的最不糟糕的选择,除了修改 ciborium(或更 drastical 的选择,比如大量的指针操作的不安全代码),就是创建一个管道,将写入器半部分传递给 ciborium,然后从读取器半部分读取数据。这... 很糟糕。

我是否漏掉了一些非常明显的东西,或者在这里完全没有运气?

英文:

I'm trying to use a crate that takes ownership of anything that implements std::io::Write to write into, but I want to get the data it writes in memory for further processing, rather than writing it straight to disk. The crate I'm trying to use is ciborium if it's relevant.

The closest I've gotten to success is to pass in a &amp;mut [u8] as the writer, as in this toy demonstration of the general problem:

use std::io::Write;

fn write_stuff&lt;W: Write&gt;(mut w: W) {
    w.write_all(b&quot;Hello\0Rust!&quot;).unwrap();
}

fn main() {
    let mut v = [0u8; 20];
    
    write_stuff(&amp;mut v[..]);
    
    dbg!(v);
}

This works, insofar as I can see the bytes that have been read, but has the rather painful flaw that I can't tell how much data was actually written into the slice. This is a serious problem for my case, as the data being serialized may very well include 0 bytes (and any other value).

Anything I've tried that keeps track of the count of bytes written (like Vec&lt;u8&gt;, Cursor, etc) can't tell me, the caller, how many bytes were written because the function I'm calling takes ownership of the value, and so anything after that from the caller falls foul of ownership rules.

At present, the least-worst option I can think of, short of modifying ciborium (or more drastic options, like great piles of pointer-mangling unsafe code) is to create a pipe, passing the writer half to ciborium, and reading the data out the reader half. Which is... hideous.

Am I missing something really obvious, or am I totally out of luck here?

答案1

得分: 5

"Write"被实现为"Vec<u8>",以及&amp;mut W其中W: Write。将这两者结合起来意味着"&mut Vec<u8>"也是"Write"。您可以使用它来保留对"Vec"的所有权。

英文:

Write is implemented for Vec&lt;u8&gt; as well as &amp;mut W where W: Write. Combining the two means that &amp;mut Vec&lt;u8&gt; is Write. You can use that to retain ownership of the Vec.

答案2

得分: 4

约翰的答案可能是最直接的,但我想提供一个使用数组的替代方法,与你的问题类似。

&amp;mut [u8]Write 实现 更新了切片,指向尚未写入的部分,因此两个切片之间的长度差告诉你写入了多少字节:

use std::io::Write;

fn write_something(mut w: impl Write) -> std::io::Result<()> {
    w.write_all(b"foobar")
}

fn main() -> std::io::Result<()> {
    let mut b = [0u8; 20];
    
    let written = b.len() - {
        let mut s = &mut b[..];

        write_something(&mut s)?;
        write_something(&mut s)?;

        s.len()
    };

    assert_eq!(written, 12);
    assert_eq!(&b[..written], b"foobarfoobar");
    
    Ok(())
}

(Playground)

英文:

John's answer is probably the most straightforward, but I wanted to provide an alternative that uses an array as in your question.

The Write implementation for &amp;mut [u8] updates the slice to point to the yet-unwritten part, so the difference in length between the two slices tells you how many bytes were written:

use std::io::Write;

fn write_something(mut w: impl Write) -&gt; std::io::Result&lt;()&gt; {
    w.write_all(b&quot;foobar&quot;)
}

fn main() -&gt; std::io::Result&lt;()&gt; {
    let mut b = [0u8; 20];
    
    let written = b.len() - {
        let mut s = &amp;mut b[..];

        write_something(&amp;mut s)?;
        write_something(&amp;mut s)?;

        s.len()
    };

    assert_eq!(written, 12);
    assert_eq!(&amp;b[..written], b&quot;foobarfoobar&quot;);
    
    Ok(())
}

(Playground)

huangapple
  • 本文由 发表于 2023年5月28日 22:36:26
  • 转载请务必保留本文链接:https://go.coder-hub.com/76352012.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定