使用dyn async traits(使用async-trait库)在生成的tokio任务中

huangapple go评论60阅读模式
英文:

Using dyn async traits (with async-trait crate) in spawned tokio task

问题

I'm working on an asynchronous rust application which utilizes tokio. I'd also like to define some trait methods as async and have opted for the async-trait crate rather than the feature in the nightly build so that I can use them as dyn objects. However, I'm running into issues trying to use these objects in a task spawned with tokio::spawn. Here's a minimal complete example:

use std::time::Duration;

use async_trait::async_trait;

#[tokio::main]
async fn main() {
    // These two lines based on the examples for dyn traits in the async-trait create
    let value = MyStruct::new();
    let object = &value as &dyn MyTrait;

    tokio::spawn(async move {
        object.foo().await;
    });
}

#[async_trait]
trait MyTrait {
    async fn foo(&self);
}

struct MyStruct {}

impl MyStruct {
    fn new() -> MyStruct {
        MyStruct {}
    }
}

#[async_trait]
impl MyTrait for MyStruct {
    async fn foo(&self) {
        tokio::time::sleep(Duration::from_secs(1)).await;
    }
}

When I compile this I get the following output:

error: future cannot be sent between threads safely
   --> src/main.rs:11:18
    |
11  |       tokio::spawn(async move {
    | __________________^
12  | |         object.foo().await;
13  | |     });
    | |_____^ future created by async block is not `Send`
    |
    = help: the trait `Sync` is not implemented for `dyn MyTrait`
note: captured value is not `Send` because `&` references cannot be sent unless their referent is `Sync`
   --> src/main.rs:12:9
    |
12  |         object.foo().await;
    |         ^^^^^^ has type `&dyn MyTrait` which is not `Send`, because `dyn MyTrait` is not `Sync`
note: required by a bound in `tokio::spawn`
   --> /home/wilyle/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.25.0/src/task/spawn.rs:163:21
    |
163 |         T: Future + Send + 'static,
    |                     ^^^^ required by this bound in `spawn`

error: could not compile `async-test` due to previous error

(The results are similar when making object boxed with let object: Box<dyn MyTrait> = Box::new(MyStruct::new()); and when moving the construction fully inside the tokio::spawn call)

By messing around and trying a few things I found that I could solve the issue by boxing object and adding additional trait bounds. Replacing the first two lines of main in my example with the following seems to work just fine:

let object: Box<dyn MyTrait + Send + Sync> = Box::new(MyStruct::new());

So I have two questions:

  1. Why doesn't my original example work? Is it some inconsistency between the two libraries I'm trying to use or am I approaching async programming in rust incorrectly?
  2. Is the solution of adding additional trait bounds the right way to solve this? I'm rather new to rust and have only been programming with it for a few months, so I wouldn't be surprised to hear I'm just approaching this wrong.
英文:

I'm working on an asynchronous rust application which utilizes tokio. I'd also like to define some trait methods as async and have opted for the async-trait crate rather than the feature in the nightly build so that I can use them as dyn objects. However, I'm running into issues trying to use these objects in a task spawned with tokio::spawn. Here's a minimal complete example:

use std::time::Duration;

use async_trait::async_trait;

#[tokio::main]
async fn main() {
    // These two lines based on the examples for dyn traits in the async-trait create
    let value = MyStruct::new();
    let object = &amp;value as &amp;dyn MyTrait;

    tokio::spawn(async move {
        object.foo().await;
    });
}

#[async_trait]
trait MyTrait {
    async fn foo(&amp;self);
}

struct MyStruct {}

impl MyStruct {
    fn new() -&gt; MyStruct {
        MyStruct {}
    }
}

#[async_trait]
impl MyTrait for MyStruct {
    async fn foo(&amp;self) {
        tokio::time::sleep(Duration::from_secs(1)).await;
    }
}

When I compile this I get the following output:

error: future cannot be sent between threads safely
   --&gt; src/main.rs:11:18
    |
11  |       tokio::spawn(async move {
    |  __________________^
12  | |         object.foo().await;
13  | |     });
    | |_____^ future created by async block is not `Send`
    |
    = help: the trait `Sync` is not implemented for `dyn MyTrait`
note: captured value is not `Send` because `&amp;` references cannot be sent unless their referent is `Sync`
   --&gt; src/main.rs:12:9
    |
12  |         object.foo().await;
    |         ^^^^^^ has type `&amp;dyn MyTrait` which is not `Send`, because `dyn MyTrait` is not `Sync`
note: required by a bound in `tokio::spawn`
   --&gt; /home/wilyle/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.25.0/src/task/spawn.rs:163:21
    |
163 |         T: Future + Send + &#39;static,
    |                     ^^^^ required by this bound in `spawn`

error: could not compile `async-test` due to previous error

(The results are similar when making object boxed with let object: Box&lt;dyn MyTrait&gt; = Box::new(MyStruct::new()); and when moving the construction fully inside the tokio::spawn call)

By messing around and trying a few things I found that I could solve the issue by boxing object and adding additional trait bounds. Replacing the first two lines of main in my example with the following seems to work just fine:

let object: Box&lt;dyn MyTrait + Send + Sync&gt; = Box::new(MyStruct::new());

So I have two questions:

  1. Why doesn't my original example work? Is it some inconsistency between the two libraries I'm trying to use or am I approaching async programming in rust incorrectly?
  2. Is the solution of adding additional trait bounds the right way to solve this? I'm rather new to rust and have only been programming with it for a few months so I wouldn't be surprised to hear I'm just approaching this wrong.

答案1

得分: 4

代码部分不要翻译,以下是翻译好的内容:

如果你不确定 SendSync 是什么意思,可以查看这些文档链接。需要注意的是,如果 TSync,那么 &T 就是 Send

问题#2很简单:是的,这是正确的方法。async-trait 之所以使用 Pin<Box<dyn Future + Send>> 作为返回类型,基本上是出于相同的原因。请注意,你只能向 trait 对象添加 auto traits

对于问题#1,有两个问题:Send 'static

Send

当你将某物强制转换为 dyn MyTrait 时,你移除了所有原始类型信息,并将其替换为类型 dyn MyTrait。这意味着你失去了 MyStruct 上自动生成的 SendSync 特性。tokio::spawn 函数需要 Send

这个问题不是异步的固有问题,而是因为 tokio::spawn 将在其线程池上运行未来,可能会将其发送到另一个线程。你可以在不使用 tokio::spawn 的情况下运行未来,例如像这样:

fn main() {
    let runtime = tokio::runtime::Runtime::new().unwrap();
    
    let value = MyStruct::new();
    let object = &value as &dyn MyTrait;

    runtime.block_on(object.foo());
}

block_on 函数在当前线程上运行未来,因此不需要 Send。它会阻塞,直到未来完成,因此也不需要 'static。这对于在运行时创建并包含整个程序逻辑的内容非常有用,但对于 dyn Trait 类型,通常有其他事情发生,使其不太有用。

'static

当某物需要 'static 时,这意味着所有引用都需要存在于 'static 之久。满足这一要求的一种方式是删除所有引用。在理想的世界中,你可以这样做:

let object = value as dyn MyTrait;

然而,Rust 不支持堆栈上或作为函数参数的动态大小类型。我们正试图消除所有引用,因此 &dyn MyTrait 不会起作用(除非你进行泄漏或拥有静态变量)。Box 允许你通过将它们放在堆上来拥有动态大小的类型,从而消除了生命周期。

你需要 Send 是因为从 Sync 升级到 Send 仅发生在 & 中,而不是在 Box 中。相反,当 TSend 时,Box<T>Send

Sync 更加微妙。虽然 spawn 不需要 Sync,但异步块确实需要 Send + Sync 才能成为 Send。由于 foo 接受 &self,这意味着它返回一个持有 &selfFuture。然后会对该类型进行轮询,因此在轮询之间可能会在不同线程之间发送 &self。如前所述,如果 TSync,那么 &T 就是 Send然而,如果你将其更改为 foo(&mut self),它将在没有 + Sync 的情况下编译。 这是有道理的,因为现在可以检查它是否正在被同时使用,但在我看来,&self 版本可能在未来会被允许。

英文:

If you're not sure what Send and Sync mean, check out those documentation links. Something to note is that if T is Sync, then &amp;T is Send.

Question #2 is simple: yes this is the right way to do it. async-trait uses Pin&lt;Box&lt;dyn Future + Send&gt;&gt; as its return type for basically the same reasons. Note that you can only add auto traits to trait objects.

For question #1, there's two issues: Send and &#39;static.

Send

When you cast something as dyn MyTrait, you're removing all the original type information and replacing it with the type dyn MyTrait. That means you lose the auto-implemented Send and Sync traits on MyStruct. The tokio::spawn function requires Send.

This issue isn't inherent to async, it's because tokio::spawn will run the future on its threadpool, possibly sending it to another thread. You can run the future without tokio::spawn, for example like this:

fn main() {
    let runtime = tokio::runtime::Runtime::new().unwrap();
    
    let value = MyStruct::new();
    let object = &amp;value as &amp;dyn MyTrait;

    runtime.block_on(object.foo());
}

The block_on function runs the future on the current thread, so Send is not necessary. And it blocks until the future is done, so &#39;static is also not needed. This is great for things that are created at runtime and contain the entire logic of the program, but for dyn Trait types you usually have other things going on that makes this not as useful.

&#39;static

When something requires &#39;static, it means that all references need to live as long as &#39;static. One way of satisfying that is to remove all references. In an ideal world you could do:

let object = value as dyn MyTrait;

However, rust doesn't support dynamically sized types on the stack or as function arguments. We're trying to remove all references, so &amp;dyn MyTrait isn't going to work (unless you leak or have a static variable). Box lets you have ownership over dynamically sized types by putting them on the heap, eliminating the lifetime.

You need Send for this because the upgrade from Sync to Send only happens with &amp;, not Box. Instead, Box&lt;T&gt; is Send when T is Send.

Sync is more subtle. While spawn doesn't require Sync, the async block does require Send + Sync to be Send. Since foo takes &amp;self, that means it returns a Future that holds &amp;self. That type is then polled, so in between polls &amp;self could be sent in between threads. And as before, &amp;T is Send if T is Sync. However, if you change it to foo(&amp;mut self) it compiles without + Sync. Makes sense since now it can check that it's not being used concurrently, but it seems to me like the &amp;self verison could be allowed in the future.

huangapple
  • 本文由 发表于 2023年2月10日 02:41:34
  • 转载请务必保留本文链接:https://go.coder-hub.com/75403078.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定