英文:
Using dyn async traits (with async-trait crate) in spawned tokio task
问题
I'm working on an asynchronous rust application which utilizes tokio. I'd also like to define some trait methods as async
and have opted for the async-trait crate rather than the feature in the nightly build so that I can use them as dyn
objects. However, I'm running into issues trying to use these objects in a task spawned with tokio::spawn
. Here's a minimal complete example:
use std::time::Duration;
use async_trait::async_trait;
#[tokio::main]
async fn main() {
// These two lines based on the examples for dyn traits in the async-trait create
let value = MyStruct::new();
let object = &value as &dyn MyTrait;
tokio::spawn(async move {
object.foo().await;
});
}
#[async_trait]
trait MyTrait {
async fn foo(&self);
}
struct MyStruct {}
impl MyStruct {
fn new() -> MyStruct {
MyStruct {}
}
}
#[async_trait]
impl MyTrait for MyStruct {
async fn foo(&self) {
tokio::time::sleep(Duration::from_secs(1)).await;
}
}
When I compile this I get the following output:
error: future cannot be sent between threads safely
--> src/main.rs:11:18
|
11 | tokio::spawn(async move {
| __________________^
12 | | object.foo().await;
13 | | });
| |_____^ future created by async block is not `Send`
|
= help: the trait `Sync` is not implemented for `dyn MyTrait`
note: captured value is not `Send` because `&` references cannot be sent unless their referent is `Sync`
--> src/main.rs:12:9
|
12 | object.foo().await;
| ^^^^^^ has type `&dyn MyTrait` which is not `Send`, because `dyn MyTrait` is not `Sync`
note: required by a bound in `tokio::spawn`
--> /home/wilyle/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.25.0/src/task/spawn.rs:163:21
|
163 | T: Future + Send + 'static,
| ^^^^ required by this bound in `spawn`
error: could not compile `async-test` due to previous error
(The results are similar when making object
boxed with let object: Box<dyn MyTrait> = Box::new(MyStruct::new());
and when moving the construction fully inside the tokio::spawn
call)
By messing around and trying a few things I found that I could solve the issue by boxing object
and adding additional trait bounds. Replacing the first two lines of main
in my example with the following seems to work just fine:
let object: Box<dyn MyTrait + Send + Sync> = Box::new(MyStruct::new());
So I have two questions:
- Why doesn't my original example work? Is it some inconsistency between the two libraries I'm trying to use or am I approaching async programming in rust incorrectly?
- Is the solution of adding additional trait bounds the right way to solve this? I'm rather new to rust and have only been programming with it for a few months, so I wouldn't be surprised to hear I'm just approaching this wrong.
英文:
I'm working on an asynchronous rust application which utilizes tokio. I'd also like to define some trait methods as async
and have opted for the async-trait crate rather than the feature in the nightly build so that I can use them as dyn
objects. However, I'm running into issues trying to use these objects in a task spawned with tokio::spawn
. Here's a minimal complete example:
use std::time::Duration;
use async_trait::async_trait;
#[tokio::main]
async fn main() {
// These two lines based on the examples for dyn traits in the async-trait create
let value = MyStruct::new();
let object = &value as &dyn MyTrait;
tokio::spawn(async move {
object.foo().await;
});
}
#[async_trait]
trait MyTrait {
async fn foo(&self);
}
struct MyStruct {}
impl MyStruct {
fn new() -> MyStruct {
MyStruct {}
}
}
#[async_trait]
impl MyTrait for MyStruct {
async fn foo(&self) {
tokio::time::sleep(Duration::from_secs(1)).await;
}
}
When I compile this I get the following output:
error: future cannot be sent between threads safely
--> src/main.rs:11:18
|
11 | tokio::spawn(async move {
| __________________^
12 | | object.foo().await;
13 | | });
| |_____^ future created by async block is not `Send`
|
= help: the trait `Sync` is not implemented for `dyn MyTrait`
note: captured value is not `Send` because `&` references cannot be sent unless their referent is `Sync`
--> src/main.rs:12:9
|
12 | object.foo().await;
| ^^^^^^ has type `&dyn MyTrait` which is not `Send`, because `dyn MyTrait` is not `Sync`
note: required by a bound in `tokio::spawn`
--> /home/wilyle/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.25.0/src/task/spawn.rs:163:21
|
163 | T: Future + Send + 'static,
| ^^^^ required by this bound in `spawn`
error: could not compile `async-test` due to previous error
(The results are similar when making object
boxed with let object: Box<dyn MyTrait> = Box::new(MyStruct::new());
and when moving the construction fully inside the tokio::spawn
call)
By messing around and trying a few things I found that I could solve the issue by boxing object
and adding additional trait bounds. Replacing the first two lines of main
in my example with the following seems to work just fine:
let object: Box<dyn MyTrait + Send + Sync> = Box::new(MyStruct::new());
So I have two questions:
- Why doesn't my original example work? Is it some inconsistency between the two libraries I'm trying to use or am I approaching async programming in rust incorrectly?
- Is the solution of adding additional trait bounds the right way to solve this? I'm rather new to rust and have only been programming with it for a few months so I wouldn't be surprised to hear I'm just approaching this wrong.
答案1
得分: 4
代码部分不要翻译,以下是翻译好的内容:
如果你不确定 Send
和 Sync
是什么意思,可以查看这些文档链接。需要注意的是,如果 T
是 Sync
,那么 &T
就是 Send
。
问题#2很简单:是的,这是正确的方法。async-trait
之所以使用 Pin<Box<dyn Future + Send>>
作为返回类型,基本上是出于相同的原因。请注意,你只能向 trait 对象添加 auto traits。
对于问题#1,有两个问题:Send
和 'static
。
Send
当你将某物强制转换为 dyn MyTrait
时,你移除了所有原始类型信息,并将其替换为类型 dyn MyTrait
。这意味着你失去了 MyStruct
上自动生成的 Send
和 Sync
特性。tokio::spawn
函数需要 Send
。
这个问题不是异步的固有问题,而是因为 tokio::spawn
将在其线程池上运行未来,可能会将其发送到另一个线程。你可以在不使用 tokio::spawn
的情况下运行未来,例如像这样:
fn main() {
let runtime = tokio::runtime::Runtime::new().unwrap();
let value = MyStruct::new();
let object = &value as &dyn MyTrait;
runtime.block_on(object.foo());
}
block_on
函数在当前线程上运行未来,因此不需要 Send
。它会阻塞,直到未来完成,因此也不需要 'static
。这对于在运行时创建并包含整个程序逻辑的内容非常有用,但对于 dyn Trait
类型,通常有其他事情发生,使其不太有用。
'static
当某物需要 'static
时,这意味着所有引用都需要存在于 'static
之久。满足这一要求的一种方式是删除所有引用。在理想的世界中,你可以这样做:
let object = value as dyn MyTrait;
然而,Rust 不支持堆栈上或作为函数参数的动态大小类型。我们正试图消除所有引用,因此 &dyn MyTrait
不会起作用(除非你进行泄漏或拥有静态变量)。Box
允许你通过将它们放在堆上来拥有动态大小的类型,从而消除了生命周期。
你需要 Send
是因为从 Sync
升级到 Send
仅发生在 &
中,而不是在 Box
中。相反,当 T
是 Send
时,Box<T>
是 Send
。
Sync
更加微妙。虽然 spawn
不需要 Sync
,但异步块确实需要 Send + Sync
才能成为 Send
。由于 foo
接受 &self
,这意味着它返回一个持有 &self
的 Future
。然后会对该类型进行轮询,因此在轮询之间可能会在不同线程之间发送 &self
。如前所述,如果 T
是 Sync
,那么 &T
就是 Send
。然而,如果你将其更改为 foo(&mut self)
,它将在没有 + Sync
的情况下编译。 这是有道理的,因为现在可以检查它是否正在被同时使用,但在我看来,&self
版本可能在未来会被允许。
英文:
If you're not sure what Send
and Sync
mean, check out those documentation links. Something to note is that if T
is Sync
, then &T
is Send
.
Question #2 is simple: yes this is the right way to do it. async-trait
uses Pin<Box<dyn Future + Send>>
as its return type for basically the same reasons. Note that you can only add auto traits to trait objects.
For question #1, there's two issues: Send
and 'static
.
Send
When you cast something as dyn MyTrait
, you're removing all the original type information and replacing it with the type dyn MyTrait
. That means you lose the auto-implemented Send
and Sync
traits on MyStruct
. The tokio::spawn
function requires Send
.
This issue isn't inherent to async, it's because tokio::spawn
will run the future on its threadpool, possibly sending it to another thread. You can run the future without tokio::spawn
, for example like this:
fn main() {
let runtime = tokio::runtime::Runtime::new().unwrap();
let value = MyStruct::new();
let object = &value as &dyn MyTrait;
runtime.block_on(object.foo());
}
The block_on
function runs the future on the current thread, so Send
is not necessary. And it blocks until the future is done, so 'static
is also not needed. This is great for things that are created at runtime and contain the entire logic of the program, but for dyn Trait
types you usually have other things going on that makes this not as useful.
'static
When something requires 'static
, it means that all references need to live as long as 'static
. One way of satisfying that is to remove all references. In an ideal world you could do:
let object = value as dyn MyTrait;
However, rust doesn't support dynamically sized types on the stack or as function arguments. We're trying to remove all references, so &dyn MyTrait
isn't going to work (unless you leak or have a static variable). Box
lets you have ownership over dynamically sized types by putting them on the heap, eliminating the lifetime.
You need Send
for this because the upgrade from Sync
to Send
only happens with &
, not Box
. Instead, Box<T>
is Send
when T
is Send
.
Sync
is more subtle. While spawn
doesn't require Sync
, the async block does require Send + Sync
to be Send
. Since foo
takes &self
, that means it returns a Future
that holds &self
. That type is then polled, so in between polls &self
could be sent in between threads. And as before, &T
is Send
if T
is Sync
. However, if you change it to foo(&mut self)
it compiles without + Sync
. Makes sense since now it can check that it's not being used concurrently, but it seems to me like the &self
verison could be allowed in the future.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论