英文:
DPDK multiple distributor
问题
我在使用DPDK distributor时遇到了一些问题。
我有2个lcore_rx线程和4个工作线程。
我有2个RX队列，每个lcore_rx线程将使用rte_eth_rx_burst获取数据包。
在获取数据包后，我想将数据包分配给每个分发器（例如，dist[0]，dist[1]）。
然后，1~4个工作线程将从分发器获取数据包，如下所示。
1~2个工作线程将请求dist[0]
3~4个工作线程将请求dist[1]
我认为这应该能够正常工作，但只有1~2个工作线程正常工作。
3~4个工作线程和第二个lcore_rx线程在获取了一些数据包后停止工作（大约10~20个数据包）。
看起来像是死锁。
是否可以使用多个分发器？
英文:
I have some problem with using DPDK distributor.
I have 2 lcore_rx threads and 4 worker threads.
I have 2 RX queue and each lcore_rx thread will get packet with rte_eth_rx_burst
after burst packet, I want set packet to each distributor (ex. dist[0], dist[1]).
Then 1~4 worker thread will get packet from distributor like below.
1~2 worker will request to dist[0]
3~4 worker will request to dist[1]
I Think this should be work, but only 1~2 worker work fine.
3~4 worker and second lcore_rx thread not working after few packet(10~20 packets)
It looks like dead-lock.
It is possible to use multiple distributor??
答案1
得分: 1
问题应该更清晰。你是使用软RSS还是硬件RSS？
假设你使用了硬件RSS，并配置了2个接收队列。
现在你的两个轮询线程，也就是DPDK接收进程，应该从这两个接收队列分别轮询。
//从第一个接收进程（rx_thread_0）
rte_eth_rx_burst(port, 0, ...);
//从第二个接收进程（rx_thread_1）
rte_eth_rx_burst(port, 1, ...);
这应该是没问题的。但如果你尝试从两个接收进程中的同一接收队列轮询，就需要一种锁定机制，这就是为什么应该分开轮询的原因。如果你配置了4个接收队列，那么你的两个接收进程可以分别从中轮询，第一个接收进程从0、1队列轮询，第二个接收进程从2、3队列轮询，但不能从两个进程中的同一接收队列轮询。
然后，你的接收进程会将这些数据包放入一个rte_ring中，对吗？确保每个rte_ring只有一个生产者和一个消费者，否则，你必须以不同的方式配置rte_ring（尽管在我的情况下这种方式不起作用）。所以，如果你有2个接收进程和4个工作进程，最好使用4个rte_ring。让你的前两个接收进程将数据包入队到rte_ring 0和1，第二个接收进程将数据包入队到rte_ring 2和3。
//从rx_thread_0
rte_ring_enqueue_bulk(rte_ring_0, ...);
rte_ring_enqueue_bulk(rte_ring_1, ...);
//从rx_thread_1
rte_ring_enqueue_bulk(rte_ring_2, ...);
rte_ring_enqueue_bulk(rte_ring_3, ...);
然后，你的4个工作进程应该独立地从这4个rte_ring中轮询，以获得尽可能高的吞吐量和速度。
//从工作线程_0
rte_ring_dequeue_burst(rte_ring_0, ...);
//从工作线程_1
rte_ring_dequeue_burst(rte_ring_1, ...);
//从工作线程_2
rte_ring_dequeue_burst(rte_ring_2, ...);
//从工作线程_3
rte_ring_dequeue_burst(rte_ring_3, ...);
英文:
The question should be more clear. Did you use soft RSS or HW RSS?
let's assume you used HW RSS, and configured 2 rx queues.
now your 2 polling threads aka DPDK rx processes should poll from these 2 rx queues separately.
//from 1st rx process (rx_thread_0)
rte_eth_rx_burst(port,0, ...);
// from 2nd rx process (rx_thread_1)
rte_eth_rx_burst(port,1, ...);
this should be fine. but if you try to poll from the same rx queue from more than 1 rx process it will need a lock mechanism which is why you should poll separately. if you configure 4 rx queue, then your 2 rx process can poll from these by 1st rx process polling from 0,1 queues, and 2nd rx process polling from 2,3 queues. but not the same Rx queue from the 2 processes.
and then your RX processes are putting those packets on a rte_ring right? make sure that 1 rte_ring has 1 producer and 1 consumer, otherwise, you have to configure the rte_ring differently (though it didn't work in my case). so if you have 2 rx processes and 4 worker processes, it's best to use 4 rte_rings. Make your first 2 rx processes enqueueing the packets to rte_ring 0 & 1, and second rx processes enqueueing the packets to rte_ring 2 & 3.
//from rx_thread_0
rte_ring_enqueue_bulk(rte_ring_0,...)
rte_ring_enqueue_bulk(rte_ring_1,...)
//from rx_thread_1
rte_ring_enqueue_bulk(rte_ring_2,...)
rte_ring_enqueue_bulk(rte_ring_3,...)
Then your 4 working processes should poll from this 4 rte_ring independently with the highest possible throughput at high speed.
//from working thread_0
rte_ring_dequeue_burst(rte_ring_0,...)
//from working thread_1
rte_ring_dequeue_burst(rte_ring_1,...)
//from working thread_2
rte_ring_dequeue_burst(rte_ring_2,...)
//from working thread_3
rte_ring_dequeue_burst(rte_ring_3,...)
答案2
得分: 0
Dpdk框架要求一个rxq只能同时在一个PMD lcore上运行，多个pmd polls需要锁定rxq以避免多核竞争。
> 多个逻辑核心永远不应共享接收或传输队列
> 因为这将需要全局锁并且会阻碍性能。
英文:
Dpdk framework requires that one rxq can only run on one PMD lcore at the same time, and multiple pmd polls need to lock rxq to avoid multi-core competition
> Multiple logical cores should never share receive or transmit queues
> for interfaces since this would require global locks and hinder
> performance.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论