英文:
Are Cassandra read and write to disks are asynchronous?
问题
我在Cassandra文档中阅读到(https://cassandra.apache.org/doc/latest/cassandra/operating/hardware.html)Cassandra以异步模式写入SSTable。我想问的是,所有写入是否都异步到磁盘?还有关于读取,它们是否也是异步的,还是根据情况而定?
英文:
I read in Cassandra documentation (https://cassandra.apache.org/doc/latest/cassandra/operating/hardware.html that Cassandra write to sstable in async mode. I want to ask that whether all writes are async to disc. Also what about reads are those also async or depends on case to case basis.
答案1
得分: 2
默认情况下,磁盘写入是异步的 - 有两种不同的写入需要考虑。
第一种是提交日志,默认情况下是异步的。行为可以通过更改某些设置来改变,同步得越频繁,写入性能/吞吐量就越低。
- commitlog_sync - 此值可以是
periodic
(默认)、group
或batch
- commitlog_sync_period - 默认值为10秒。
所以默认情况下,它会每10秒同步一次,使节点上的更改变得持久。每个节点独立地将数据同步到磁盘,因此在3个副本中,你可以期望平均每3.33毫秒将数据同步到其中一个节点的磁盘上。
第二种是刷新内存中的memtables,即当前保存在内存中的数据,它们可以在任何时候刷新,当内存中保存的数据超过阈值时。还有一些设置也会影响这种行为:
- memtable_heap_space
- memtable_offheap_space
- memtable_cleanup_threshold
除非你有一些特定的问题/存在刷新瓶颈的证据,否则我建议不要从默认值开始更改这些设置。
在你提到的问题中,你提到读取 - 它们是同步的(在大多数情况下) - 如果你在查询中请求数据,它将执行读取操作。
在Cassandra的3.11版本和更早的版本中,有可能会触发用于修复目的的异步读取的read_repair_chance。
英文:
By default, writes to disk are asynchronous - and there are two different writes to consider.
The first is the commit log which by default is asynchronous. The behaviour can be altered by changing some settings, the more frequently you sync, the lower performance / throughput you could expect on writes.
- commitlog_sync - this value is either
periodic
(default),group
orbatch
- commitlog_sync_period - default value of 10 seconds.
So by default, it will sync every 10 seconds, making the changes on that node durable. Each node sync's to disk independently, so across 3 replicas, you can expect the data to by synced every 3.33ms on average to once of the node's disks.
The second is the memtables being flushed, e.g. the data currently held in memory, they can flush at any time, when sufficient data is held in memory to exceed a threshold. There are a number of settings which also influence this behaviour:
- memtable_heap_space
- memtable_offheap_space
- memtable_cleanup_threshold
Unless you have some specific problems / evidence of a flush bottleneck, I would not alter these from the defaults to start with.
In the question you mention reads - they are synchronous (for the most part) - in that if you request the data in a query, it will perform the read.
In 3.11 and earlier versions of Cassandra, there was the potential for a read_repair_chance to trigger an asynchronous read for repair purposes.
答案2
得分: 0
严格来说,我认为你正在谈论两种不同的事情。
你所提到的“write”涉及到将提交日志与Unix的fsync()
函数一起持久化到磁盘,以使它们持久化。这与客户端/驱动程序的读/写请求是异步的概念完全不同。
默认情况下,Cassandra的提交日志由操作系统在缓存中进行缓冲,并且每隔10秒通过fsync()
刷新到磁盘。
另一方面,你可以配置你的应用程序执行异步读取或写入,以便客户端请求不会因等待协调器的结果或确认而被阻塞。
所以直接回答你的问题,不是所有的读取或写入都是异步的。这取决于你如何编写你的应用程序。我们建议使用驱动程序的异步API来最大化集群的性能。干杯!
英文:
Strictly speaking, I believe you are talking about two different things.
The "write" you are referring to relates to the commitlog being persisted disk with the Unix fsync()
function to make them durable. This is a completely different concept to the client/driver read/write requests being asynchronous.
By default, Cassandra's commitlog is buffered by the operating system in a cache the is periodically flushed to disk with fsync()
every 10 seconds.
On the other hand, you can configure your application to perform asynchronous reads or writes so that client requests are not blocked waiting for either the result or acknowledgement from the coordinator.
So to answer your question directly, not all reads or writes are asynchronous. It depends on how you've written your application. Our recommendation is to use the drivers' asynchronous API to maximise the performance of your cluster. Cheers!
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论