在同步映射中的键并发操作

huangapple go评论87阅读模式
英文:

Concurrency on keys in a sync map

问题

尝试在golang中使用sync.Map来提供对特定字符串(比如"LOCK1")进行加锁的能力。

package main

import (
	"fmt"
	"sync"
	"time"
)

var lockMap sync.Map

func main() {

	counter := func(key string, routineId string) {
		_, ok := lockMap.Load(key)
		if ok {
			fmt.Println(key + " skipped by " + routineId)
			return
		}

		lockMap.Store(key, true)
		defer lockMap.Delete(key)
		for i := 0; i < 10; i++ {
			fmt.Println(key + " locked by " + routineId)
		}
	}

	// 启动一些goroutines
	go counter("LOCK1", "routine1")
	go counter("LOCK1", "routine2")
	go counter("LOCK1", "routine3")
	go counter("LOCK1", "routine4")
	go counter("LOCK1", "routine5")
	// 添加一些延时以便让这些goroutines执行一段时间
	time.Sleep(time.Second)
}

我知道sync.Map在内部使用了RWMutex。但是,我想了解一下sync.Map是否允许同时对不同的键进行多次写入,还是只能有一个例程同时对整个映射进行写入?

假设键"LOCK1"正在被routine1设置,那么routine2能否同时设置一个新值"LOCK2",还是它必须等待第一次写入完成?

英文:

Trying to use a sync map in golang to provide ability to acquire a lock on a particular string(say "LOCK1").

package main

import (
	&quot;fmt&quot;
	&quot;sync&quot;
	&quot;time&quot;
)

var lockMap sync.Map
func main(){

	counter := func(key string,routineId string) {
		_,ok := lockMap.Load(key)
		if(ok){
			fmt.Println(key+ &quot; skipped by &quot;+ routineId)
			return}

        lockMap.Store(key,true)
		defer lockMap.Delete(key)
		for i := 0; i &lt; 10; i++ {
			fmt.Println(key+ &quot; locked by &quot;+ routineId)
		}
	}

	// Starting some goroutines
	go counter(&quot;LOCK1&quot;,&quot;routine1&quot;)
	go counter(&quot;LOCK1&quot;,&quot;routine2&quot;)
	go counter(&quot;LOCK1&quot;,&quot;routine3&quot;)
	go counter(&quot;LOCK1&quot;,&quot;routine4&quot;)
	go counter(&quot;LOCK1&quot;,&quot;routine5&quot;)
	//adding some sleep so the  routines can execute for sometime
	time.Sleep(time.Second)
}

I am aware that sync map uses a RWMutex under the hood.
But, I am trying to understand if sync map allows multiple writes on different keys at the same time or only a single routine can do a write on the entire map at one time?

So let's say the key "LOCK1" is being set by routine1, would routine2 be able to set a new value "LOCK2" concurrently or does it have to wait for the first write to finish.

答案1

得分: 1

对于每个键,您希望访问现有的锁,如果没有锁,则创建一个新的锁。使用sync.Map.LoadOrStore,您可以原子地执行此操作。

// 预先分配一个新的锁,以防需要为此键创建锁。
newLockIfNeeded = new(sync.Mutex)
lock, _ = lockMap.LoadOrStore(key, newLockIfNeeded)
// 安全地获取了(可能是新的)键的锁
lock.Lock()
defer lock.Unlock()
// 进行工作...
英文:

For each key, you want to access the existing lock, or create a new lock if there isn't one. With sync.Map.LoadOrStore you can do this atomically.

// Pre-allocate a new lock, in case it&#39;s needed for this key.
newLockIfNeeded = new(sync.Mutex)
lock, _ = lockMap.LoadOrStore(key, newLockIfNeeded)
// Safely acquired the (potentially new) lock for the key
lock.Lock()
defer lock.Unlock()
// do work ...

答案2

得分: 1

我会尝试直接回答OP的问题。

sync.Map允许同时在不同的键上进行多个写操作吗?

是的,但有特定条件。

sync.Map针对具有多个读取器的唯一写入进行了优化。如果有多个例程更新同一个键,它的性能会较慢(参见sync.Map用例)。在内部,它包含一个读取映射、一个脏映射以及指针的原子读/写操作。某些操作使用互斥锁,而其他操作则绕过互斥锁以提高速度。


是的,只有在以下情况下才允许同时进行写入/删除操作:

  • 键已经存在(m.read.Load()
    • atomic.LoadPointer获取快速读取映射,但调用者仍然可以同时到达下一步
  • 上述键没有被更新/删除
    • 没有被更新:sync.Map检查expunged指针,以查看它是否在其他地方被更新
    • 没有被更新:在检查是否被清除时,键没有发生变化(CompareAndSwapPointer

是的,如果满足以下条件,同时进行读取是允许的:

  • 键不存在,且没有添加新键
  • 键存在且没有被更新(如上所述)

当满足以下情况时,会使用互斥锁:

  • 存储新键
    • 示例:批量添加不同的键会很慢
  • 在添加新键后加载丢失的键
    • 示例:频繁删除和检查删除的键会很慢
  • 读取或写入正在被更新的键
    • 示例:多个例程同时更新同一个键会很慢

然而,互斥锁不会阻塞“是”的情况。

英文:

I'll try to provide a direct answer to OP

> sync map allow multiple writes on different keys at the same time?

Yes, but under specific conditions

sync.Map is optimized for unique writes with many readers. It is slow for having many routines update the same key (see sync.Map use cases). Internally it contains a read map, a dirty map, and atomic read/writes to pointers. Some operations use a mutex, while others bypass it for speed.


Yes, simultaneous writes/deletes are allowed only if

  • The key already existed (m.read.Load())
    • atomic.LoadPointer gets the fast read map, but callers can still arrive at the next step together
  • The above key wasn't or isn't being updated/deleted
    • Wasn't updated: sync.Map checks the expunged pointer to see if it was updated elsewhere
    • Isn't being updated: key was not changed while checking if it was expunged (CompareAndSwapPointer)

Yes, simultaneous reads are allowed if either

  • The key does not exist while no new keys were added
  • The key exists and is not being updated (like above)

No, a mutex lock is used when

  • Storing a new key
    • Example: bulk adding distinct keys is slow
  • Loading a missing key after a new key was added
    • Example: frequent deleting and checking for deleted is slow
  • Reading or writing a key that is being updated
    • Example: many updating the same key is slow

However, the mutex does not block the Yes cases

答案3

得分: 0

我最终使用了一个键值对应的同步映射表,键为 key,值为 mutex。感谢 Hymns for Disco 给出的指导。以下是可工作的实现代码:

package main

import (
	"fmt"
	"sync"
	"time"
)

//var lockMap sync.Map
var lockMap2 sync.Map

func main(){

	//lockMap2 := make(map[string]*sync.Mutex)
	counter := func(key string,routineId string) {
		mutex,_ := lockMap2.LoadOrStore(key,&sync.Mutex{})
		mutex.(*sync.Mutex).Lock()
		defer mutex.(*sync.Mutex).Unlock()
		for i := 0; i < 100; i++ {
			fmt.Println(key+ " locked by "+ routineId)
		}
	}

	// Starting some goroutines
	go counter("LOCK1","routine1")
	go counter("LOCK1","routine2")
	go counter("LOCK1","routine3")
	go counter("LOCK1","routine4")
	go counter("LOCK1","routine5")
	//adding some sleep so the  routines can execute for sometime
	time.Sleep(time.Second)
}

希望对你有帮助!

英文:

I ended up using a sync map of key -> mutex. Thanks for the pointers Hymns for Disco. For reference here is the working impl:

package main

import (
	&quot;fmt&quot;
	&quot;sync&quot;
	&quot;time&quot;
)

//var lockMap sync.Map
var lockMap2 sync.Map
func main(){

	//lockMap2 := make(map[string]*sync.Mutex)
	counter := func(key string,routineId string) {
		mutex,_ := lockMap2.LoadOrStore(key,&amp;sync.Mutex{})
		mutex.(*sync.Mutex).Lock()
		defer mutex.(*sync.Mutex).Unlock()
		for i := 0; i &lt; 100; i++ {
			fmt.Println(key+ &quot; locked by &quot;+ routineId)
		}
	}

	// Starting some goroutines
	go counter(&quot;LOCK1&quot;,&quot;routine1&quot;)
	go counter(&quot;LOCK1&quot;,&quot;routine2&quot;)
	go counter(&quot;LOCK1&quot;,&quot;routine3&quot;)
	go counter(&quot;LOCK1&quot;,&quot;routine4&quot;)
	go counter(&quot;LOCK1&quot;,&quot;routine5&quot;)
	//adding some sleep so the  routines can execute for sometime
	time.Sleep(time.Second)
}

huangapple
  • 本文由 发表于 2022年6月30日 02:52:52
  • 转载请务必保留本文链接:https://go.coder-hub.com/72806454.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定