在监控资源使用情况时请求资源。

huangapple go评论108阅读模式
英文:

Requesting resources when monitoring the resource usage

问题

我尝试监控我的资源,并根据文档中的解释这里做到了。

我已经成功实现了,您可以在下面的代码中看到。我的问题是当我有多个资源实例时,我无法弄清楚如何调用一个可用的资源,或者在请求时它们都忙碌时如何等待直到有一个可用。

以下是一个简单的模型,其中一个项目依次经过两个任务,两个任务都由我有十个实例的相同资源完成。我将资源实例添加到一个字典中,同时还显示了记录的时间是在第一个任务还是第二个任务中完成的。

  1. # 代码部分

正如您在这段代码中所看到的,我使用了项目ID来调用十个资源中的一个,使用id%len(10)作为调用特定资源的技巧,使代码工作。我想要能够调用这十个资源中的任何一个来执行一个任务。如果所有十个都不可用,那么进程会等待直到有一个可用。

我尝试使用循环迭代资源并查找可用的资源,如下所示:

  1. # 代码部分

不幸的是,这没有返回任何结果,所有记录的时间都是零。我希望有人能帮助解决这个问题。另外,我想按ID记录每个项目的时间。问题是,我在哪些地方记录这些时间,使用的是env.now(),是否正确?正如您所看到的,我有task1_queue、task1_start等等。我只想确保我在正确的地方记录了这些时间。

谢谢大家。这个社区在我学习SimPy的过程中非常有帮助。

英文:

I try to monitor my resources and as explained in the documentation here

I was able to achieve that as you can see in the code below. My issue is when I have multiple instance of a resource, I cannot figure out how to call an available resource, or wait until one becomes available if they all are busy when requested.

Here is a simple model where an item goes through two tasks sequentially and both tasks are done by the same resource that I have ten instances of. I add the resource instance to a dictionary along with a label showing whether the recorded time was done in the first or second task

  1. import simpy
  2. import random
  3. def arrival(env, worker, arrival_time, task1_mean, task1_std, task2_mean, task2_std):
  4. id = 0
  5. while True:
  6. w = first_task(env, worker, task1_mean, task1_std, task2_mean, task2_std, id)
  7. env.process(w)
  8. yield env.timeout(random.expovariate(1/arrival_time))
  9. id += 1
  10. def first_task(env, worker, task1_mean, task1_std, task2_mean, task2_std, id):
  11. task1_queue = env.now
  12. with worker[id % len(10)]['time'].request() as req:
  13. task1_start = env.now
  14. worker[id % len(10)]['task'].append('Task 1')
  15. yield req
  16. yield env.timeout(abs(random.normalvariate(task1_mean, task1_std)))
  17. task1_end = env.now
  18. w = first_task(env, worker, task2_mean, task2_std, id)
  19. env.process(w)
  20. def second_task(env, worker, task2_mean, task2_std, id):
  21. task2_queue = env.now
  22. with worker[id % len(10)]['time'].request() as req:
  23. worker[id % len(10)]['task'].append('Task 2')
  24. task2_start = env.now
  25. yield req
  26. yield env.timeout(abs(random.normalvariate(task2_mean, task2_std)))
  27. task2_end = env.now
  28. class TimeMonitoredResource(simpy.Resource):
  29. def __init__(self,*args,**kwargs):
  30. super().__init__(*args,**kwargs)
  31. self.start_time = None
  32. self.end_time = None
  33. self.working_time = []
  34. self.starting_time = []
  35. self.releasing_time = []
  36. def request(self,*args,**kwargs):
  37. self.start_time = self._env.now
  38. return super().request(*args,**kwargs)
  39. def release(self,*args,**kwargs):
  40. self.end_time = self._env.now
  41. self.working_time.append(self.end_time - self.start_time)
  42. self.starting_time.append(self.start_time)
  43. self.releasing_time.append(self.end_time)
  44. return super().release(*args,**kwargs)
  45. def calculate_metrics(op):
  46. total_operation_time = []
  47. total_starting_time = []
  48. total_releasing_time = []
  49. total_operation_time = op.working_time
  50. total_starting_time = op.starting_time
  51. total_releasing_time = op.releasing_time
  52. return total_starting_time, total_releasing_time, total_operation_time
  53. env = simpy.Environment()
  54. worker = {}
  55. for i in range(10):
  56. worker[i] = {'time': TimeMonitoredResource(env, 1), 'task': []}
  57. task1_mean, task1_std, task2_mean, task2_std = 6, 2, 8, 2
  58. arrival_time = 2
  59. env.process(arrival(env, worker, arrival_time, task1_mean, task1_std, task2_mean, task2_std))
  60. env.run(1200)
  61. worker_op_start = {}
  62. worker_op_release = {}
  63. worker_op_total = {}
  64. worker_op_task = {}
  65. for i, op in worker.items():
  66. start, release, total = calculate_metrics(op['time'])
  67. worker_op_start[i] = start
  68. worker_op_release[i] = release
  69. worker_op_release[i] = total
  70. worker_op_task[i] = op['task']

As you can see in this code I used the item id to call one of the ten resources using id % len(10) as a trick to call a specific resource and make the code work. What I want to be able to achieve is calling any of the ten resources to do a task. If all ten are not available, then the process wait until one becomes available.

I tried to use a loop and iterate over the resources and find an available one as follows:

  1. for _, resource in worker.items():
  2. if resource['time'].count == 0:
  3. with resource['time'].request() as req:
  4. resource['task'].append('Task 1')
  5. yield env.timeout(abs(random.normalvariate(task1_mean, task1_std)))
  6. yield req
  7. break

Unfortunately, this retuned nothing and all the recorded times were zeros. I hope someone can help with this. Also I want to record the time for each item by id. The question is, is the places where I recorcd those times using env.now() correct or not? As you can see I have task1_queue, task1_start, etc. I just want to make sure I'm recording these times in the right place.

Thank you all. This community has been really helpful in my SimPy journey.

答案1

得分: 2

  1. 问题在于使用 simpy.Resource 收集资源级别的资源指标时,simpy.Resource 没有资源对象。它只有可用/部署资源的计数。我创建了一个资源对象来收集指标,并使用了一个 simpy.Store 来管理请求。我将 simpyStore 封装在一个类中,以在请求和释放时更新资源对象。使用 simpy store 的问题是它没有上下文管理器来自动将资源返回到 simpy.Store,就像 simpy.Resouce 池一样。所以我创建了一个特殊的 simpy Event,通过添加一个上下文管理器来包装 simpy.Store get(),并在 "with" 退出时添加了一个释放调用。没有进行太多测试,但在正常情况下可以工作。所以在这个示例中,你得到了两件事情。一个带有上下文管理器的 simpy.Store,会在 "with" 退出时返回资源,并且会收集资源级别的使用指标。比我想象的要复杂一些。
英文:

So the problem with collecting resource metrics at the resource level with a simpy.Resource is simpy.Resource does not have resouce objects. It only has a count of available / deployed resources. I created a resource object to collect the metrics and used a simpy.Store to manage the requests. I wrapped the simpyStore in a class that updates the resource objects as they are requested and released. The issue with a simpy store is that it does not have a context manager to automatically return the resources to the simpy.Store like a simpy.Resouce pool does. So I created a special simpy Event by adding a context manager which wraps the simpy.Store get(), and adds a release call on the "with" exit. Did not do a lot of testing, but it works in the happy path. So your getting two things in this example. a simpy.Store with a context manager the returns resources on exit of the "with", and resources that collect usage metrics at the resource level. This was a bit more trick then I thought it would be.

  1. """
  2. Demo of resources where each resource collects usage data
  3. Composes a resource pool that uses a simpy.Store to hold
  4. resouces objects. The pool updates the resouce stats as
  5. they are requested and released
  6. Creatd a special event to act as a context manager for
  7. resource requests
  8. programmer: Michael R. Gibbs
  9. """
  10. import simpy
  11. import random
  12. class ResourceEvent(simpy.Event):
  13. """
  14. A event for rquesting a resouce.
  15. Is also a context manager so
  16. 'with' syntax can be use to
  17. atomatic release the resource
  18. event returns a resouce when it succeeds
  19. """
  20. def __init__(self, env, request_process, release_process):
  21. """
  22. starts the process to get a resouce
  23. saves processes
  24. parameters
  25. ----------
  26. request_process: process
  27. a process to get the resource
  28. release_process: process
  29. a process to release a resouce
  30. """
  31. super().__init__(env)
  32. self.env = env
  33. self.request_process = request_process
  34. self.release_process = release_process
  35. env.process(self._echo())
  36. def _echo(self):
  37. """
  38. process to get a resouce
  39. triggers this event when process gets the resouce
  40. returning the resource
  41. """
  42. self.request_event = self.env.process(self.request_process())
  43. resource = yield self.request_event
  44. # save resouce so it can be released later
  45. self.resource = resource
  46. self.succeed(resource)
  47. def __enter__(self):
  48. """
  49. Context enter, returns self
  50. """
  51. return self
  52. def __exit__(self, *arg, **karg):
  53. """
  54. Context exit
  55. if the resource request was successful, then
  56. return the resouce
  57. else
  58. cancel the pending resouce request
  59. """
  60. if self.request_event.triggered:
  61. self.release_process(self.resource)
  62. else:
  63. self.request_event.cancel()
  64. class MyResourcePool():
  65. """
  66. Wraps a simpy.Store to behaive more like a simpy.Resouce
  67. Uses resouces objects that collect usage stats
  68. """
  69. def __init__(self, env, recourse_cnt):
  70. """
  71. creates a simpy.Store and load it with the desirec
  72. number of resouces
  73. parameter
  74. ---------
  75. env: simpy.Environment
  76. resourse_cnt: int
  77. number of resouces in the pool
  78. """
  79. self.env = env
  80. self._pool = simpy.Store(env)
  81. self.resources = [] # master list to simplify resouce interation
  82. for i in range(recourse_cnt):
  83. resource = self.MyResource(i+1)
  84. self.resources.append(resource)
  85. self._pool.put(resource)
  86. def _request_event(self):
  87. """
  88. helper method to get a resouce
  89. and to update metrics for the resouce
  90. """
  91. print(f'{self.env.now} requesting a resouce')
  92. request_start = self.env.now
  93. # using a "with" insures the get gets
  94. # cancled if this process gets canceled
  95. with self._pool.get() as req:
  96. resource = yield req
  97. request_time = self.env.now - request_start
  98. resource.req_wait_time += request_time
  99. idle = self.env.now - resource._relase_time
  100. resource.idle_time += idle
  101. # save dispatch time to calc the busy/work time later
  102. resource._dispatch_time = self.env.now
  103. resource.busy = True
  104. print(f'{self.env.now} resouce {resource.id} has been deployed')
  105. return resource
  106. def request(self):
  107. """
  108. Creates a event that returns a resouce when a resouce
  109. becomes available
  110. event is also a context manager so it can be uses
  111. in a 'with' syntax
  112. """
  113. req = ResourceEvent(self.env, self._request_event, self.release)
  114. return req
  115. def release(self, resource):
  116. """
  117. Release a resouce and updates it stats
  118. """
  119. print(f'{self.env.now} resouce {resource.id} has been released')
  120. if resource in self.resources:
  121. work = self.env.now - resource._dispatch_time
  122. resource.busy_time += work
  123. resource._relase_time = self.env.now
  124. resource.busy = False
  125. self._pool.put(resource)
  126. else:
  127. raise Exception("Trying to release a object that does not belong to pool")
  128. class MyResource():
  129. """
  130. Resouce class for collecting usage stats
  131. """
  132. def __init__(self, id):
  133. """
  134. initalize collectors and trackers
  135. """
  136. self.id = id
  137. self.busy = False
  138. self.req_wait_time = 0
  139. self.busy_time = 0
  140. self.idle_time = 0
  141. self._dispatch_time = 0
  142. self._relase_time = 0
  143. def use_resouce(env, pool):
  144. """
  145. test grabing a resource
  146. """
  147. with myPool.request() as req:
  148. resource = yield req
  149. yield env.timeout(random.triangular(1,3,10))
  150. def test(env, myPool):
  151. # doing a test without the with syntax
  152. req = myPool.request()
  153. resource = yield req
  154. yield env.timeout(5)
  155. myPool.release(resource)
  156. # do real stress test
  157. for _ in range(20):
  158. env.process(use_resouce(env, myPool))
  159. yield env.timeout(random.triangular(1,1,4))
  160. env = simpy.Environment()
  161. myPool = MyResourcePool(env, 3)
  162. env.process(test(env, myPool))
  163. env.run(100)
  164. print()
  165. print('done')
  166. for r in myPool.resources:
  167. print(f'id: {r.id}, busy: {r.busy_time}, idle: {r.idle_time}, usage: {r.busy_time / (r.busy_time + r.idle_time)}')

huangapple
  • 本文由 发表于 2023年3月23日 08:59:31
  • 转载请务必保留本文链接:https://go.coder-hub.com/75818447.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定