Mule 4 Object store and caching – need to store records from Db in cache and then use them from cache – is it possible?

huangapple go评论60阅读模式
英文:

Mule 4 Object store and caching - need to store records from Db in cache and then use them from cache - is it possible?

问题

这是对先前问题的追问:

使用 Mule 缓存,我是否可以刷新底层对象存储(被 Mule 缓存使用),以便定期从数据库检索数据并将其传输到对象存储?

我尝试实现的目标是每当收到请求时,尝试从缓存中检索数据。如果未找到(未命中),我不希望立即从数据库检索数据(相反,从数据库检索数据并将其推送到对象存储和缓存应该作为计划任务发生)。

根据先前的问题,我的理解是:

#1 使用 Mule 提供的内置缓存机制无法实现此操作,因为该缓存管理底层数据本身。

#2 我认为可以通过使用对象存储来实现我的用例,但不使用缓存。因此,我可以定期存储数据并在查询时使用它,但不能使用 Mule 缓存。

#3 也许比对象存储更好的解决方案是类似于 Redis 缓存,但这需要大量工作!

请确认这是否正确。谢谢。

英文:

This is a follow up question to an earlier one asked

So using Mule cache , can I refresh the underlying object store ( used by Mule cache ) so that periodically I can retrieve the data from database and pump it into object store .

What I am trying to achieve is whenever a request comes in - the data is attempted to be retrieved from cache . If not found ( miss ) I do NOT want to go immedicately retrieve data from the database ( rather retrieving data from database and pushing int into object store and cache should happen as a scheduled task )

Based on the earlier question my understanding is :

#1 This cannot be done with the inbuilt cache mechanism provided by Mule since this cache manages underlying data itself

#2 I think my use case can be achieved by using an Object store BUT without a cache - so while I can store data periodically and use it while querying I cannot use mule cache

#3 Probably a better solution / rather than an Object store could be something like redis cache but thats a lot of work !

Kindly confirm if this is correct
Thanks

This is a followup to earlier question

Update#1
So in this case I think a picture will be better to explain what I am trying to do and to also elaborate on what @aled mentioned in the first part of his answer ( thanks @aled )

Here is the customer request flow ( customer wants to retrieve employee data based on employee id )
customer flow

And here is the scheduler flow which is pumping data into cache:
enter image description here

So now the scheduler is able to load data into cache successfully
( used keyGenerationExpression='#[(payload.id default "")]'

So lets say the scheduler loads into cache :

	{
	"id" : "001",
	"name" : "John"
    }

Now if customer requests for id 001 it correctly retrieves from cache
However if customer requests for any other id ( say 002 ) then inadverently its a cache miss causing this id to be populated in the cache - this is NOT what I want , my requirement is to simply not populate the cache and return with empty results which tells me this empId is not present.

I am not sure if this is possible using Mule cache ?

Here is the code :
Customer flow :

<flow name="customer_requesting_info_flow">
	<http:listener doc:name="Listener" path="/caching/*" config-ref="HTTP_Listener_config"/>
	
	<set-payload value='#[%dw 2.0
output application/java
---
{
	"id" : payload.empId
}]' doc:name="get emp id from request and then try to extract details from cache"  />
	<ee:cache doc:name="Cache" cachingStrategy-ref="Caching_Strategy">
		<logger level="INFO" doc:name="Logger" message="Cache Miss - Not found in cache"/>
</ee:cache>
		
		<ee:transform doc:name="Transform Message" doc:id="72bf1493-7f55-42de-9d30-84a9b03d257f" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="9fdbd52e-4f31-414d-8443-2fc5fcde0298" message="done"/>
	</flow>

And here is the code for scheduler flow to push data into cache :

<flow name="load_cache">
		<scheduler doc:name="Scheduler">
			<scheduling-strategy >
				<fixed-frequency frequency="300000"/>
			</scheduling-strategy>
		</scheduler>
		<logger level="INFO" doc:name="Logger" message="started cron schedule"/>
<ee:transform doc:name="Transform Message">
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/java
---
	{
		"id" : "001",
		"name" : "John"
	}]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<flow-ref doc:name="load_cache_sub_flow" name="load_cache_sub_flow"/>
	</flow>
	<sub-flow name="load_cache_sub_flow">
		<ee:invalidate-cache doc:name="Invalidate Cache"  cachingStrategy-ref="Caching_Strategy"/>
		<ee:cache doc:name="Cache" cachingStrategy-ref="Caching_Strategy">
			<logger level="INFO" doc:name="Logger" message='#["Loading data into cache" ++ (payload.id)]'/>
		</ee:cache>
	</sub-flow>

答案1

得分: 2

你可以在不使用缓存范围并直接处理对象存储的情况下实现它。

  1. 创建一个对象存储并在从调度程序触发的流中填充它。使用 os:store 而不是缓存。
  2. 在查询所需数据的流/子流中,使用 os:retrieve 操作 不设置默认值。如果键存在,它将成功,并且您将获得缓存的数据。如果不存在,它将失败并显示 KEY_NOT_FOUND 错误,然后您可以根据需要处理数据。
英文:

You can achieve it without using the cache scope and handling the Object Store directly.

  1. Create an object store and populate that in a flow triggered from a scheduler. Use os:store and not cache
  2. In the flow/subflow that is querying the required data, use a os:retrieve operation without a default value. If the key is present it will be successful, and you will get the cached data. If not, it will fail with a KEY_NOT_FOUND error and you can handle the data as you like

答案2

得分: 1

一个可能的替代方案是将缓存范围放在子流程中。然后,您可以从带有调度程序的流程中调用该子流程,设置正确的键生成表达式,并且还可以从回答客户的流程中调用它。

您可以使用Mule SDK for Java实现自己的缓存模块。这需要Java开发工作,但您可以在Redis或其他您想要的实现中实现后端,包括如何刷新缓存。

此外,Redis可以用作Cache Scope的Object Store实现,但这并不改变缓存的功能,只是它的支持Object Store如何持久化数据。

使用缓存的通用子流程的示例(伪代码):


<flow name="load-cache">
  <scheduler...>
  生成数据
  <flow-ref name="cache"/>
</flow>

<flow name="listen">
  <http:listener ...>
  <flow-ref name="cache"/>
</flow>

<flow name="cache">
   <ee:cache ...>
    ...
   </ee:cache>
</flow>
英文:

One possible alternative would be to put the cache scope in a subflow. Then you can call that subflow from a flow with the scheduler, setting the right key generation expression, and also from the flow that answers the customer.

You can implement your own cache module with the Mule SDK for Java. That requires Java development effort, but you can implement your backend in Redis or whatever implementation you want, including how to refresh the cache.

Also Redis can be used as the implementation of an Object Store for the Cache Scope but that doesn't change the functionality of the cache, only how its backing Object Store persists data.

Example of using a common sub flow for the cache (pseudocode):


<flow name="load-cache">
  <scheduler...>
  Generate data
  <flow-ref name="cache"/>
</flow>

<flow name="listen">
  <http:listener ...>
  <flow-ref name="cache"/>
</flow>

<flow name="cache">
   <ee:cache ...>
    ...
   </ee:cache>
</flow>

huangapple
  • 本文由 发表于 2023年5月17日 17:12:07
  • 转载请务必保留本文链接:https://go.coder-hub.com/76270410.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定