如何在一定时间内的不活动后清除地图条目

huangapple
go评论125阅读模式
英文:

How to clear Map entry after certain time of inactivity

问题

我正在开发一个由OpenAI提供支持的聊天机器人。我正在使用新的gpt-3.5-turbo模型进行ChatCompletion请求。我已经想出了一种方法，让机器人记住每个用户的对话，使用HashMap来实现，一切都正常工作。机器人能够记住所有先前的回应，并使用历史作为上下文生成新的回应。然而，查询中的回应越多，它将使用的代币越多，增加成本。我希望如果有一定数量的不活动时间（比如与机器人不交谈的2分钟），它会清除历史记录并开始一个全新的对话，以节省代币。如何实现这一点的最佳方法是什么？

这是我的代码：

  1. private static OpenAiService service;
  2. 	
  3. private static Map<String, List<ChatMessage>> conversations = new HashMap<String, List<ChatMessage>>();
  4. 			
  5. public static void initializeOpenAi() {
  6. 	service = new OpenAiService(Settings.OPENAI_ACCESS_KEY);
  7. }
  9. public static String generateChatResponse(User user, String prompt) {
  10. 	List<ChatMessage> conversation;
  11. 		
  12. 	if (conversations.containsKey(user.getId())) {
  13. 		conversation = conversations.get(user.getId());
  14. 	} else {
  15. 		conversation = new ArrayList<ChatMessage>();
  16. 	}
  17. 		
  18. 	if (conversation.size() < 1) {
  19. 		ChatMessage system = new ChatMessage();
  20. 		system.setRole("system");
  21. 		system.setContent("Omnis is a chatbot with sarcasm");
  22. 		conversation.add(system);
  23. 	}
  24. 		
  25. 	ChatMessage userPrompt = new ChatMessage();
  26. 	userPrompt.setRole("user");
  27. 	userPrompt.setContent(prompt);
  28. 	conversation.add(userPrompt);
  29. 		
  30. 	ChatCompletionRequest chatRequest = ChatCompletionRequest.builder()
  31. 			.model("gpt-3.5-turbo")
  32. 			.maxTokens(1000)
  33. 			.temperature(0.9)
  34. 			.topP(0.3)
  35. 			.frequencyPenalty(0.9)
  36. 			.presencePenalty(0.0)
  37. 			.messages(conversation)
  38. 			.build();
  39. 		
  40. 	ChatCompletionResult chatResult = null;
  41. 		
  42. 	try {
  43. 		chatResult = service.createChatCompletion(chatRequest);
  44. 	} catch (Exception e) {
  45. 		System.out.println("An OpenAI request failed!");
  46. 			
  47. 		if (e.getMessage().contains("timeout")) {
  48. 			return "Your request timed out. Please try again after a brief wait or try a different request.";
  49. 		}
  50. 			
  51. 		if (e.getMessage().contains("Rate limit")) {
  52. 			return "Rate limit for chat requests has been reached!";
  53. 		}
  54. 			
  55. 		return "Something went wrong with your request. Cause of error is " + e.getMessage();
  56. 	}
  57. 		
  58. 	if (chatResult != null) {
  59. 		System.out.println("Created chat completion request that used " + chatResult.getUsage().getTotalTokens() + " tokens with " + chatResult.getModel());
  60. 	}
  61. 		
  62. 	String response = chatResult.getChoices().get(0).getMessage().getContent();
  63. 		
  64. 	ChatMessage assistantPrompt = new ChatMessage();
  65. 	assistantPrompt.setRole("assistant");
  66. 	assistantPrompt.setContent(response);
  67. 	conversation.add(assistantPrompt);
  68. 		
  69. 	conversations.put(user.getId(), conversation);
  70. 		
  71. 	return response;
  72. }

每当用户发送消息时，都会调用“generateChatResponse”方法。它获取用户的ID，并从HashMap中提取对话（如果存在的话），如果不存在，则创建一个新对话。

我不知道要尝试什么。

我不想清除整个HashMap，只想清除与不活动用户关联的条目。

英文:

I'm working on a chatbot powered by OpenAI. I'm using the new gpt-3.5-turbo model with ChatCompletion requests. I've already come up with a way for the bot to remember conversations for each individual user using a HashMap and it all works. The bot is able to remember all the previous responses and generate a new response using the history as context. However, the more responses in the query, the more tokens that it'll use, increasing costs. I want make it where if there is a certain amount of inactivity (like 2 minutes of not talking to the bot), it'll clear the history and start a fresh new conversation to save on tokens. What would be the best way to accomplish this?

Here is my code:

  1. private static OpenAiService service;
  2. private static Map&lt;String, List&lt;ChatMessage&gt;&gt; conversations = new HashMap&lt;String, List&lt;ChatMessage&gt;&gt;();
  3. public static void initializeOpenAi() {
  4. service = new OpenAiService(Settings.OPENAI_ACCESS_KEY);
  5. }
  6. public static String generateChatResponse(User user, String prompt) {
  7. List&lt;ChatMessage&gt; conversation;
  8. if (conversations.containsKey(user.getId())) {
  9. conversation = conversations.get(user.getId());
  10. } else {
  11. conversation = new ArrayList&lt;ChatMessage&gt;();
  12. }
  13. if (conversation.size() &lt; 1) {
  14. ChatMessage system = new ChatMessage();
  15. system.setRole(&quot;system&quot;);
  16. system.setContent(&quot;Omnis is a chatbot with sarcasm&quot;);
  17. conversation.add(system);
  18. }
  19. ChatMessage userPrompt = new ChatMessage();
  20. userPrompt.setRole(&quot;user&quot;);
  21. userPrompt.setContent(prompt);
  22. conversation.add(userPrompt);
  23. ChatCompletionRequest chatRequest = ChatCompletionRequest.builder()
  24. .model(&quot;gpt-3.5-turbo&quot;)
  25. .maxTokens(1000)
  26. .temperature(0.9)
  27. .topP(0.3)
  28. .frequencyPenalty(0.9)
  29. .presencePenalty(0.0)
  30. .messages(conversation)
  31. .build();
  32. ChatCompletionResult chatResult = null;
  33. try {
  34. chatResult = service.createChatCompletion(chatRequest);
  35. } catch (Exception e) {
  36. System.out.println(&quot;An OpenAI request failed!&quot;);
  37. if (e.getMessage().contains(&quot;timeout&quot;)) {
  38. return &quot;Your request timed out. Please try again after a breif wait for try a different request.&quot;;
  39. }
  40. if (e.getMessage().contains(&quot;Rate limit&quot;)) {
  41. return &quot;Rate limit for chat requests has been reached!&quot;;
  42. }
  43. return &quot;Something went wrong with your request. Cause of error is &quot; + e.getMessage();
  44. }
  45. if (chatResult != null) {
  46. System.out.println(&quot;Created chat completion request that used &quot; + chatResult.getUsage().getTotalTokens() + &quot; tokens with &quot; + chatResult.getModel());
  47. }
  48. String response = chatResult.getChoices().get(0).getMessage().getContent();
  49. ChatMessage assistantPrompt = new ChatMessage();
  50. assistantPrompt.setRole(&quot;assistant&quot;);
  51. assistantPrompt.setContent(response);
  52. conversation.add(assistantPrompt);
  53. conversations.put(user.getId(), conversation);
  54. return response;
  55. }

The "generateChatResonse" method gets called each time a message is sent by a user. It get's the user's ID and pulls the conversation from the HashMap if it exists, if not it creates a new conversation.

I don't know what to try.

I don't want to clear the entire HashMap, just the entry associated with the inactive user.

答案1

得分: 1

I've added Guava to my project and replaced the Map with a LoadingCache object. It takes the same type of parameters and allows me to easily set the time before expiration.

  1. private static LoadingCache<String, List<ChatMessage>> conversations;
  3. public static void initializeOpenAi() {
  4. 	service = new OpenAiService(Settings.OPENAI_ACCESS_KEY);
  5. 		
  6. 	conversations = CacheBuilder.newBuilder()
  7. 			.maximumSize(10000)
  8. 			.expireAfterWrite(2, TimeUnit.MINUTES)
  9. 			.build(
  10. 			    new CacheLoader<String, List<ChatMessage>>() {
  11. 					@Override
  12. 					public List<ChatMessage> load(String key) throws Exception {
  13. 						return conversations.get(key);
  14. 					}
  15. 			    });
  16. }
  18. public static String generateChatResponse(User user, String prompt) {
  19. 	List<ChatMessage> conversation = conversations.getIfPresent(user.getId());
  20. 		
  21. 	if (conversation == null) {
  22. 		conversation = new ArrayList<ChatMessage>();
  23. 			
  24. 		ChatMessage system = new ChatMessage();
  25. 		system.setRole("system");
  26. 		system.setContent("Omnis is a chatbot with sarcasm");
  27. 			
  28. 		conversation.add(system);
  29. 	}
  31. 	// Remaining code is unchanged
  32. }
英文:

I've added Guava to my project and replaced the Map with a LoadingCache object. It takes the same type of parameters and allows me to easily set the time before expiration.

  1. private static LoadingCache&lt;String, List&lt;ChatMessage&gt;&gt; conversations;
  2. public static void initializeOpenAi() {
  3. service = new OpenAiService(Settings.OPENAI_ACCESS_KEY);
  4. conversations = CacheBuilder.newBuilder()
  5. .maximumSize(10000)
  6. .expireAfterWrite(2, TimeUnit.MINUTES)
  7. .build(
  8. new CacheLoader&lt;String, List&lt;ChatMessage&gt;&gt;() {
  9. @Override
  10. public List&lt;ChatMessage&gt; load(String key) throws Exception {
  11. return conversations.get(key);
  12. }
  13. });
  14. }
  15. public static String generateChatResponse(User user, String prompt) {
  16. List&lt;ChatMessage&gt; conversation = conversations.getIfPresent(user.getId());
  17. if (conversation == null) {
  18. conversation = new ArrayList&lt;ChatMessage&gt;();
  19. ChatMessage system = new ChatMessage();
  20. system.setRole(&quot;system&quot;);
  21. system.setContent(&quot;Omnis is a chatbot with sarcasm&quot;);
  22. conversation.add(system);
  23. }
  24. // Remaining code is unchanged
  25. }

答案2

得分: -1

检查是否有更改，如果没有，经过一段时间将其清除？

英文:

How about a thread ? Check if there are changes , and if not for an amount of time to clear it ?

huangapple
  • 本文由 发表于 2023年3月4日 02:32:16
  • 转载请务必保留本文链接：https://go.coder-hub.com/75630689.html
  • java
  • openai-api
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定