How to clear Map entry after certain time of inactivity

I'm working on a chatbot powered by OpenAI. I'm using the new gpt-3.5-turbo model with ChatCompletion requests. I've already come up with a way for the bot to remember conversations for each individual user using a HashMap and it all works. The bot is able to remember all the previous responses and generate a new response using the history as context. However, the more responses in the query, the more tokens that it'll use, increasing costs. I want make it where if there is a certain amount of inactivity (like 2 minutes of not talking to the bot), it'll clear the history and start a fresh new conversation to save on tokens. What would be the best way to accomplish this?

Here is my code:

private static OpenAiService service;
private static Map<String, List<ChatMessage>> conversations = new HashMap<String, List<ChatMessage>>();
public static void initializeOpenAi() {
service = new OpenAiService(Settings.OPENAI_ACCESS_KEY);
}
public static String generateChatResponse(User user, String prompt) {
List<ChatMessage> conversation;
if (conversations.containsKey(user.getId())) {
conversation = conversations.get(user.getId());
} else {
conversation = new ArrayList<ChatMessage>();
}
if (conversation.size() < 1) {
ChatMessage system = new ChatMessage();
system.setRole("system");
system.setContent("Omnis is a chatbot with sarcasm");
conversation.add(system);
}
ChatMessage userPrompt = new ChatMessage();
userPrompt.setRole("user");
userPrompt.setContent(prompt);
conversation.add(userPrompt);
ChatCompletionRequest chatRequest = ChatCompletionRequest.builder()
.model("gpt-3.5-turbo")
.maxTokens(1000)
.temperature(0.9)
.topP(0.3)
.frequencyPenalty(0.9)
.presencePenalty(0.0)
.messages(conversation)
.build();
ChatCompletionResult chatResult = null;
try {
chatResult = service.createChatCompletion(chatRequest);
} catch (Exception e) {
System.out.println("An OpenAI request failed!");
if (e.getMessage().contains("timeout")) {
return "Your request timed out. Please try again after a breif wait for try a different request.";
}
if (e.getMessage().contains("Rate limit")) {
return "Rate limit for chat requests has been reached!";
}
return "Something went wrong with your request. Cause of error is " + e.getMessage();
}
if (chatResult != null) {
System.out.println("Created chat completion request that used " + chatResult.getUsage().getTotalTokens() + " tokens with " + chatResult.getModel());
}
String response = chatResult.getChoices().get(0).getMessage().getContent();
ChatMessage assistantPrompt = new ChatMessage();
assistantPrompt.setRole("assistant");
assistantPrompt.setContent(response);
conversation.add(assistantPrompt);
conversations.put(user.getId(), conversation);
return response;
}

The "generateChatResonse" method gets called each time a message is sent by a user. It get's the user's ID and pulls the conversation from the HashMap if it exists, if not it creates a new conversation.

I don't know what to try.

I don't want to clear the entire HashMap, just the entry associated with the inactive user.

I've added Guava to my project and replaced the Map with a LoadingCache object. It takes the same type of parameters and allows me to easily set the time before expiration.

private static LoadingCache<String, List<ChatMessage>> conversations;
public static void initializeOpenAi() {
service = new OpenAiService(Settings.OPENAI_ACCESS_KEY);
conversations = CacheBuilder.newBuilder()
.maximumSize(10000)
.expireAfterWrite(2, TimeUnit.MINUTES)
.build(
new CacheLoader<String, List<ChatMessage>>() {
@Override
public List<ChatMessage> load(String key) throws Exception {
return conversations.get(key);
}
});
}
public static String generateChatResponse(User user, String prompt) {
List<ChatMessage> conversation = conversations.getIfPresent(user.getId());
if (conversation == null) {
conversation = new ArrayList<ChatMessage>();
ChatMessage system = new ChatMessage();
system.setRole("system");
system.setContent("Omnis is a chatbot with sarcasm");
conversation.add(system);
}
// Remaining code is unchanged
}

How about a thread ? Check if there are changes , and if not for an amount of time to clear it ?