How do I forcibly remove focus from an element on screen-readers?

I'm working on a single-page app where components get generated and destroyed constantly on the DOM. I'm using the following button with ARIA labels so it's selectable and readable by screen-readers:

<div class="button" role="button" aria-label="Select">Select</div>

In the screenshot below you can see the iOS safari screen-reader focuses on it as expected.

When the user clicks on the button, I remove it from the DOM. However, the screen-reader focus remains where that button used to be:

I've tried changing role="none", adding aria-hidden="true", and removing the aria-label for a full second before removing the button:

<div class="button" role="none" aria-hidden="true">Select</div>

... but nothing seems to remove this focus after the button is destroyed.
Is there a way to forcibly remove the focus from this element? Or is there a way to force another element to receive focus instead (like a persistent website logo) so the box doesn't stay floating over an empty space?

The only way to forcibly remove focus is by putting it somewhere else. Focus is always on some element, it must never get lost.

For keyboard users, it must go somewhere that makes sense, on an element that is the next step in the task flow.

For example, for a modal dialog, focus must go from the triggering button to an element inside the dialog, and when closing it, it must go back to the button.

If there is no logical next element, the patterns should be reconsidered. In general, removing the very element that is pressed is quite unexpected behaviour to any user.

See also this discussion: aria-expanded on the button that gets removed from DOM

Don’t forget that if using a custom button instead of a real one, you also need to handle keyboard events.