LazyVGrid with Text() affects spacing when content is empty

Total newbie, in Swift and SwiftUI but I couldn't find a straightforward answer hence, asking here.

I have a LazyVGrid that contains a ForEach that renders RoundedRectangle that contains Text. The grid wraps the rectangles correctly and adds the spacing between the rows. However, when tapping on a rectangle, I remove the contained Text and when I do this for all items in a row, the items shift momentarily.

I get that there is some inner padding/margin/etc. going coming from within Text but this is not visible in the content view preview. I can of course, easily fix it by having an empty Text, but this doesn't feel right.

Can someone explain why this is happening and any ideas on how to properly fix it? -- Thanks

related code:

LazyVGrid(columns: [GridItem(.adaptive(minimum: 65))]) {
    ForEach(emojis[0..<emojiCount], id: \.self) { emoji in
        CardView(content: emoji).aspectRatio(3/4, contentMode: .fit)
    }
}

//----

struct CardView: View {
    var content: String
    @State var isFaceUp = true
    var body: some View {
        ZStack {
            let shape = RoundedRectangle(cornerRadius: 20)
            
            if isFaceUp {
                shape.fill().foregroundColor(.white)
                shape.strokeBorder(lineWidth: 3)
                Text(content).font(.largeTitle)
            } else {
                shape.fill()
            }
        }.onTapGesture {
            isFaceUp = !isFaceUp
        }
    }
}

I'm not entirely sure what is happening here either, but setting maxHeight to infinity works:

Text(content).font(.largeTitle).frame(maxHeight: .infinity)

ZStack would normally try to fit its contents, and the Text has some mysterious bottom padding, requiring the ZStack to expand. I suspect that this works by telling the Text to fit the ZStack, rather than the other way round.

The solution looks very similar to this question, where a Text has undesired padding because its frame is squeezed to fit the content, rather than the container.

Another solution is to use an overlay:

shape.strokeBorder(lineWidth: 3).overlay {
    Text(content).font(.largeTitle)
}

This works because the parent view's frame is not affected by its overlays, and so the existence/absence of an overlay does not change the parent view's frame.