Serious Lag When Switching Between Views When Using HoverEffect/ContextMenu

After adding a HoverEffect to a Button, that when tapped, changes the current view, the Button with the HoverEffect delays when hiding, overlaying the new view. The same thing also occurs with ContextMenu.

Screenshot of lag/overlay:

This lasts for around a second before the tapped Button finally hides.

Example Code:

struct ContentView: View {
  @State var showOtherButton = false
   
  var body: some View {
    if showOtherButton {
      Button {
        showOtherButton.toggle()
      } label: {
        Text("First Button")
      }
      .padding()
      .hoverEffect()
      .contextMenu {
        Button {
          showOtherButton.toggle()
        } label: {
          Text("Toggle Button")
        }
      }
    } else {
      Button {
        showOtherButton.toggle()
      } label: {
        Text("Second Button")
      }
      .padding()
      .hoverEffect()
      .contextMenu {
        Button {
          showOtherButton.toggle()
        } label: {
          Text("Toggle Button")
        }
      }
    }
  }
}

Is this a known SwiftUI bug? Any way to fix this? I've thought about using a delay after tapping the Button between switching the views, but that's not a good solution.