How to persist view size within HSplitView separator?

I have a HSplitView with two views. When I want to drop a file to this view, I replace the view with a Drop file here label. If I choose to abort the drop operation, view sizes get messed up, one view takes complete priority over another. If I complete the drop, things remain okay. So, in general, how to manage this in HSplitView?

Reproducible code below. Before dropping a file, you need to move the separator to another desired position.

struct ContentView: View {
    @State private var isDragging: Bool = false

    var body: some View {
        GeometryReader { geo in
            if isDragging {
                VStack {
                    Spacer()
                    Text("Drop file here")
                    Spacer()
                }
            } else {
                HSplitView {
                    ZStack {
                        Rectangle()
                            .fill(.background)
                        Text("Some Text")
                    }
                    ZStack {
                        Rectangle()
                            .fill(.background)
                        Text("Some Other Text")
                    }
                }
            }
        }
        .padding()
        .frame(width: 500, height: 500)
        .onDrop(of: ["public.file-url"],
                delegate:
                    DropFileDelegate<Any>(isDragging: $isDragging.animation()))
    }
}

struct DropFileDelegate<Model>: DropDelegate {
    @Binding var isDragging: Bool

    func dropExited(info: DropInfo) {
        isDragging = false
    }
    
    func dropUpdated(info: DropInfo) -> DropProposal? {
        isDragging = true
        return nil
    }

    func validateDrop(info: DropInfo) -> Bool {
        return info.hasItemsConforming(to: ["public.file-url"])
    }
    
    func performDrop(info: DropInfo) -> Bool {
        return true
    }
}