How to expand a row cell smoothly with a custom view in SwiftUI List?

I am trying to expand a row cell (which is a custom view) on tap gesture. The cell height has to be increased and a button is being moved in the expanded area. But I get the following jumpy animation effect. The pressed blue card shall remain steady but instead it jumps around.

Simple Code to reproduce this jumpy animation:

import SwiftUI

struct ContentView: View {
    var body: some View {
        VStack {
            Text("Hello, World!")
            List {
                Detail(isExpanded: false)
                Detail(isExpanded: false)
                Detail(isExpanded: false)
            }
        }
    }
}
struct ContentView_Previews: PreviewProvider {
    static var previews: some View {
        ContentView()
    }
}



struct Detail: View {
    @State var isExpanded :Bool
    var body: some View {
        ZStack {
            ZStack {
                RoundedRectangle(cornerRadius: 8)
                .fill(Color(red: 0.0, green: 1.0, blue: 1.0, opacity: 1.0)).frame(height: 115)
                Text("Hello, World!")
            }.zIndex(3).frame(height: 115).contentShape(Rectangle()).onTapGesture {
                withAnimation {
                    self.isExpanded.toggle()
                }
            }
            Button(action: {
            }) {
                ZStack {
                    RoundedRectangle(cornerRadius: 50)
                        .fill(Color(red: 1.0, green: 0.0, blue: 1.0, opacity: 1.0)).frame(height: 70)
                        .cornerRadius(8)
                        .shadow(radius: 3)
                        Text("Test")
                }
            }.padding([.leading, .trailing], 12)
                .padding(.top, 6)
                .frame(height: 70)
                .buttonStyle(BorderlessButtonStyle())
                .offset(y: self.isExpanded ? 0 : 40)
                .disabled(!self.isExpanded)
        }.frame(height: self.isExpanded ? 120 : 200)
    }
}

I am grateful for any tips or hints on how to smoothe this out.

Edit

When clicked the hello world card should remain aligned at the top in the cell itself. Like so: