How to change SwiftUI view @State **without** animating

I have a view that reacts to changes from an external @ObservedObject therefore I am using an implicit animation to animate changes happening when something from the observed object changes:

.animation(.easeInOut(duration: 1))

This works great but I have a problem.

I also want the view to be draggable but when changing the offset state when the view is dragged, it now uses that slow animation. I have tried explicitly setting the animation to .none or nil but neither works.

So my question is, how can I have my explicit animation overrule my implicit animation as I would with highPriotityGesture or similar. Is this possible in SwiftUI?

I am using Xcode 12 and iOS 14.

Here is an example:

import SwiftUI

struct CardView: View {

    @ObservedObject var myObject: MyObject
    @State var translation = CGSize(width: 0, height: 0)

    var body: some View {
        ZStack {
            RoundedRectangle(cornerRadius: 12)
            Text(myObject.someVal)
        }
        .animation(.easeInOut(duration: 1))
        .offset(x: translation.width, y: translation.height)
        .gesture(
            DragGesture()
                .onChanged { value in
                    withAnimation(nil) {
                        // I don't want this to be animated!
                        translation = value.translation
                    }
                }
                .onEnded { _ in
                    // I'd preferably like to also animate this with .spring() and ignore the implicit animation
                    withAnimation(.spring()) {
                        translation = .zero
                    }
                }
         )
    }
}

Thanks!