How do I stop a SwiftUI view animation cascading?

I previously asked this question, but know I've distilled it down to a single block of example code that can be run in a playground.

The basis of the question is that I have two views being switched by a parent view. A content view and a settings view. The content view has 3 labels with the 3rd label's frame being set to match the widest size of the other two labels.

This works fine until I add an animation to the view that switches between the content view and settings view. But when I add that animation it 'cascades' into the content view, causing the 3 label to fly across the screen rather than just appear right justified.

Here's the sample code, cut and paste into a playground if you want to play with it.

import PlaygroundSupport
import SwiftUI

// Preference key which tracks the largest with of a label.
struct LabelWidthPreferenceKey: PreferenceKey {
    static var defaultValue: CGFloat = .zero
    static func reduce(value: inout CGFloat, nextValue: () -> CGFloat) {
        value = max(value, nextValue())
    }
}

// View modifier for tracking the size of a label.
struct LabelWidthObserver: ViewModifier {

    func body(content: Content) -> some View {
        content.background {
            GeometryReader { geometry in
                Color.clear.preference(key: LabelWidthPreferenceKey.self, value: geometry.size.width)
            }
        }
    }
}

struct RootView: View {
    @State private var showSettings = false
    var body: some View {
        Group {
            if showSettings {
                SettingsView { showSettings.toggle() }
            } else {
                ContentView { showSettings.toggle() }
            }
        }
        .animation(.easeInOut, value: showSettings)
    }
}

struct SettingsView: View {
    let toggleSettings: () -> Void
    var body: some View {
        Text("Here be settings")
        Button("Cancel") { toggleSettings() }
    }
}

struct ContentView: View {
    let toggleSettings: () -> Void
    @State private var labelWidth: CGFloat = 20.0
    var body: some View {
        VStack(alignment: .leading, spacing: 4.0) {
            Text("Line 1").modifier(LabelWidthObserver()).border(.red)
            Text("longer line two").modifier(LabelWidthObserver()).border(.red)

            Text("Line 3")
                .frame(minWidth: labelWidth, alignment: .trailing)
                .border(.green)
            Button("Show settings") { toggleSettings() }
        }
        .onPreferenceChange(LabelWidthPreferenceKey.self) {
            labelWidth = max(labelWidth, $0)
        }
    }
}

PlaygroundPage.current.setLiveView(RootView())

Does anyone have any idea how I can animate the transition between the two views without triggering an animation of the third label?