How to remove or replace a view modifier on a view in SwiftUI?

I am trying to create a view modifier that takes a sound file as input and plays it on tap of the view.

struct AddSoundModifier: ViewModifier {
    var soundType: TapSoundType = .none

    func body(content: Content) -> some View {
        content
            .simultaneousGesture(TapGesture()
                .onEnded {
                    self.playSoundOnTap()
                })
    }
    
    private func playSoundOnTap() {
        SoundUtility.playBackgroundSoundWithUrl(soundType: soundType, bundle: nil, loopCount: 0)
    }
}

However, I also want to dynamically remove or change the sound in a different part of the code (essentially, I want the modifier to be uniquely applied for a view instance and override any previously applied modifier of the same type, in this case AddSoundModifier ), but I can't figure out how to store a reference to the previous view modifier or the gesture. In the objective-C world, I could use objc_getAssociatedObject and objc_setAssociatedObject to add functionalty in extensions, but is it possible in SwiftUI?

extension View {
    
    func addingSound(_ soundType: TapSoundType) -> some View {
        modifier(AddSoundModifier(soundType: soundType))
    }
}

Text("hello")
.addingSound(.primary)
.addingSound(.none) //This should cancel the previous modifier