blur all elements except one in a div

I am trying to blur all elements in a div except one. I have looked at many similar questions like this one and I get this:

.table-responsive:not(.confirm) {
    filter: blur(3px);
    pointer-events: none;
}

This is basically supposed to blur all elements except the confirm class. Here is a js fiddle demonstrating it: https://jsfiddle.net/qbuyuhts/1/

It's some janky html but does the job to show the problem. Does anyone know why the .confirm div is blurring?