Differential Privacy decreases the model performance significantly

user_13163412 0
Viewed 350

Background Information

I trained a classifier to predict three labels: COVID/Pneumonia/Healthy based on chest X-Ray images. It's a PyTorch implementation of COVID-Net. I use a training set to train on, validation set to save the best performing model, and then a test set to measure the "real" performance of the model. However, I noticed that my model "learned" to classify normal/pneumonia really good, but it just ignored the underpopulated COVID set. Therefore I choose to undersample (reduce the number of training instances of the other classes (normal and pneumonia) in order to get equal populations). This worked well, but my sample set has been reduced to ~1500 samples (low!). The results are somewhat worse than COVID-Net, I achieve an accuracy of ~80% and lower sensitivity on underpopulated classes (COVID) then they report. I suppose that they report better performance because they do not use a validation-set and use the test-set each epoch. I figured that they might indirectly overfit on the test-set because of that. I have chosen to explain this so that the reader gets a context.

Question

I tried adding privacy to the training procedure by using Differential Privacy. Specifically, I used Facebook's PyTorch-DP module. Training works just as well if I choose to add almost no-privacy (this can be achieved by choosing a really low noise multiplier value (sigma), i.e. 1e-7) and a really high delta. So it's not that the module itself is not working/faulty, but, if I use a lower sigma (so I add more noise) then I get more privacy (epsilon decreases) but the model fails to fit the data at all. The question is: how do I manage to add privacy to a somewhat meaningful degree while making sure that my model somewhat fits the data still?

Performance differences

Confusion Matrix of Model without Differential Privacy added Confusion Matrix of Model without Differential Privacy added. It's not "good" but it's at least somewhat meaningful and the model reaches an accuracy of ~80%.

Confusion Matrix of Model with Differential Privacy (epsilon: 2.3) after 100 epochs Confusion Matrix of Model with Differential Privacy (epsilon: 2.3) after 100 epochs. It looks as if the model does not know what to do, at all.

Possible explanations

I read a paper that stated that adding Differential Privacy can cause bad performance in the sense that the accuracy decreases for underpopulated classes. But, I used undersampling and I think this should've solved that, but the accuracy stays bad (for all classes!).

Maybe because my sample set is so small, differential privacy is much harder to achieve, and therefore the performance is bad? However, even if add a really tiny bit of privacy, with an epsilon value >20000, the model still struggles in learning how to classify. So I'm not sure.

confusion-matrix pytorch privacy machine-learning python
1 Answers

It seems the PyTorch Differential Privacy library from Facebook Research is built on the concept of Renyi differential privacy guarantee that is well-suited for expressing guarantees of privacy-preserving algorithms and for composition of heterogeneous mechanisms. We need to have a good estimation of the heterogenity in this COVID-Net dataset.

In particular, Rényi divergence satisfies the data processing inequality. It seems the current library is more suitable for machine learning problems with more heterogeneity in datasets. This library is using an implementation of the differentially private stochastic gradient descent (SGD) algorithm. It follows the sequence of random initialisation, computation of gradient, clipping gradient, adding noise and doing the descent. The clipping and noise parameters may vary with the number of training steps and epochs.

The success of differential privacy on deep learning problems are driven by the extent of pre-processing the gradient to protect privacy and privacy_accounting which keeps track of the privacy spending over the course of training. It is highlighted that in the differentially private deep learning the model accuracy is more sensitive to training parameters such as batch size and noise level than to the structure of a neural network.

In the PyTorch library, we can see the examples on ImageNet, MNIST, DCGAN etc. In all these examples we can see how each of the mentioned parameters such as clipping, batch size etc. can be varied for getting the required accuracy levels. Kindly refer to the following example scripts in the PyTorch DP library.

PyTorch DP Example Scripts for Various Models

user_11533510
Related