Is Scikit Learn's Support Vector Classifier hard margin or soft margin

user_13191462 0
Viewed 846

I was wondering if the support vector classifier below from scikit learn is of hard margin or soft margin?

from sklearn import svm
clf = svm.SVC()
scikit-learn svm
1 Answers

Though very late, I don't agree with the answer that was provided for the following reasons:

  • Hard margin classification works only if the data is linearly separable (and be aware that the default option for SVC() is that of a 'rbf' kernel and not of a linear kernel);
  • The primal optimization problem for an hard margin classifier has this form:
  • On the other hand, as you might see from the guide, the considered primal optimization problem in scikit-learn is the following:

This formulation - which in literature identifies the optimization problem for a soft margin classifier - makes it work also for non-linearly separable datasets and introduces:

  • zeta_i, which measures how much instance i is allowed to violate the margin (the functional margin can be less than 1 in passing from the first to the second formulation);
  • hyperparameter C, which is a portion of the "penalty" imposed to the objective function for permitting instances to have functional margin less than 1.

Eventually, as you might see in sklearn doc, hyperparameter C must be strictly positive, which enforces the idea that SVC() does provide soft margin classification. Here is another SO reference.

user_133503410
Related