Using SVMs

There are many packages we can use for implementing SVMs, but sklearn is one of the top Python packages for it. We can also use pycaret to easily search the hyperparameter space.

Using SVMs in sklearn

The sklearn package has a few different SVC and SVR implementations:

• Linear SVMs (svm.LinearSVC, LinearSVR, linear_model.SGDClassifier, and SGDRegressor)
• General SVMs (svm.SVC and SVR)
• Nu SVMs (svm.NuSVC and NuSVR)

The linear SVM can be implemented with svm.LinearSVC and svm.SVC, although the LinearSVC implementation is better (because it scales better to large datasets and has more flexibility, as described in the documentation: https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html). The SVC implementation allows any kernel to be used, and has pre-made options for using different kernels: polynomial (poly), RBF (rbf), and sigmoid (sigmoid).

The Nu SVMs introduce a hyperparameter, nu, which is an upper bound...