Embedded Files
Once the data has been processed, the SVM algorithm is utilized for modeling. The dataset is divided into two disjoint subsets, with a split of 70% for the training set and 30% for the testing set. There are 3 different kernels which are used in this case:
Linear Kernel
RBF Kernel
Polynomial Kernel
Regularization Parameter (C):
Regularization Parameter (C):
While modeling SVM in Python, the hyperparameter C is important to consider. This is because it represents the regularization parameter that determines the tradeoff between achieving a low training error and a low testing error. If the value of C is very high, it reduces the training error and it may lead to overfitting. A small C value allows misclassification leading to more training error. Having a balance of both helps in the reliability of the model in classification. By default, if not mentioned, Python uses the value of C as 1. Therefore selecting the correct value of C is important for achieving high accuracy, as well as good generalization of the model
The results of the model are shown below.
LINEAR KERNEL
LINEAR KERNEL
Model 1: Linear SVM with C = 1
Model 2: Linear SVM with C = 0.1
Model 3: Linear SVM with C = 10
From the result of Linear SVM, it can be observed that the model fails to classify the extended delay class, but it has accurately classified the short delay class. Another thing to note is that the change in C value hasn't affected the results. From this, it can be concluded that Linear Kernel is not a good fit for this data, as SVM's cannot draw a linear boundary accurately to separate these classes.
RBF KERNEL
RBF KERNEL
Model 1: RBF Kernel SVM with C = 1
Model 2: RBF Kernel SVM with C = 5
Model 3: RBF Kernel SVM with C = 10
From the result of RBF Kernel SVM, it can be observed that the model is performing better compared to linear SVM in terms of individual classes, but overall performance is low. With the change in the C value, it can be observed that there is a slight change in the accuracy. But again like linear SVM, RBF kernels SVM also fail to correctly classify the classes. From this, it can be concluded that RBF Kernel SVM is not a good fit for this data, as SVM's cannot draw a linear boundary accurately to separate these classes.
Polynomial KERNEL
Polynomial KERNEL
Model 1: Polynomial Kernel SVM with degree = 2 and C = 1
Model 2: Polynomial Kernel SVM with degree = 3 and C = 5
Model 3: Polynomial Kernel SVM with degree = 4 and C = 10
From the result of Polynomial Kernel SVM, it can be observed that the model is performing better for the extended delay class whereas it fails to classify the short delay class. Even with the change in the degree of curve or the C parameter, the performance hasn't been much improved. From this, it can be concluded that RBF Kernel SVM is not a good fit for this data, as SVM's cannot draw a linear boundary accurately to separate these classes.
Decision Boundary of Linear SVM
From the plot of the decision boundary of Linear SVM, it is clear why the model fails to give a good result. In the current feature space, the model cannot be linearly separable. May be the usage of some other kernel may yield better results. May be the usage of some other kernel may yield better results.
Decision Boundary of RBF Kernel SVM
From the plot of the decision boundary of RBF Kernel SVM, it is clear why the model fails to give a good result. In the current feature space, the model cannot be linearly separable. May be the usage of some other kernel may yield better results.
Page updated
Google Sites
Report abuse