A Novel Optimized Method for Feature Selection Using Non-linear Kernel-Free Twin Quadratic Surface Support Vector Machine

Abstract

Feature Selection (FS) is an important topic in the domain of machine learning. Support Vector Machine (SVM) is one of the most popular ML models for classification tasks. Efficient feature selection may ensure enhanced classification accuracy. Although there are several feature selection algorithms in practice, they are either separately used, combinedly used with linear SVM or, used with kernel-based SVM. Additionally, there exists another problem in classification methods, which is the selection of correct kernel function. There seems to be as such no general rule for selecting a kernel that maximizes the model’s performance. To handle the issues together, we propose a model called non-linear kernel-free twin quadratic surface SVM with optimized feature selection (LTQSSVM-OFS) which can eventually tackle both feature selection and classification tasks efficiently. We are doing feature selection in a kernel-free way by applying optimization method on a Laplacian Twin Quadratic Surface SVM classifier. We have validated our proposed model using (i) min-max approach-based SVM without FS, (ii) linear SVM, and (iii) radial basis function-based SVM models applied on four datasets, namely ‘Star3642balanced’, ‘Diabetes’, ‘Health care: Heart attack possibility’, and ‘Blood Transfusion’. Experimental results reveal that the proposed model outperforms the other models. Additional worth noting benefit of our model is that it yields better results in case of working with fewer features.