Elastic Net

In our project, we used the Elastic Net regression model to perform feature selection and identify the most significant features for predicting credit scores. Here is a detailed explanation of the process we followed:

Fitting the Elastic Net Model: We used the Elastic Net regression model with specified hyperparameters (alpha=0.15 and l1_ratio=0.1). The alpha parameter controls the strength of regularization, while the l1_ratio parameter balances the Lasso and Ridge regularization. At first, when testing around with the hyperparameters, a l1_ratio of 0.5 was found to be too high as it extracted too many features in the data. We decided to look further into this by creating a correlation matrix with all the features in the dataset. While looking in the correlation matrix, we found that many of the features have some correlation relationship. The lasso regression model is not well suited for datasets with correlated features [14]. Since a lower l1 ratio means lasso regression has less of an effect on the feature selection, we ended up lowering it and decided on 0.1 in the end. 

Extracting Coefficients: After fitting the model, we extracted the coefficients associated with each feature and then select Non-Zero coefficient features. These coefficients indicate the importance of each feature in predicting the target variable.