[TBD]Role of features in machine learning

Introduction

Laymen explanation

Features are the basic building blocks of datasets. The quality of the features in your dataset has a major impact on the quality of the insights you will gain when you use that dataset for machine learning.

If you want to know how it is related to ML engine performance, then this document helps.

Technical explanation

In machine learning and pattern recognition, a feature is an individual measurable property or characteristic of a phenomenon being observed. [1] Choosing informative, discriminating and independent features is a crucial step for effective algorithms in pattern recognition, classification and regression.

How machine use features

• While making the predictions, models use such features to make the predictions.

  • Diverse features in training dataset helps machine to generalise learning

How deep learning learns features

In supervised learning, Neural network learns features during the training. It learns using training dataset provided by the data scientist.

Desirable characteristic of training dataset

Dataset plays very important role in training neural networks

• • Dataset must include all samples which covers all features needed by machine for prediction

  • For each feature, dataset should have enough samples for machine to learn

  • Dataset must not contain unnecessary features since it can result in overfitting issue in ML

  • Dataset should also contain noisy samples. It helps the machine to be robust. For example, if dataset contains blur pictures, then trained machine will better predict pictures taken in the foggy weather.

Desirable characteristics of ML models

Many times, we need that An ML model should not be confined to looking at each feature individually. They should learn interactions among groups of features. For example, in a spam classifier, “Nigeria” and “Western Union” appearing together in an email can indicate spam but that separately they do not.

For such case handling non-linear model is needed, not the linear model.

Why linear model doesn't learn interactions among features?

Note that a linear model is linear combination of variables(features) and so, each item in its mathematical equation is a feature, not the multiples of features[Verify it]

So, we need those models, which use non-linear equations having non-zero term with multiple of variables.

Real life dataset problems

In real life, dataset may not be perfectly fitting to its requirement. To make to closer to perfection, feature engineering is needed. there are following approaches

• • One or more feature values may be missing in some samples. If this dataset is supplied, then machine learning will be skewed. To handle such case, ML concept namely imputation is used.

  • Based on domain, some features might be rare. In this case, machine will have too few samples to learn feature. For example, in credit card domain, fraud samples will be relatively very low compared to non-fraud case. To handle such case, ML concept namely upsampling/downsampling is used.

  • Similar to above, anomalies related samples are rare. To handle this, anomaly detection related ML algorithms are needed.

  • For making ML robust, noise is being introduced in the machine as part of pre-processing of training data. Noisy samples are created using algorithms. For example, incrementing the pixel value in existing sample can bring a new set of samples.

  • There are cases where dataset size itself is very low. In this case, there are machine learning techniques which generates labelled data using existing info. This document provides one such approach.

  • There can be features where their values are of non-compatible scales. This can affect the learning accuracy. Feature scaling can be needed for such case. This document provides more info on this.

  • Dataset can have redundant features (features which have co-relation with each other). Redundant features act as noise and such dataset increases learning time and also generalisation capability of ML algorithm. This document provides more info on this.

Reference

https://en.wikipedia.org/wiki/Feature_(machine_learning)

https://cogitotech.medium.com/what-are-features-in-machine-learning-and-why-it-is-important-e72f9905b54d

https://sites.google.com/site/jbsakabffoi12449ujkn/home/machine-intelligence/handling-missing-data-in-ml-dataset

https://sites.google.com/site/jbsakabffoi12449ujkn/home/machine-intelligence/handling-class-imbalance-in-machine-learning

https://machinelearningmastery.com/train-neural-networks-with-noise-to-reduce-overfitting/

https://www.tensorflow.org/neural_structured_learning/framework

https://www.datarobot.com/wiki/feature/

https://images.app.goo.gl/HwjSRYT12gLypmXJA

https://youtu.be/vuc93jbO2Dw?t=320

https://sites.google.com/site/jbsakabffoi12449ujkn/home/machine-intelligence/role-of-pca-in-current-data-science

https://youtu.be/y2Jsa4sgD5w?t=465

https://sites.google.com/site/jbsakabffoi12449ujkn/home/machine-intelligence/role-of-redundant-features-in-machine-learning

https://colab.research.google.com/github/d2l-ai/d2l-en-colab/blob/master/chapter_multilayer-perceptrons/dropout.ipynb?authuser=1#scrollTo=V8SZpajouNhM