Feature scaling is a method used in machine learning for pre-processing data. This method is used to normalize the range of independent variables in the data, or features, so that the data should have the properties of a standard normal distribution with a mean of 0 and a standard deviation of 1. It is also known as data reduction or data normalization.
Feature scaling is generally performed after mean normalization and is the process of dividing each feature (a column in the data matrix) in the data set by a certain constant. It is used to avoid the potential numerical issues of the model by minimizing the numerical values of each variable to a reasonable range. This technique can be applied to speed up convergence rate for computationally expensive algorithms.
The two main types of feature scaling used in machine learning are:
* Standardization: This technique scales the values such that it follows a Gaussian distribution (also known as a bell curve) with a mean of 0 and standard deviation of 1. This is used in algorithms which require inversion of data, such as inverse distance-weighted interpolation and kernel-based learning methods.
* Normalization: This technique changes the values of numeric columns in the dataset to a common scale, usually 0 to 1. This is sometimes referred to as min-max scaling, as it also brings all values in the feature set into a specific range (usually 0 to 1). This technique is often used in neural networks for efficient training.
In addition to these two main types of feature scaling, some standard pre-processing methods are used for certain machine learning algorithms. For example, the normalization of logarithmic data for decision tree algorithms is commonly used to improve accuracy. Similarly, scaling and centering data for neural networks is done to avoid any numerical issues that may occur.
Feature scaling is an essential pre-processing step for many machine learning algorithms. Not performing this step can lead to inaccurate results due to irregularities in the data. It also helps improve the accuracy of the model by making the dataset more homogeneous and easy to process.
