Limitations of linear regression

Limited to Linear Relationships: Linear regression cannot capture complex, nonlinear relationships between variables. If the true relationship is nonlinear, linear regression may not provide accurate predictions.

Sensitivity to Noise: another apparent disadvantage of linear regression is its high sensitivity to noise. Within the training set, the presence of noisy data points (incorrect data) can significantly influence the model’s performance.

This issue can be addressed by preprocessing the data, such as removing noise before utilizing it. Alternatively, it can also be mitigated by employing loss functions like Huber loss.

Huber loss is indeed a combination of mean squared error (MSE) and mean absolute error (MAE), which can be represented as follows:

L_δ (y_i,f(x_i ))={█(1/2 (y_i-f(x_i ))^2 for |y_i-f(x_i )|<δ,@δ×(|y_i-f(x_i )|-1/2 δ), otherwise.)┤