REGRESSION METRICS

Theoritical Understanding

Why we are going to use the regression metrics?

  • It is used whenever we have a regression problem and we need to know how well our model is performing, and how well it is doing it.
  • Metrics measures the performannce of Machine Learning Algorithms.
  • Metrics help to select the best algorithm for the problem.
  • It helps to understand the various charecteristics of the results.
  • This is a way to evaluate Algorithms

    Types of Regression Metrics:

  • R Squared

  • It is a measure of how close the regression line is to the actual data.
  • It is a measure of goodness of the best fit line.
  • Higher the value,better the fit.
  • Range of values: 0 to 1, but when our best fit line is worse than our average line then it is less than 0.
  • regm8
    • Residuals are the difference between the actual data and the predicted data.
    • Let’s say y is the actual data and yhat is the predicted data, then the residuals are y - yhat.
    • Sum of square of residuals is the sum of the squared residuals.
    • Total sum of square is the sum of the squares of the residuals but here instead of using best fit line we are using a avarage line and find the residuals.
      • regm7

  • Adjusted R Squared

    • Why we need adjusted R Squared?

      • As we add a feature in a linear regression model, value of R squared increases. It is due to the property of Linear Regression that it assigns a value to a coefficient of a feature such that sum of squares of residuals is minimized. So R squared increases and it never decreases.
      • To penalize the model for adding a feature and to decrease the value of R squared we use the adjusted R squared.
  • regm2

    • How adjusted R squared value decreases?

      • From the formula, in denominator as we subtract the independent variables with total sample size we get a lower value.
      • Also when we divide a numerator part in formula by lower denominator we get a higher value.
      • When this higher value multiplied with (1-R^2) we get a higher value because (1-R^2) is already a higher value.

      • Now when we substract a higher value from a 1 we get a lower value.

      • Adjusted R Squared value only decreases when we add a non correlated variable. But if we add a correlated variable then R^2 value will increase so that our adjusted R squared value also increases.

    • What is the difference between R Squared and Adjusted R Squared?

      • Every time you add a independent variable to a model, the R Squared increases, even if the independent variable is insignificant. It never declines. Whereas Adjusted R Squared increases only when independent variable is significant and affects dependent variable.

      • Adjusted R Squared value always be less than or equal to R Squared value.

  • Mean Squared Error

  • MSE is the square of the difference between the predicted value and the actual value.
  • MSE is much like MAE in sense that it provides a rough idea of the magnitude of error.
  • Higher the value, the more error and lower the value, the less error.
  • It is useful when we have a very unexpected dataset i.e very high or very low.
  • It can overestimate the model’s badness in case of noisy data.
  • In a nutshell noise are exxagerated and large error are punished.

  • regm3

  • Mean Absolute Error

  • MAE is the sum of absolute differences between the predicted values and the actual values.
  • It gives the idea of how wrong the prediction are.
  • MAE gives the estimation of magnitude of error but no idea of the direction.
  • For example: If MAE is 20K then we know that if the predicted price is 100K then the actual price is 80K to 120K.
  • MAE = 0 means no error or perfect Predictions.

  • regm4

  • Root Mean Squared Error

  • It is given as square root of MSE.
  • It tells us how much data are concentrated around a best fit line.
  • We perform square root of mean square error to bring the error in the same scale as the target variable.
  • RMSE has the ability to penalize the large error so that it is more appropriate at some cases.
  • Lower value are more desirable.
    • regm5

INTERVIEW QUESTION

  • When we use MAE,MSE and RMSE?

    • If there are many outliers then you may consider using Mean Absolute Error.
    • When there is very less outlier we can go with MSE or RMSE.
    • RMSE is used when large error is undesirable.

  • Difference between R Squared and Adjusted R Squared?

    • Every time you add a independent variable to a model, the R-squared increases, even if the independent variable is insignificant. It never declines. Whereas Adjusted R-squared increases only when independent variable is significant and affects dependent variable.
    • Adjusted r-squared can be negative when r-squared is close to zero.
    • Adjusted r-squared value always be less than or equal to r-squared value.
    • R^2 score tells how much data has been fitted by best fit line. However R^2 error dosen’t tell whether the features used to create model is really helping in prediction or not. So to penalize the non correlated features with respect to target we use adjusted r-squared.
    • R^2 and adjusted r-squared are used for specific linear problems.

  • Is a high R-squared good?

    • If the training set’s R-squared is higher and the R-squared of the validation set is much lower, it indicates overfitting. If the same high R-squared translates to the validation set as well, then we can say that the model is a good fit.

  • Is a low R-squared bad?

    • This completely depends on the type of the problem being solved. In some problems which are hard to model, even an R-squared of 0.5 may be considered a good one. There is no rule of thumb to confirm the R-squared to be good or bad. However, a very low R-squared indicates underfitting and adding additional relevant features or using a complex model might help.