Abzooba Admin 
– Nihal Karne, Riddhi Kontamwar
Regression Model Evaluation – RMSE, MAPE, MAE
Model evaluation is an integral part of the model development process. It helps us to choose the best model that fits and generalise on our data.
In this article, we will discuss evaluation of supervised learning models where output is continuous, such as regression models. In these models, prediction error is used to define the model performance. Prediction error is defined as the difference between the actual value and the predicted value. It is also referred as residuals.
Regression is a problem where we try to predict a continuous dependent variable using a set of independent variables. Let’s try to understand with few of the metric that are used for getting an estimate of prediction error.
Important point:
Evaluating the model performance is more subjective than objective. Consider a salary prediction regression problem as an example.
- Suppose we are using MAE as the metric to get an estimate of our prediction error, so typically we would want our model to have a “smaller” MAE. Let’s say for a model the MAE is 426. This can be interpreted as our predictions will be off by 426 (on average) from the actual values.
- But is this value actually “small”? Is the model performing great?
a) A model with MAE of 426 would be doing great if population’s salary ranges from 100 to 1,000,000 (currency units). But if salary ranges from 1,000 to 2,500 (currency units), 426 MAE can indicate that model is making a huge error in prediction.
b) So, it’s necessary to analyze the evaluation metrics wrt the problem we are solving. For e.g. if we use MAPE here, it would have given us deviation in prediction wrt actuals.
_______________________________________________________________________________________________________________________________________________________________