Generating and visualizing regression residuals

Goals

This tutorial builds on the previous Linear Regression tutorial. It is recommended that you complete that tutorial prior to this tutorial. This tutorial demonstrates how to predict outcomes and generate residuals using the parameter estimates from a linear model. After this tutorial you should be able to

• Predict outcomes and find residuals using matrix operations
• Plot residuals on a scatter plot
• Add a zero line to your residual plot

Finding residuals

One convenient method for testing our model is to compare predicted outcomes to the observed outcomes. This is commonly done using the regression residuals.

Finding predicted outcomes

Predicted y values are found using the observed x values and the estimated parameters. In our previous tutorial we found that our estimated $ \alpha = 1.2795 $ and the estimated $ \beta = 5.7218 $. This means our predicted y values are given by

$$ \hat{y} = \hat{\alpha} + \hat{\beta}x $$ $$ \hat{y} = 1.2795 + 5.7218x $$

// Predicted Y
alpha_hat = 1.2795;
beta_hat = 5.7218;
y_hat = alpha_hat + beta_hat*x;

Computing residuals

The residuals from our regression are found by finding the difference between the predicted dependent variable and the estimated dependent variable

$$ e = y - \hat{y} $$

// Residual
e = y - y_hat;

Plotting residuals

A well performing model will have residuals that center around zero with random fluctuations. While there are many statistical methods for testing residuals, one quick and easy way to examine the behavior of residuals is to plot them.

/*
** Plot residuals
*/

// Declare plotControl structure and fill
// with default scatter settings
struct plotControl myPlot;
myPlot = plotGetDefaults("scatter");

// Add title to graph
plotSetTitle(&myPlot,"Residual Plot", "Arial", 16);
plotSetYLabel(&myPlot, "Residuals");

// Draw graph
plotScatter(myPlot, x, e);

After running the above code, you should get a graph that looks similar to the image below:

Adding zero line

It may help us see the size of the residuals more clearly if our graph has to a line at $ y = 0 $. This is easy to do in GAUSS using the plotAddXY command.

// Set up x values for line
x_zero = -4 | 4;

// Add zero line
// Construct vector of zeros
y_zero = zeros(2, 1);

// Fill myPlot with default settings for xy graphs
myPlot = plotGetDefaults("xy");

// Change line color to black
plotSetLineColor(&myPlot, "black");

// Make line 1 pixel thick
plotSetLineThickness(&myPlot, 1);

// Add line to plot
plotAddXY(myPlot, x_zero, y_zero);

After running the above code, your residual plot should now have a black zero line as we see below.

Conclusion

Congratulations! You have:

• Predicted outcomes and calculated residuals.
• Created a scatter plot of residuals.
• Added a zero line to your plot.

The next tutorial examines methods for testing error term normality.

For convenience, the full program text from this tutorial is reproduced below.

// Predicted Y
alpha_hat = 1.2795;
beta_hat = 5.7218;
y_hat = alpha_hat + beta_hat*x;

// Residual
e = y - y_hat;

/*
** Plot residuals
*/

// Declare plotControl structure and fill
// with default scatter settings
struct plotControl myPlot;
myPlot = plotGetDefaults("scatter");

// Add title to graph
plotSetTitle(&myPlot,"Residual Plot", "Arial", 16);
plotSetYLabel(&myPlot, "Residuals");

// Draw graph
plotScatter(myPlot, x, e);

/*
**Add zero line
*/

// Set up x_min and x_max on axis
x_zero = -4 | 4;

// Construct vector of zeros
y_zero = zeros(2, 1);

// Fill myPlot with default settings for xy graphs
myPlot = plotGetDefaults("xy");

// Change line color to black
plotSetLineColor(&myPlot, "black");

// Make line 1 pixel thick
plotSetLineThickness(&myPlot, 1);

// Add line to plot
plotAddXY(myPlot, x_zero, y_zero);