Linear Regression

Illustrates the concept of linear regression. Scroll down below the applet for more background and details.

Details

Use the left arrow and right arrow to change the slope of the regresion line.

Use up arrow and down arrow to change the intercept of the regression line. (You may need to hold down the Shift key while using the up/down arrow keys to avoid scrolling.)

Use the r key to return to the maximum likelihood slope and intercept.

The likelihood is the product of 5 normal densities, one for each observed data point. These normal densities are shown in a brick red color centered at the predicted y value. The regression line determines the predicted y value for each data point given the x value for that point, and the difference between the observed y value and the predicted y value is the residual, shown as a dashed line.

The best-fitting regression line has slope and intercept such that the sum of squared residuals (SS) is minimized (the “least-squares” solution). This is equivalent to maximizing the log likelihood, which is proportional to the negative of SS. Note that the estimated regression slope and intercept will be the same regardless of the variance of the individual normal densities (but the numerical value of the log-likelihood will differ).

The regression line becomes more red as SS becomes larger (and the log likelihood becomes smaller) and is a neutral gray color if the slope and intercept of the regression line are at their maximum likelihood (or least squares) values.

Acknowledgements

This applet makes use of the excellent d3js javascript library. Please see the GitHub site for details about licensing of other libraries that may have been used in the source code for this applet.

Licence

Creative Commons Attribution 4.0 International. License (CC BY 4.0). To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.