Multiple Regression Examples: Step-by-Step Calculations & Interpretation

What Is Multiple Regression?

The equation has two versions. The population model contains the true parameters β₀, β₁, …, β_k, which are never directly observed. The estimated regression equation uses the sample-based estimates b₀, b₁, …, b_k calculated from data:

OLS finds the values of b₀, b₁, …, b_k that minimize the sum of squared residuals: SSE = Σ(Y_i − Ŷ_i)². In matrix form, the solution is β̂ = (X^TX)⁻¹X^TY, provided the matrix X^TX is invertible.

The key difference from simple linear regression is that each coefficient in multiple regression is a partial coefficient. It describes the estimated association between one predictor and the outcome while the other predictors are accounted for in the model.

Multiple Regression Example at a Glance

Before the step-by-step details, here is what a finished multiple regression model looks like and how to read it.

Interactive Multiple Regression Calculator

Example 1: Predicting House Price (Full Walkthrough)

This example walks through every step from raw data to interpreted output. The dataset contains 10 houses with square footage and number of bedrooms as predictors, and sale price (in thousands of dollars) as the outcome.

Dataset

What Does "Holding Other Variables Constant" Mean?

This phrase appears in every regression textbook, and it is worth spending time on it because it is widely misunderstood.

In the house price model, b₁ = 0.1627 describes the estimated change in price for a one-unit increase in square footage, assuming bedrooms does not change. This is a mathematical property of the partial regression coefficient. When you compare two houses that differ only in square footage (same number of bedrooms), the model predicts a $0.1627K difference per square foot.

A concrete illustration: Suppose b₂ = 5.21 for bedrooms. That coefficient is not simply "the relationship between bedrooms and price." It is the estimated difference in price between houses with the same square footage but one more bedroom. If you ran a simple regression of price on bedrooms alone (ignoring square footage), the bedrooms coefficient would be different, possibly much larger, because it would absorb part of the square footage effect. This is the Simpson's paradox–adjacent phenomenon in regression: coefficients change when you add or remove predictors.

Use precise language: "an estimated association," "holding other included predictors constant," or "the expected change in predicted outcome." Avoid language that implies the coefficient establishes a causal relationship unless the study design supports that interpretation.

How to Interpret Partial Regression Coefficients

Each coefficient b_j in a multiple regression is called a partial regression coefficient. Three things to keep in mind:

Example 2: Salary Prediction with a Categorical Variable

This example introduces a categorical predictor through dummy coding. The outcome is annual salary, and the predictors are years of experience (continuous) and education level (categorical: High School, Bachelor's, Master's).

Dummy Coding Education Level

A categorical predictor with k categories requires k − 1 dummy variables to avoid perfect multicollinearity (the dummy variable trap). High School is the reference category.

The model is: Salarŷ = b₀ + b₁(Experience) + b₂(D_Bach) + b₃(D_Mast)

Running OLS on this data produces the fitted equation (approximate values for illustration; enter the data into the calculator above to verify):

Example 3: Marketing Sales Prediction

A business analyst wants to understand how advertising spend, website traffic, and number of sales calls relate to monthly sales revenue. This is a common applied regression problem in business analytics.

The model: Saleŝ = b₀ + b₁(Advertising) + b₂(Traffic) + b₃(SalesCalls)

Note that advertising, traffic, and sales calls are likely correlated with each other (months with higher ad budgets tend to have more traffic and more sales activity). Run the data through the calculator and inspect VIF values. High VIF does not necessarily reduce overall predictive performance, but it makes the individual coefficients less stable and harder to interpret in isolation.

Example 4: Healthcare Research (Blood Pressure)

This example mirrors the structure of observational health research. The outcome is systolic blood pressure (SBP), and the predictors are age, BMI, physical activity level (hours/week), and smoking status (binary: 1 = current smoker, 0 = non-smoker).

The fitted model (approximate, from OLS on this data):

R² in Multiple Regression

R² comes from the decomposition of total variation. For any OLS model with an intercept:

R² ranges from 0 to 1. An R² of 0.72 means the fitted model accounts for 72% of the observed variation in Y in the sample. The remaining 28% is captured by the residuals (SSE).

Adjusted R²

Adjusted R² penalizes model complexity. It can decrease when a predictor is added that does not sufficiently improve model fit. For the house price model: n = 10, k = 2, R² = 0.959. Adjusted R² = 1 − [(1 − 0.959)(9)/(7)] = 1 − [0.041 × 1.286] = 1 − 0.0527 = 0.947.

The Overall F-Test

The overall F-test evaluates whether the model with all k predictors improves on the intercept-only model.

H₀: β₁ = β₂ = … = β_k = 0 (all slopes are zero)

H_A: At least one β_j ≠ 0

A significant F-test means at least one slope is estimated to differ from zero under the assumed model. It does not mean every predictor is statistically significant individually, and it does not guarantee the model is well-specified or practically useful.

Complete ANOVA Table (House Price Example)

Verify: SST = SSR + SSE = 87,756 + 5,584 = 93,340. R² = 87,756 / 93,340 = 0.940. RMSE = √MSE = √797.7 ≈ 28.24 ($K).

Individual Coefficient Tests

Each coefficient has its own t-test:

H₀: β_j = 0. A small p-value gives evidence against H₀ for that predictor given all the others in the model.

A predictor can be non-significant in the t-test even when the overall F-test is significant. This happens when two or more predictors share explanatory power (multicollinearity), or when one predictor accounts for most of the model fit. Conversely, a single predictor can drive significance in the overall test without all predictors being individually significant.

The 95% confidence interval for each coefficient is: b_j ± t*_α/2 × SE(b_j), where t*_α/2 has df = n − k − 1 degrees of freedom. A confidence interval that does not contain zero corresponds to a two-tailed p-value below 0.05.

Multiple Regression with Multicollinearity

Multicollinearity refers to strong linear relationships among the predictors. It does not make regression impossible, but it does cause specific problems with individual coefficient estimates.

What Multicollinearity Does

Variance Inflation Factor (VIF)

For the house price example, SqFt and Bedrooms have a correlation of roughly 0.98 in this dataset. This gives VIF values around 14–15 for both predictors. A VIF this high means the standard error for each coefficient is about 3.7 times larger than it would be if the two predictors were uncorrelated. The overall model still fits well (R² = 0.959), but separating the contributions of SqFt and Bedrooms individually is unreliable with these 10 observations.

One approach when facing high multicollinearity: if the goal is prediction rather than individual coefficient interpretation, multicollinearity is less of a concern as long as the prediction stays within the range of the training data. If the goal is interpretation of individual predictors, consider collecting more data, removing one of the correlated predictors (if theory supports it), or combining them into a single composite predictor.

Assumptions of Multiple Linear Regression

The classical OLS assumptions govern when the estimators are efficient and when hypothesis tests are valid. These assumptions should be checked before drawing conclusions from a model.

Residual Diagnostics

The most useful visual checks are plots rather than tests alone, especially for smaller datasets:

Outliers and Influential Observations

Three related but distinct concepts:

Response outlier: An observation with a large residual (far from the regression line in the Y direction).

High leverage point: An observation with unusual predictor values (far from the center of the X space). Leverage does not depend on Y.

Influential observation: An observation that, when removed, substantially changes the estimated coefficients. Cook's distance combines leverage and residual size to measure overall influence.

Do not delete observations automatically because they are outliers or influential. Investigate first. An influential point may reveal a data entry error, a measurement problem, or a genuinely interesting observation that the current model cannot accommodate. See the discussion of influential points in regression.

How to Calculate a Predicted Value

To use a fitted model to make a prediction, substitute specific values for each predictor into the equation.

How to Calculate a Residual

From Example 1: House 5 has Y₅ = 390, X₁ = 1800, X₂ = 4.

Ŷ₅ = 43.98 + 0.1627(1800) + 5.21(4) = 43.98 + 292.86 + 20.84 = 357.68

e₅ = 390 − 357.68 = +32.32

A positive residual means the observed value exceeded the model's prediction. A negative residual means the model predicted higher than the actual value. Residuals scattered randomly around zero with no pattern in the residual plot support the linearity and constant variance assumptions.

Running Multiple Regression in Excel

Running Multiple Regression in SPSS

Navigate: Analyze → Regression → Linear

Move your outcome variable into the Dependent box. Move your predictor variables into the Independent(s) box. Click Statistics and check: Estimates, Confidence Intervals, Model fit, R squared change, and Collinearity diagnostics (for VIF). Click Plots and check: ZPRED vs ZRESID (residual vs fitted plot) and Normal probability plot. Click OK.

Read: Model Summary (R, R², Adjusted R², Std. Error of the Estimate), ANOVA table (F, Sig.), and Coefficients table (B, Std. Error, Beta, t, Sig., 95% CI, Tolerance, VIF).

Running Multiple Regression in R

# Fit the model
model <- lm(price ~ sqft + bedrooms, data = df)

# Full summary with coefficients, standard errors, t-tests, p-values
summary(model)

# 95% confidence intervals for all coefficients
confint(model)

# VIF — requires the car package
car::vif(model)

# Diagnostic plots (2x2 grid): residuals vs fitted, Q-Q, scale-location, Cook's
par(mfrow = c(2, 2))
plot(model)

The summary() output shows Coefficients, Std. Error, t value, and Pr(>|t|) for each term, plus Residual standard error (RMSE), Multiple R-squared, Adjusted R-squared, F-statistic, and p-value. The car::vif() function requires the car package to be installed and loaded separately.

Running Multiple Regression in Python

import statsmodels.formula.api as smf
import pandas as pd

# Fit the model using formula API
model = smf.ols("price ~ sqft + bedrooms", data=df).fit()

# Full summary table
print(model.summary())

# 95% confidence intervals
print(model.conf_int())

# VIF calculation
from statsmodels.stats.outliers_influence import variance_inflation_factor
from statsmodels.tools.tools import add_constant

X = add_constant(df[['sqft', 'bedrooms']])
vif_data = pd.DataFrame()
vif_data["Feature"] = X.columns
vif_data["VIF"] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]
print(vif_data)

The model.summary() output includes two tables. The first covers overall model fit (R², Adj. R², F-statistic, log-likelihood). The second shows the coefficient table (coef, std err, t, P>|t|, and 95% confidence intervals). Fitted values and residuals are available as model.fittedvalues and model.resid.

Running Multiple Regression in Stata

* Fit the model
regress price sqft bedrooms

* VIF after regression
estat vif

* Confidence intervals are included in regress output by default
* Residuals vs fitted plot
rvfplot, yline(0)

The regress output in Stata shows the ANOVA table at the top, followed by the coefficient table (Coef., Std. Err., t, P>|t|, and a 95% confidence interval for each coefficient). The estat vif post-estimation command reports VIF and tolerance for each predictor.

How to Read Multiple Regression Output

When you receive regression output from any software, read it in this order:

How to Report Multiple Regression Results

A standard reporting format for research papers, dissertations, and technical reports follows this structure:

Reporting conventions vary by field. APA style requires F(df1, df2) = value, p = value. Many journals request that p-values below .001 be reported as p < .001 rather than exact values. Always include coefficient standard errors or confidence intervals alongside point estimates.

Standardized vs. Unstandardized Coefficients

Standardized coefficients (β weights) allow a rough comparison of predictor "contributions" on a common scale, but they do not establish causal importance. A larger β does not mean that predictor has a stronger causal effect; it means that a one-SD change in that predictor is associated with a larger change in the outcome in SD units, within this model and data.

In R, standardize your data first with scale(), then run lm(). In SPSS, the Coefficients table includes both B (unstandardized) and Beta (standardized) columns. In statsmodels (Python), standardize predictors manually before fitting.

Extensions and Related Topics

Interaction Effects

An interaction term allows the association between one predictor and the outcome to vary depending on the value of another predictor. The model becomes:

Y = b₀ + b₁X₁ + b₂X₂ + b₃(X₁ × X₂) + ε

The coefficient b₃ represents how the slope of X₁ changes for each unit increase in X₂. For example, the estimated association between advertising spend and sales might differ across high-traffic and low-traffic periods. In R: lm(sales ~ advertising * traffic, data = df). Interactions should generally be motivated by theory or prior evidence rather than added purely to increase R².

Polynomial Regression

To model curvature in the relationship between a predictor and the outcome, add a squared or higher-order term:

Y = b₀ + b₁X + b₂X² + ε

This is still a linear model because the coefficients enter linearly. In R: lm(y ~ x + I(x^2), data = df). Inspect a residual vs. fitted plot to decide whether a polynomial term is warranted.

Simple vs. Multiple Regression

Learn more about simple regression in the simple linear regression guide and see worked examples at simple regression examples.

Multiple Regression vs. Logistic Regression

Ordinary multiple linear regression predicts a continuous outcome. When the outcome is binary (e.g., pass/fail, survived/died), logistic regression models the log-odds of the outcome rather than the outcome itself. Using linear regression on a binary outcome can produce predicted values outside 0–1 and heteroscedastic residuals, making logistic regression the appropriate choice in those situations.

Stepwise Regression

Automated variable selection methods (forward, backward, stepwise) search for a "best" subset of predictors based on statistical criteria. They have well-documented problems: inflated Type I error rates from repeated testing, biased coefficient estimates, overfitting, and results that depend on the order of entry. Many statisticians recommend theory-driven model selection, cross-validation for predictive purposes, or regularization methods (ridge, lasso) rather than standard stepwise selection.

How Many Observations Do You Need?

There is no universal rule. Common heuristics (like "10 observations per predictor") are rough starting points that do not account for effect size, expected R², collinearity, or desired power. A proper sample size calculation uses power analysis based on the expected effect size, significance level, desired power (often 0.80), and number of predictors. Statistical software (pwr package in R, G*Power) can perform these calculations. Larger samples allow detection of smaller effects and produce more stable coefficient estimates.

Common Mistakes in Multiple Regression

Practice Problems

Frequently Asked Questions

Key Takeaways

Related guides on this site: multiple linear regression, simple linear regression, R-squared, residuals, Pearson correlation, statistical assumptions, F-distribution, p-values, and logistic regression.