What Is OLS Regression?
OLS full form: Ordinary Least Squares. The name describes exactly what the method does: it finds the ordinary (unweighted) least-squares solution to the coefficient estimation problem.
There is one distinction worth stating immediately, because it is glossed over in most introductions. Linear regression is the statistical model — a specification of how a response Y relates to predictors through a linear-in-parameters equation. OLS is one estimation method used to fit that model. The two are often treated as synonyms in practice, but they are not the same thing. OLS is the method; the linear regression model is the structure the method fits.
- OLS full form: Ordinary Least Squares
- Purpose: Estimate coefficients of a linear regression model
- Objective: Minimize SSE = Σ(yᵢ − ŷᵢ)²
- Simple slope formula: b₁ = Σ(xᵢ − x̄)(yᵢ − ȳ) / Σ(xᵢ − x̄)²
- Simple intercept formula: b₀ = ȳ − b₁x̄
- Matrix form: β̂ = (XᵀX)⁻¹Xᵀy (requires XᵀX invertible)
- Key condition for unbiasedness: E(ε | X) = 0 (zero conditional mean)
- Does NOT require: X or Y to be normally distributed
- Does NOT minimize: Perpendicular distance to the line
OLS vs Linear Regression
The distinction matters in practice. When someone says "I ran a linear regression," they usually mean they used OLS to estimate the coefficients of a linear model. But other estimation methods — weighted least squares, generalized least squares, ridge regression — can also be applied to a linear model. OLS is the default choice, not the only one.
| Concept | Linear Regression | OLS |
|---|---|---|
| What it is | A statistical model | An estimation method |
| Specifies | Relationship between Y and predictors | How to choose coefficient values |
| Contains | Unknown population parameters β | Sample estimates b (or β̂) |
| Notation | Y = β₀ + β₁X + ε | ŷ = b₀ + b₁x |
| Can be estimated by | OLS, WLS, GLS, and others | Applies to linear models (simple and multiple) |
The population model for simple linear regression is:
Yᵢ = observed response
Xᵢ = predictor value
β₀ = population intercept
β₁ = population slope
εᵢ = unobserved error
OLS uses sample data to estimate β₀ and β₁, producing sample estimates b₀ and b₁ (sometimes written β̂₀ and β̂₁). The fitted values are then:
ŷᵢ = fitted (predicted) value
b₀, b₁ = OLS coefficient estimates
One more precision: OLS is not limited to simple regression with one predictor. The same principle extends to multiple linear regression with many predictors. That is covered in the matrix form section below.
How Ordinary Least Squares Works
The logic of OLS follows a clear sequence. Walk through it once and the formula derivation becomes obvious rather than arbitrary.
Observed values and fitted values
For each data point, you observe the actual response yᵢ. Once the coefficients b₀ and b₁ are chosen, you can compute a fitted value ŷᵢ = b₀ + b₁xᵢ — what the model predicts for that observation.
Residuals
The residual eᵢ = yᵢ − ŷᵢ measures the vertical gap between the observed value and the fitted value. Positive residuals sit above the line; negative residuals sit below. These are fitted residuals — distinct from the population error εᵢ, which is unobserved.
Square each residual
Each residual is squared: eᵢ². Squaring serves three purposes. It prevents positive and negative residuals from canceling. It penalizes large errors more than small ones (a residual of 10 contributes 100 to the total, while a residual of 2 contributes only 4). And it creates a smooth, differentiable objective function.
Sum the squared residuals (SSE)
Add up all the squared residuals: SSE = Σeᵢ² = Σ(yᵢ − ŷᵢ)². This single number — the residual sum of squares, or sum of squared errors — captures the total fit of the line to the data.
Choose the coefficients that minimize SSE
OLS finds the specific values of b₀ and b₁ that make SSE as small as possible. No other intercept-slope pair produces a smaller sum of squared residuals for the same data. That is what makes the OLS solution unique (when it exists).
Standard OLS minimizes squared vertical distances between observed and fitted values — differences in the response direction. It does not minimize perpendicular distance between points and the line. Methods like orthogonal (Deming) regression minimize perpendicular distance and are appropriate in different settings.
Why Are Residuals Squared? A Note on the Raw Sum
In any OLS fit that includes an intercept, the raw residuals always sum to exactly zero: Σeᵢ = 0. That is an algebraic property of the intercept estimation, not something you need to check. The consequence is that the simple sum of residuals cannot distinguish a tight fit from a loose one — it is always zero regardless of how well the line fits. Squaring the residuals before summing removes this problem, while also ensuring that the optimization has a unique, identifiable solution.
OLS Formula for Simple Linear Regression
Slope Estimator
Taking the partial derivatives of SSE with respect to b₀ and b₁ and setting them to zero yields two simultaneous equations — the normal equations. Solving them gives the OLS slope:
Sxy = Σ(xᵢ − x̄)(yᵢ − ȳ) — cross-product sum
Sxx = Σ(xᵢ − x̄)² — sum of squared x deviations
x̄, ȳ = sample means of X and Y
An equivalent form using sample covariance and variance is b₁ = Cov(X,Y) / Var(X), or in terms of correlation: b₁ = r(sᵧ/sₓ), where r is the sample Pearson correlation and sᵧ, sₓ are the sample standard deviations. These are all the same formula in different notation.
If all x values are identical, then Sxx = 0 and the slope cannot be estimated. Variation in X is required for OLS to produce a slope estimate.
Intercept Estimator
ȳ = sample mean of Y
x̄ = sample mean of X
b₁ = estimated slope
A useful geometric property follows from this formula: the OLS fitted line with an intercept always passes through the point (x̄, ȳ) — the centroid of the data. This is not a coincidence; it is a direct consequence of how the intercept is derived from the slope.
Worked OLS Regression Example
One complete hand calculation makes the minimization concrete. The dataset below is small enough to follow step by step, and the arithmetic has been verified.
Dataset: Study hours (X) and test scores (Y)
| Obs | x (hours) | y (score) | xᵢ − x̄ | yᵢ − ȳ | (xᵢ−x̄)(yᵢ−ȳ) | (xᵢ−x̄)² |
|---|---|---|---|---|---|---|
| 1 | 1 | 2 | −2 | −2 | 4 | 4 |
| 2 | 2 | 3 | −1 | −1 | 1 | 1 |
| 3 | 3 | 5 | 0 | 1 | 0 | 0 |
| 4 | 4 | 4 | 1 | 0 | 0 | 1 |
| 5 | 5 | 6 | 2 | 2 | 4 | 4 |
| Sum | 15 | 20 | 0 | 0 | 9 | 10 |
Compute the means: x̄ = 15/5 = 3.0 | ȳ = 20/5 = 4.0
Compute the deviations and cross-products: Sxy = Σ(xᵢ − x̄)(yᵢ − ȳ) = 4 + 1 + 0 + 0 + 4 = 9 | Sxx = Σ(xᵢ − x̄)² = 4 + 1 + 0 + 1 + 4 = 10
Estimate the slope: b₁ = Sxy / Sxx = 9 / 10 = 0.9
Estimate the intercept: b₀ = ȳ − b₁x̄ = 4.0 − 0.9 × 3.0 = 4.0 − 2.7 = 1.3
Fitted equation: ŷ = 1.3 + 0.9x Check: at x = 3, ŷ = 1.3 + 0.9(3) = 4.0 = ȳ ✓ (line passes through the centroid)
Calculate fitted values, residuals, and verify SSE is minimized:
| Obs | x | y | ŷ = 1.3 + 0.9x | e = y − ŷ | e² |
|---|---|---|---|---|---|
| 1 | 1 | 2 | 2.2 | −0.2 | 0.04 |
| 2 | 2 | 3 | 3.1 | −0.1 | 0.01 |
| 3 | 3 | 5 | 4.0 | 1.0 | 1.00 |
| 4 | 4 | 4 | 4.9 | −0.9 | 0.81 |
| 5 | 5 | 6 | 5.8 | 0.2 | 0.04 |
| Sum | — | — | — | 0.0 ✓ | SSE = 1.90 |
✅ OLS result: ŷ = 1.3 + 0.9x, with SSE = 1.90. The residuals sum to zero (as always with an intercept). No other b₀ and b₁ combination produces a smaller SSE for this dataset. Try the Simple Linear Regression Calculator to verify.
Why OLS Has the Smallest SSE: Three Candidate Lines
To make the minimization concrete, compare the OLS line against two plausible alternatives on the same data. The OLS line wins by construction:
| Line | Equation | SSE | Note |
|---|---|---|---|
| Poor fit | ŷ = 1 + 0.5x | ≈ 5.0 | Slope too shallow |
| Close but not OLS | ŷ = 1.5 + 0.8x | ≈ 2.1 | Better, still not minimal |
| OLS | ŷ = 1.3 + 0.9x | 1.90 | Minimum possible SSE |
This table shows the pattern, but comparing three candidates does not mathematically prove the OLS line is globally optimal. The analytic derivation via the normal equations provides that guarantee.
OLS Regression Calculator
Enter paired x,y values separated by commas (one pair per line, or paste from a spreadsheet). The calculator computes b₀, b₁, SSE, and R² using the formulas above.
OLS Simple Linear Regression Calculator
For a full regression analysis with diagnostics, see the Simple Linear Regression Calculator and the dedicated guide at simple linear regression.
OLS in Matrix Form
When the regression model has more than one predictor — or when working with multiple observations at once — matrix notation becomes far more compact. The general linear model is:
y = n×1 vector of responses
X = n×p matrix of predictors (with a column of 1s for the intercept)
β = p×1 vector of unknown coefficients
ε = n×1 vector of errors
The OLS estimator that minimizes the scalar SSE = (y − Xβ)ᵀ(y − Xβ) is:
Xᵀ = transpose of X
(XᵀX)⁻¹ = inverse (requires XᵀX to be invertible)
1. Full rank required: The formula (XᵀX)⁻¹Xᵀy requires XᵀX to be invertible (full column rank). If any predictor column is an exact linear combination of others — perfect multicollinearity — XᵀX is singular and the ordinary inverse does not exist, making the coefficient vector non-uniquely identified.
2. Production software doesn't literally invert XᵀX: Statistical packages use numerically stable algorithms such as QR decomposition or SVD to solve the least-squares problem. The matrix formula is the mathematical derivation; it is not the recommended computational approach for real data.
High but imperfect multicollinearity — predictors that are correlated but not exact linear combinations — does not prevent unique estimation, but it can inflate coefficient standard errors considerably and make individual estimates unstable. This is different from the mathematical failure caused by perfect multicollinearity.
What Does OLS Actually Assume?
The most common treatment of OLS assumptions presents a flat list of five or six items as though they are all equally necessary for the same purpose. That reading misses something important. Different conditions support different statistical properties — and confusing them leads to misapplied corrections and wrong conclusions.
Ask not "what does OLS assume?" but "what do I need for this specific property?" Unbiasedness, efficiency, classical standard errors, and exact finite-sample inference each have their own conditions.
Condition 1: Correct Model Specification
The conditional mean of Y given X should be correctly captured by the chosen linear-in-parameters form. A curved relationship left unmodeled produces systematic residual structure and potentially biased interpretations of coefficients. Importantly, "linear" here means linear in the unknown coefficients — not that every predictor must appear in raw, untransformed form. OLS can estimate models with X², log(X), interaction terms, or indicator variables, provided they remain linear in β.
Condition 2: Zero Conditional Mean / Exogeneity
The most consequential condition for coefficient validity is: E(ε | X) = 0. After conditioning on the predictors, the remaining error should have zero expected value. When this holds, OLS coefficient estimates are unbiased for the parameters of the specified model under classical assumptions. When it fails — typically because an omitted variable is related to both Y and X — OLS estimates can be biased for the intended relationship.
Note that omitting a predictor only causes bias when the omitted variable both affects Y and is correlated with an included predictor. Omitting a variable that is independent of the included predictors doesn't bias those coefficients.
A coefficient estimated by OLS reflects a conditional association, not necessarily a causal effect. Causal interpretation requires appropriate study design, confounding control, and identification assumptions beyond OLS itself. A statistically significant slope does not tell you that X caused Y to change.
Condition 3: No Perfect Multicollinearity
For the unique OLS solution to exist, no predictor column in X can be an exact linear combination of other columns. In simple regression, this reduces to the requirement that X must have some variation — all identical values make the slope undefined. In multiple regression, this means no perfect collinearity between predictors.
Condition 4: Appropriate Sampling / Error Independence
Standard inference assumes that observations come from a sampling process where errors are not systematically related across observations. Dependence arises naturally in repeated measurements on the same subject, clustered data (e.g., students within schools), time series, or geographic groupings. Treating dependent observations as independent can produce invalid standard errors and misleading tests — not biased coefficients, but unreliable inference.
Condition 5: Homoskedasticity
Under the classical linear model, error variance is constant: Var(εᵢ | X) = σ². In a residual plot, this corresponds approximately to similar vertical spread of residuals across the fitted value range. Homoskedasticity is not needed to compute OLS coefficients or for them to be unbiased. Its role is in efficiency and in the validity of standard formulas for standard errors.
Condition 6: Normal Errors — and When This Actually Matters
This condition is frequently misrepresented. Normal errors are not required to calculate OLS coefficients. They are not required for OLS estimates to be unbiased. They are not a condition of the Gauss–Markov theorem. What normality provides is exact finite-sample distributional justification for classical t-tests, F-tests, and confidence intervals under the classical linear model. In large samples, the Central Limit Theorem provides approximate justification without requiring normally distributed errors.
Model specification + variation in X
The linear form must be appropriate and X cannot be constant. No distributional assumptions on X or Y are required.
E(ε | X) = 0 (exogeneity)
If predictors are systematically related to unmodeled determinants of Y, estimates may be biased for the intended relationship.
Homoskedasticity + uncorrelated errors
Under these conditions (plus exogeneity and full rank), OLS is BLUE — the best linear unbiased estimator.
Normal errors (εᵢ ∼ N(0, σ²))
With normally distributed errors, the exact t and F distributions apply to standard regression tests in small samples.
The Gauss–Markov Theorem and BLUE
The Gauss–Markov theorem gives OLS its theoretical justification under the classical assumptions. The result:
Each word in BLUE carries a specific meaning:
| Letter | Meaning | What it does NOT mean |
|---|---|---|
| Best | Minimum variance among all linear unbiased estimators | Best prediction method overall; best among all estimators including biased ones |
| Linear | Estimator is linear in the observed response vector y | The relationship must visually appear as a straight line with raw X |
| Unbiased | Expected value of the estimator equals the true parameter β | Every individual estimate is exactly correct |
| Estimator | A rule applied to sample data to produce a coefficient estimate | — |
Normal errors are not a condition of the Gauss–Markov theorem. The theorem holds under homoskedastic uncorrelated errors without any normality requirement. Adding normality is a separate step that enables exact finite-sample inference.
Heteroskedasticity and Robust Standard Errors
When error variance is not constant across observations — when the spread of residuals varies systematically with fitted values or predictor values — the errors are heteroskedastic. This is worth getting right because a common statement in textbooks is misleading: heteroskedasticity does not automatically bias OLS coefficient estimates.
Under exogeneity, OLS coefficients can remain consistent even with heteroskedastic errors. What changes is efficiency and inference. The conventional standard errors derived under homoskedasticity will be wrong — potentially much too small or too large — leading to misleading t-statistics and p-values. The OLS estimates themselves may still be the right central values; it's the uncertainty around them that gets distorted.
Heteroskedasticity-Consistent (Robust) Standard Errors
When heteroskedasticity is present, you can keep the OLS coefficient estimates and replace the standard error formula with a heteroskedasticity-consistent (HC) version. Common variants are HC0 through HC3, with HC3 generally performing well in small samples. Most statistical software implements these under labels like "robust," "sandwich," or "White" standard errors.
| Method | OLS | WLS |
|---|---|---|
| Objective | Minimize Σeᵢ² (equal weight) | Minimize Σwᵢeᵢ² (unequal weights) |
| Coefficient weight | Each observation weighted equally | Observations receiving higher weights for lower variance |
| Best when | Variance is roughly constant | Variance structure is known or can be modeled |
| Inference | Standard or robust SEs | Weighted SEs |
If the correlation structure of errors can be modeled — as in time series or clustered data — generalized least squares (GLS) provides an alternative that accounts for that structure directly. Cluster-robust or HAC (Newey-West) standard errors offer inference corrections without changing the coefficient estimates.
OLS and R-Squared
R² — the coefficient of determination — measures what proportion of total response variation the model accounts for. With an intercept, it is:
SSE = Σ(yᵢ − ŷᵢ)² — residual sum of squares
SST = Σ(yᵢ − ȳ)² — total sum of squares
Because OLS minimizes SSE for a fixed dataset and model, it simultaneously maximizes R² among all possible coefficient choices for that model. That is a mathematical consequence, not an additional goal OLS pursues.
A few things R² cannot tell you: that the model is correctly specified, that assumptions hold, that the relationship is causal, or that predictions will be reliable out of sample. A high R² can coexist with serious misspecification. A low R² can coexist with a meaningful, well-estimated coefficient in the right context. Neither threshold guarantees anything about model validity. For more detail, see the guide on R-squared and coefficient of determination.
Residual Diagnostics
After fitting an OLS model, residual analysis checks whether the patterns in the data are consistent with the assumptions that make coefficient estimates and inference reliable. The goal is not to achieve a perfect-looking plot — all real datasets have some residual irregularity — but to detect gross violations that matter for interpretation.
What to Plot and What to Look For
| Diagnostic | What it examines | Concern if you see… |
|---|---|---|
| Residuals vs fitted values | Functional form and equal variance | Curved pattern (misspecification) or funnel shape (heteroskedasticity) |
| Normal Q-Q plot | Residual distribution shape | Strong S-curve or heavy tails (matters for exact small-sample inference) |
| Scale-location plot | Variance stability | Trend in spread with fitted values |
| Residuals vs leverage | Influential observations | Points with high leverage and large residuals |
| Residuals vs time/order | Serial dependence | Runs of same-sign residuals |
The Residual Plot Generator lets you visualize these patterns interactively.
Outliers and Influential Points
Because OLS squares residuals, a single large deviation contributes disproportionately to SSE. An observation with residual 10 contributes 100 to the total — 25 times more than a residual of 2. This means OLS is potentially sensitive to unusual observations.
But sensitivity to a point is not the same as "remove it." Leverage (how far an observation's predictor values are from the center of X) and Cook's distance (which combines leverage and residual size) are more informative measures. An observation can have high leverage but a small residual, or a large residual but low leverage — the combination is what determines influence on the fitted coefficients. See the guide on influential points for a full treatment. The standard advice is to investigate, verify, and document any unusual observation — not to delete it because it weakens model fit.
OLS vs Other Estimation Methods
| Method | Core idea | When it applies |
|---|---|---|
| OLS | Minimize unweighted Σeᵢ² | Standard linear model, constant variance |
| WLS | Minimize Σwᵢeᵢ² with known/modeled weights | Variance structure is understood and varies by observation |
| GLS | Model the full error covariance structure | Correlated or heteroskedastic errors with modeled structure |
| Robust regression | Reduce sensitivity to extreme observations | Concern about influential points distorting the fit |
| Ridge | Add L2 penalty: SSE + λΣβⱼ² | Unstable coefficient estimates, prediction focus |
| Lasso | Add L1 penalty: can set coefficients to zero | Variable selection, sparse models |
| Logistic regression | Model log-odds of binary outcome | Binary response variable |
OLS and Binary Outcomes
For a binary (0/1) response, logistic regression is generally more appropriate because it constrains predicted values to the (0,1) range and directly models probability through a log-odds link. OLS applied to a binary response is called a linear probability model in econometrics. It has interpretive advantages but can predict outside [0,1] and exhibits heteroskedasticity by construction. Neither approach is universally "correct" — choice depends on context and what question the analysis is meant to answer.
OLS vs Correlation
Pearson correlation and OLS slope both measure linear association, but they are not the same thing and are not interchangeable. Correlation is symmetric: r(X,Y) = r(Y,X). Regression is directional: regressing Y on X produces a different fitted line than regressing X on Y, because each minimizes vertical residuals in a different direction. The slope and correlation are related — b₁ = r(sᵧ/sₓ) — but they answer different questions. Correlation quantifies association strength on a dimensionless scale; the OLS slope gives the fitted change in Y units per unit of X.
Common OLS Regression Mistakes
| Mistake | What Is Wrong | Correction |
|---|---|---|
| "OLS and linear regression are the same thing" | ✗ Conflates a model with an estimation method | ✓ Linear regression is the model; OLS is one method to fit it |
| "Y must be normally distributed for OLS" | ✗ Confuses the model with a marginal distributional requirement | ✓ OLS coefficients exist without any distributional assumption on Y |
| "X must be normally distributed for OLS" | ✗ No classical regression assumption requires normal predictors | ✓ Predictors can follow any distribution; no normality assumption applies |
| "Normality makes OLS unbiased" | ✗ Mixes up what drives unbiasedness | ✓ Unbiasedness comes from E(ε|X) = 0, not from normality |
| "Heteroskedasticity biases the OLS coefficients" | ✗ A very common textbook error | ✓ Under exogeneity, OLS coefficients remain consistent; it's standard errors and efficiency that are affected |
| "Multicollinearity biases OLS estimates" | ✗ Confuses imprecision with bias | ✓ High (imperfect) collinearity inflates standard errors; only perfect collinearity prevents unique estimation |
| "High R² means the model is valid" | ✗ R² measures fit, not assumption validity or causality | ✓ A high R² can coexist with misspecification, confounding, or extrapolation problems |
| "OLS minimizes perpendicular distance to the line" | ✗ Describes orthogonal regression, not standard OLS | ✓ OLS minimizes squared vertical (Y-direction) distances |
| "Linear models can't include X² or log(X)" | ✗ Confuses linear-in-parameters with linear-in-X | ✓ OLS can estimate models with polynomial or transformed terms, provided coefficients enter linearly |
| "Outliers should be deleted to improve fit" | ✗ Cherry-picks the sample | ✓ Investigate unusual observations, verify data quality, analyze influence — document rather than delete |
| "OLS proves X causes Y" | ✗ Association ≠ causation | ✓ Causal interpretation requires identification assumptions beyond regression |
Frequently Asked Questions
OLS regression is the method used to draw the "best-fit" line through data. "Best" is defined precisely: it finds the slope and intercept that minimize the sum of squared vertical gaps between actual data points and the line. The word "ordinary" distinguishes it from weighted or generalized variants, and "least squares" refers to the squared-residual objective.
Linear regression is the statistical model that specifies Y = β₀ + β₁X + ε. OLS is one method to estimate the unknown coefficients β₀ and β₁ from data by choosing values that minimize SSE. Other methods (weighted least squares, generalized least squares, ridge) can also estimate a linear regression model. In practice many people use the terms interchangeably, but technically they refer to different things.
Three reasons. First, positive and negative residuals would cancel if simply summed — any line through the centroid would have Σeᵢ = 0. Squaring prevents cancellation. Second, squaring penalizes large errors more than small ones, so the OLS line is pulled toward minimizing big mistakes rather than small ones. Third, squaring produces a smooth differentiable objective function with a unique analytic solution via the normal equations.
No on both counts. OLS coefficients can be calculated from any data — neither X nor Y needs to follow a normal distribution. The classical normality assumption concerns the conditional error distribution ε | X, and even that is not required to compute or interpret OLS estimates. Normality of errors matters for exact small-sample distributional results (t-tests, F-tests). In large samples, approximate inference is available without it.
The Gauss–Markov theorem states that under the classical conditions (correct specification, E(ε|X) = 0, homoskedastic uncorrelated errors, no perfect multicollinearity), OLS is BLUE: the Best Linear Unbiased Estimator. "Best" means minimum variance among all linear unbiased estimators. Normality is not one of the Gauss–Markov conditions.
Not by itself. If E(ε|X) = 0 holds (exogeneity), OLS coefficient estimates can remain unbiased and consistent even with heteroskedastic errors. What changes is efficiency — OLS is no longer BLUE — and the validity of conventional standard errors. Heteroskedasticity-consistent (robust) standard errors correct the inference problem without changing the coefficients themselves.
Consider alternatives when: the response is binary or categorical (logistic regression); the error covariance structure has a known pattern like clustering or serial correlation (robust SEs, GLS, mixed models); important measurement error in X leads to attenuation bias (errors-in-variables methods); known variance structure suggests differential weighting (WLS); or the goal is prediction under collinearity rather than unbiased estimation (ridge, lasso).
Yes. The requirement for OLS is that the model is linear in the unknown coefficients, not that the predictor appears in raw untransformed form. A model like Y = β₀ + β₁X + β₂X² is linear in β₀, β₁, β₂, even though X appears squared. Similarly, log(X), interaction terms, and indicator variables can all enter an OLS model without violating the linearity requirement.
Key Takeaways
📌 Summary — OLS Regression
- OLS = Ordinary Least Squares — an estimation method, not the model itself
- Objective: Minimize SSE = Σ(yᵢ − ŷᵢ)² by choosing b₀ and b₁
- Simple slope: b₁ = Sxy/Sxx | Intercept: b₀ = ȳ − b₁x̄
- Matrix form: β̂ = (XᵀX)⁻¹Xᵀy — requires XᵀX invertible, not what software actually computes
- Unbiasedness requires E(ε|X) = 0, not normality
- Gauss–Markov BLUE requires homoskedastic uncorrelated errors — normality is not part of this theorem
- Heteroskedasticity affects efficiency and standard errors, not necessarily the coefficients
- OLS minimizes vertical Y residuals, not perpendicular distance to the line
- Neither X nor Y needs to follow a normal distribution
- Regression coefficient ≠ causal effect — that requires identification beyond OLS