Simple Linear Regression

1.2 & 1.3 Assessing Conditions for SLR

Agenda

• Quantitative Response Variable
• One Quantitative Predictor (i.e., Explanatory Variable)
• choose-fit-ASSESS-use

Announcements

• no office hour today (Wed 1/24)

Fitting a Simple Linear Regression Model

• Our SLR model is simply the mean for every possible value of \(X\)
• If you focus on one “slice” of the data at a time, you’re looking at the distribution of \(Y\) for a given value of \(X\)

\[Y = \beta_0 + \beta_{1}X + \epsilon\]

Where \(\epsilon \sim N(0,\sigma)\) : the errors are assumed to follow a Normal distribution with with mean 0 and standard deviation \(\sigma\)

Q’s about our Simple Linear Regression Model

• What’s the overall pattern in the relationship between \(X\) & \(Y\)?
• Within any single “slice” of \(X\), what does the distribution of residuals look like?
• Across all “slices” of \(X\), what is similar & different about the distribution of \(Y\)?
• What do the individual “dots” represent here? In what way(s) are the “dots” related to one another or not?

1.2 Conditions for a (Simple) Linear Regression Model: L-I-N-E-R

• Linear relationship between X & Y
• Independent errors (i.e., cases/observations)
• Normally distributed \(Y\) for any given \(X\) (i.e., within any slice of \(X\))
• Equal variance of \(Y\) for any given \(X\) (i.e., across all slices of \(X\))
• Random sample of data

Compact notation for SLR conditions, “iid Normal” residuals

• You might hear people say the errors are i.i.d. Normal, or see this compact notation:

\[\varepsilon \overset{iid}{\sim} N(0,\sigma)\]

• To expand slightly, this means that the errors are “independent and identically distributed” according to the Normal distribution shown.

• Q: Can you unpack this to map each part of our L-I-N-E-R conditions?

Standard Error of Regression

\[\varepsilon \overset{iid}{\sim} N(0,\sigma)\]

• The standard deviation of the residuals, \(\sigma\), above is called the “Standard Error of Regression”
• We estimate the Standard Error of Regression from the data

\[\hat{\sigma}_{\epsilon} = \sqrt{\frac{\sum{(y_i - \hat{y})^2}}{n - 2}} = \sqrt{\frac{SSE}{n - 2}}\]

• We have \(n - 2\) degrees of freedom for estimating the standard error of regression here
• We lose a “degree of freedom” for each \(\beta\) we have to estimate (i.e., \(\beta_0\) & \(\beta_1\))

Randomness

• Random is NOT haphazard

source: https://m.xkcd.com/1210/

Randomness (Two Types)

• Random selection of cases (“units” in the figure) from the target population
• Random assignment of explanatory variable. \(X\)

Source: Ramsey, F. L. & Schafer, D. W. (2002). The Statistical Sleuth. Pacific Grove, CA: Duxbury.

1.3 Assessing Conditions for SLR

• Residuals can help us assess most of the L-I-N-E-R conditions
  • Linearity
  • Normality
  • Equal variance
• Must understand how the data were collected to assess others
  • Independence
  • Randomness
• Q: …But how does a Statistician find out how the data were collected?

Residual Plots: Assessing Conditions for SLR

• Residuals can help us assess most of the L-I-N-E-R conditions
  • Linearity: residuals vs fitted values
  • Normality: residuals resemble a Normal distribution
  • Equal variance: residuals vs fitted values

Plot Residuals vs Fitted Values

• ASSESS: Linearity & Equal Variance conditions

No evidence of “pattern” left in the residuals (L & E conditions look OK)

Common Problems revealed by Residuals vs Fitted Values

Evidence of nonlinearity in the residuals (“L” violation)

Evidence of non-constant variance in the residuals (“E” violation)

Normal Plots

• ASSESS: Normality condition
• We’re making a comparison between Quantiles observed in our data & Quantiles we would theoretically expect from a true Normal distribution
• So-called “Quantile-Quantile” plot (or QQ Plot)

Top: ideal situation (“N” condition looks great); Bottom shows right skew (problem with “N” condition)

Normal Quantile Plot can be difficult to judge

Example: Length of Baseball Games

• In 2023, Major League Baseball (MLB) introduces 30 second pitch clock
• (calls to implement something similar in Cricket came soon after)

RQ: In 2020, what factors could help predict how long a Major League Baseball game lasted?

• Files on Canvas: Starter code and data set

1. read the data into R
2. explore relationships between time & potential predictors
3. CHOOSE a model (hint: it’s simple linear regression)
4. FIT a simple linear regression model to the data
5. ASSESS the model fit (L-I-N-E-R Conditions)

MLB2020 <- read_csv("https://sites.psu.edu/sar320/files/2023/08/MLB2020.csv")

Rows: 16 Columns: 7── Column specification ────────────────────────────────────────────────────────────────────────────────────────────────────────────
Delimiter: ","
chr (2): Game, Day or Night
dbl (5): Score, Margin, Time, Pitchers, Temp
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

glimpse(MLB2020)

Rows: 16
Columns: 7
$ Game           <chr> "LA Dodgers vs LA Angels", "San Diego Padres vs Arizona Diamondbacks", "Washington Nationals vs Baltimore O…
$ Score          <dbl> 11, 13, 10, 6, 9, 11, 10, 4, 3, 3, 6, 6, 16, 8, 13, 5
$ Margin         <dbl> 1, 1, 4, 4, 3, 1, 2, 2, 1, 1, 2, 2, 6, 4, 1, 1
$ Time           <dbl> 197, 203, 190, 139, 150, 204, 202, 192, 150, 179, 150, 162, 185, 165, 218, 217
$ `Day or Night` <chr> "Night", "Night", "Night", "Day", "Day", "Day", "Night", "Night", "Night", "Night", "Day", "Day", "Night", …
$ Pitchers       <dbl> 11, 10, 9, 6, 12, 11, 7, 8, 7, 8, 9, 6, 8, 6, 11, 13
$ Temp           <dbl> 88, 77, 75, 86, 84, 84, 90, 86, 73, 78, 73, 76, 76, 79, 86, 82

head(MLB2020)

NA

Extra Practice: Textbook Prices