Why are outliers problematic in regression?

• A. They can distort coefficients and predictions
• B. They have no effect on model fit
• C. They always improve accuracy
• D. They only affect the intercept

Answer: (A)

In regression, the fitting process usually minimizes squared errors, so extreme values have outsized influence. An outlier can tug the estimated line toward itself, changing both the slope and the intercept. If the outlier has a typical x but an extreme y, it pulls the line away from where most data lie, leading to biased coefficients and poorer predictions for the rest of the data. If the outlier has an extreme x (high leverage), it can pull the line toward itself even if its y isn’t extreme, again shifting the coefficients and affecting predictions. This distortion also tends to worsen fit metrics and can make standard errors unreliable, undermining inference.

So the strongest reason is that outliers can distort the estimated coefficients and the resulting predictions because the common fitting method treats large residuals as particularly costly. The other statements don’t fit: there is a real effect on model fit, outliers don’t always improve accuracy, and they can influence more than just the intercept.