Regression Analysis: Lines, Equations & Coefficients

1. What Is Regression Analysis?

The term Regression was first coined in 1877 by British polymath Sir Francis Galton in his famous biological study showing that tall fathers tend to have sons who “regress” (step back) towards the average height of the population.

In modern Business Statistics, Regression Analysis is defined as the mathematical technique of estimating or predicting the unknown value of one variable (the dependent variable, Y) from the known value of another variable (the independent variable, X).

2. Distinction: Correlation vs. Regression

3. The Two Lines of Regression

In bivariate data, there are two distinct lines of regression:

Crucial Exam Rule:
The two lines of regression always intersect at the point of Means (x̄, ȳ)!

4. Regression Coefficients (b_yx and b_xy)

The slope of each regression line is called its Regression Coefficient:

Formulas for b_yx (Regression Coefficient of Y on X):

• Using Correlation & SD:
  b_yx = r × (σy ÷ σx)
• Using Raw Data Deviations:
  b_yx = [ N(ΣXY) - (ΣX)(ΣY) ] ÷ [ N(ΣX²) - (ΣX)² ]

Formulas for b_xy (Regression Coefficient of X on Y):

• Using Correlation & SD:
  b_xy = r × (σx ÷ σy)
• Using Raw Data Deviations:
  b_xy = [ N(ΣXY) - (ΣX)(ΣY) ] ÷ [ N(ΣY²) - (ΣY)² ]

5. Golden Properties of Regression Coefficients

These 5 properties are tested in nearly every DAVV BBA statistics exam:

1. Both Coefficients Have the SAME Algebraic Sign:
   Either both b_yx and b_xy are positive (+), or both are negative (-). It is mathematically impossible for one to be positive and the other negative!
2. Correlation Coefficient is the Geometric Mean of Regression Coefficients:
   r = ± √(b_yx × b_xy)
   (The sign of r is identical to the common sign of the regression coefficients!)
3. If One Coefficient Is Greater than 1, the Other MUST Be Less than 1:
   Since r² ≤ 1, the product (b_yx × b_xy) cannot exceed 1. If b_yx = 2.5, then b_xy must be 0.4 or lower!
4. The Arithmetic Mean of Regression Coefficients is Greater than or Equal to r:
   [ (b_yx + b_xy) ÷ 2 ] ≥ r
5. Independent of Change of Origin, but NOT Scale:
   Shifting the origin (adding/subtracting a constant) does not alter regression coefficients, but changing the scale (multiplying/dividing) changes their values proportionally.

6. Practical Activity 2: Temperature vs. Electricity Bill Forecasting

Scenario: A student team recorded the average monthly summer temperature (X in °C) and household electricity bill (Y in ₹ thousands) over 5 months:

• Mean Temperature (x̄) = 35°C
• Mean Electricity Bill (ȳ) = ₹4.5 (thousands) = ₹4,500
• Standard Deviation of Temperature (σx) = 4°C
• Standard Deviation of Bill (σy) = ₹1.2 (thousands)
• Correlation between Temperature and Bill (r) = +0.80

Task: Predict the electricity bill when summer temperature reaches 42°C!

Step-by-Step Forecasting Solution:

1. Identify the Required Equation:
   We want to predict Y (Electricity Bill) from given X (Temperature = 42°C). Therefore, we need the Regression Equation of Y on X!
2. Compute Regression Coefficient b_yx:
   b_yx = r × (σy ÷ σx)
b_yx = 0.80 × (1.2 ÷ 4) = 0.80 × 0.3 = 0.24
3. Set Up the Regression Equation of Y on X:
   Y - ȳ = b_yx × (X - x̄)
Y - 4.5 = 0.24 × (X - 35)
Y - 4.5 = 0.24X - (0.24 × 35)
Y - 4.5 = 0.24X - 8.4
Y = 0.24X - 8.4 + 4.5
   Prediction Equation: Y = 0.24X - 3.9
4. Substitute Given Temperature (X = 42):
   Y = (0.24 × 42) - 3.9
Y = 10.08 - 3.9 = ₹6.18 (in thousands) = ₹6,180

Executive Takeaway:
When temperature spikes to 42°C in peak summer, the household can budget for a ₹6,180 electricity bill, driven primarily by higher air conditioning loads!