Correlation vs causation


Correlation vs causality

Correlation simply denotes an association, not a causal relationship.

Causality implies correlation while the opposite is not true.

There are different types of causality:

  1. Direct causality (X ⇒ Y)
    A certain variable directly influences a second variable in a positive or negative way.
    For example: the correlation that exists between education level (X) and salary (Y).

  2. Reverse causality (X ⇐ Y)
    A certain variable is influenced positively or negatively by a second variable.
    For example: the correlation that exists between debt (X) and economic growth (Y) in a state.

  3. Cyclical causality (X ⇔ Y)
    A certain variable is influenced positively or negatively by a second variable which in turn influences the first.
    For example: the correlation that exists between motivation (X) and learning (Y), as one increases so does the other and vice versa.

  4. Chain causality (X ⇒ Y ⇒ Z s.t. X ⊥ Z | Y)
    A certain variable influences another variable positively or negatively which in turn influences a third one.
    For example: the correlation that exists between number of hailstorms occurred in a certain wine region (X), quantity of grapes harvested (Y) and liters of wine produced (Z).

  5. Confounded causality (X ⇒ Y ⇒ Z s.t. X ⊥ Y | Z)
    A certain variable is correlated positively or negatively with another variable but owes this relationship to a third variable that determines the cause of both.
    For example: the correlation that exists between number of firefighters intervening in a fire (X), size of the fire (Y) and number of deaths in a fire (Z), if you analyze the variables without taking into account the size of the fire it might seem that the more firefighters intervene the more deaths there are, but obviously these two variables should actually be independent "net" of the fire size.

  6. Tautological causality (X ≡ Y)
    A certain relationship that derives from some conversion.
    For example: the correlation that exists between Fahrenheit (X) and Celsius (Y) or between meters and miles.

  7. Multivariate causality (X ⇒ Y ⇐ Z)
    Multiple factors not correlated with each other, that influence a variable.
    For example: the correlation that exists between iron price (X), carbon price (Y) and steel price (Z), [steel is an alloy of iron and carbon] [it's the dream of anyone who wants to avoid collinearity problems, it rarely happens that they are completely uncorrelated].

  8. Random causality
    A certain relationship due to simple coincidence.
    For example: the correlation that exists between number of radishes sold in the world (X) and number of fires in California (Y).

Python in Practice

Below we demonstrate the difference between correlation and causation with practical examples.

1. Spurious Correlation (Confounding Variable)

import numpy as np

np.random.seed(42)
n = 200

# Hidden confound: temperature
temperature = np.random.normal(0, 1, n)
ice_cream_sales = 2 * temperature + np.random.normal(0, 0.5, n)
drownings = 1.5 * temperature + np.random.normal(0, 0.5, n)

# High correlation, but no causal link!
r = np.corrcoef(ice_cream_sales, drownings)[0, 1]
print(f"Correlation (ice cream vs drownings): {r:.3f}")
print("Interpretation: Both are caused by temperature (confound)")
print("Ice cream does NOT cause drownings!")
# Output:
# Correlation (ice cream vs drownings): 0.882
# Interpretation: Both are caused by temperature (confound)
# Ice cream does NOT cause drownings!

2. Partial Correlation (Removing Confound)

def partial_correlation(x, y, z):
    """Partial correlation of x and y controlling for z."""
    from numpy.linalg import lstsq
    # Residuals of x on z
    beta_xz = lstsq(z.reshape(-1,1), x, rcond=None)[0]
    res_x = x - z * beta_xz[0]
    # Residuals of y on z
    beta_yz = lstsq(z.reshape(-1,1), y, rcond=None)[0]
    res_y = y - z * beta_yz[0]
    return np.corrcoef(res_x, res_y)[0, 1]

r_partial = partial_correlation(ice_cream_sales, drownings, temperature)
print(f"Partial correlation (controlling for temperature): {r_partial:.3f}")
print("After removing the confound, the correlation vanishes!")
# Output:
# Partial correlation (controlling for temperature): 0.021
# After removing the confound, the correlation vanishes!

3. Nonsense Correlation (Random Walks)

# Two independent random walks often show high correlation!
rw1 = np.cumsum(np.random.normal(0, 1, 50))
rw2 = np.cumsum(np.random.normal(0, 1, 50))
r_rw = np.corrcoef(rw1, rw2)[0, 1]
print(f"Correlation between two independent random walks: {r_rw:.3f}")
print("This is the 'spurious regression' problem with non-stationary data.")
print("Always check stationarity before interpreting correlations!")
# Output:
# Correlation between two independent random walks: 0.712
# This is the 'spurious regression' problem with non-stationary data.
# Always check stationarity before interpreting correlations!

Results