Parametric methods used to solve univariate and multivariate problems have, as a limitation, the need to introduce very restrictive assumptions (normality, homoscedasticity, independence and identical distribution of the error component) which are rarely satisfied in practice. Nonparametric methods provide an alternative when:
In the nonparametric framework, the representative indicator of a distribution is the median, which unlike the mean, is a robust estimator. Exploiting the fact that for any continuous random variable:
\[ Pr(X \geq Me) = Pr(X \leq Me) = \frac{1}{2} \]
Alternatively, we can use rank variables, defined as the integer corresponding to the position that the random variable occupies when passing from the random sample \((X_1, X_2, \ldots, X_n)\) to the ordered random sample \((X_{(1)}, X_{(2)}, \ldots, X_{(n)})\). Ranks are very robust even in the presence of notable variations in data — any monotone transformation does not alter them.
Let \(Me\) be the median of a continuous random variable \(X\). We construct a test to verify \(H_0: Me = Me_0\) against \(H_1: Me \neq Me_0\).
If \(H_0\) is true, approximately half the observations should be above (below) \(Me_0\). For a random sample \((X_1, X_2, \ldots, X_n)\), the number of observations \(T_n\) greater than \(Me_0\) is a binomial random variable:
\[ T_n \sim Bi(n, \theta) \]
Testing \(H_0: Me = Me_0\) is equivalent to testing \(H_0: \theta = \frac{1}{2}\) vs. \(H_1: \theta \neq \frac{1}{2}\).
Under \(H_0\), \(T_n \sim Bi(n, 1/2)\), so on average the sample will contain \(n/2\) observations above \(Me_0\). The critical region is:
\[ |T_n - n/2| \geq c_{\alpha/2} \]
Using the normal approximation to the binomial with continuity correction:
\[ c_{\alpha/2} \approx z_{\alpha/2} \frac{\sqrt{n} - 1}{2} \]
The procedure is called the sign test because to compute the test statistic we mark with + (−) values above (not above) \(Me_0\) and count the number of positive signs.
This test can be used for paired data. Suppose we want to verify the effect of a known action on the same statistical unit: \(X_i\) is measured before the experiment and \(Y_i\) after. Let:
If \(H_0: X_i = Y_i\) is true (no effect), then \(\theta = 1/2\). The number of + signs follows \(Bi(n, \theta)\).
This test can be used to verify whether a random sample has a certain median or whether paired differences have median equal to 0. It is the nonparametric equivalent of Student's t-test for paired (dependent) samples.
Given the random sample of paired observations \((X_1, Y_1), (X_2, Y_2), \ldots, (X_n, Y_n)\), let \(D_i = (Y_i - X_i)\) be the corresponding differences. We assume the \(D_i\) are continuous, symmetric, independent and all with the same median.
The hypotheses to test are:
The test statistic is the sum of ranks \(r(|D_i|)\) corresponding to differences \(D_i > 0\):
\[ T_n = \sum_{i=1}^{n} r(|D_i|) \cdot I(D_i > 0) \]
where \(I(\cdot)\) is the indicator function. Under the null hypothesis:
\[ E(T_n) = \frac{n(n+1)}{4} \qquad V(T_n) = \frac{n(n+1)(2n+1)}{24} \]
For large \(n\) (n > 15), we can use the normal approximation (with continuity correction):
\[ \frac{T_n - n(n+1)/4 - 1/2}{\sqrt{n(n+1)(2n+1)/24}} \xrightarrow{d} N(0,1) \]
Consider paired data. Let:
Under \(H_0\), the statistic \(U\) has a binomial distribution with parameters \(\nu\) and \(1/2\), i.e., \(U \sim Bin(\nu, 1/2)\). Under the alternative \(H_1\), \(U\) is still binomial but with parameters \(\nu\) and \(\theta > 1/2\).
For example, with \(\nu = 20\) and \(U = 17\):
\[ Pr(U \geq 17 | D) = \sum_{i \geq 17} \binom{20}{i} 2^{-20} = 0.0013 \]
which is significant at level \(\alpha = 0.005\).
Permutation tests are characterized by conditioning on the set of observed data, which is a set of sufficient statistics regardless of the underlying model. These tests are called distribution-free: the distributions of the tests are completely independent of the law governing the random variable on which we want to make inference.
The hypothesis \(H_0: \{Y_A \stackrel{d}{=} Y_B\}\) implies the exchangeability of variables \(Y_A\) and \(Y_B\) within each unit. The sign of each difference \(X_i\) can be thought of as assigned with probability \(1/2\).
Consider the test statistic \(T = \sum_i X_i\). The conditional distribution \(F_T(t|X)\) when the observed points \(X = \{X_i, i=1,\ldots,n\}\) are fixed, is obtained under \(H_0\) by randomly assigning signs + and − to each difference with equal probability.
The permutation sample space \(\mathcal{X}/X\) contains \(M = 2^\nu\) points (where \(\nu\) is the number of non-zero differences).
When the permutation space is too large to enumerate exhaustively, we use the Conditional Monte Carlo method, which samples points from the conditional permutation orbit:
The \(B\) values \(T^*\) simulate the null permutation distribution of \(T\) and allow estimation of the permutation c.d.f. \(F(z|X)\). The estimated p-value from the observed value \(T_0\) is:
\[ \hat{\lambda} = \hat{L}_B^*(T_0) = \frac{\#(T^* \geq T_0)}{B} \]
If \(\hat{\lambda} \leq \alpha\), we reject \(H_0\) according to the usual hypothesis testing rules.