Chi-squared, Student’s t, and F Distributions
Exact distributions generated by normal samples and quadratic forms
1 Topic at a Glance
One-sentence summary: Chi-squared, Student’s t, and F distributions arise from sums of squared standard normals and ratios built from independent normal quantities, providing exact reference laws for classical normal-sample inference.
| Distribution | Construction | Support | Degrees of freedom | Classical role |
|---|---|---|---|---|
| \(\chi^2_\nu\) | Sum of squared standard normals | \(x>0\) | \(\nu>0\) | Variances and quadratic forms |
| \(t_\nu\) | Normal divided by an independent estimated scale | \(t\in\mathbb R\) | \(\nu>0\) | Means and regression coefficients |
| \(F_{\nu_1,\nu_2}\) | Ratio of independent scaled chi-squared variables | \(f>0\) | \(\nu_1,\nu_2>0\) | Variance ratios and model comparisons |
| Prerequisite | Normal and multivariate normal distributions |
Degrees of freedom record how many independent directions remain in a quadratic calculation after constraints or fitted quantities have been accounted for. They are structural parameters, not sample sizes in disguise.
2 The Chi-squared Distribution
Let \(Z_1,\ldots,Z_\nu\) be independent standard normal variables. Then
\[ \boxed{U=\sum_{i=1}^{\nu}Z_i^2\sim\chi^2_\nu.} \]
For any real \(\nu>0\), the density is
\[ f_U(u)= \frac{1}{2^{\nu/2}\Gamma(\nu/2)} u^{\nu/2-1}e^{-u/2}, \qquad u>0. \]
Thus \(\chi^2_\nu\) is a gamma distribution with shape \(\nu/2\) and scale 2. Its main moments are
\[ \boxed{E(U)=\nu, \qquad \operatorname{Var}(U)=2\nu.} \]
The distribution is right-skewed at small \(\nu\) and becomes more nearly symmetric as \(\nu\) grows. Independent chi-squared variables add:
\[ U_1\sim\chi^2_{\nu_1},\quad U_2\sim\chi^2_{\nu_2},\quad U_1\perp\!\!\!\perp U_2 \quad\Longrightarrow\quad U_1+U_2\sim\chi^2_{\nu_1+\nu_2}. \]
The moment-generating function \(M_U(t)=(1-2t)^{-\nu/2}\) for \(t<1/2\) proves this closure immediately.
3 Student’s t Distribution
Let
\[ Z\sim N(0,1), \qquad U\sim\chi^2_\nu, \qquad Z\perp\!\!\!\perp U. \]
Then
\[ \boxed{T=\frac{Z}{\sqrt{U/\nu}}\sim t_\nu.} \]
Its density is
\[ f_T(t)= \frac{\Gamma\{(\nu+1)/2\}} {\sqrt{\nu\pi}\,\Gamma(\nu/2)} \left(1+\frac{t^2}{\nu}\right)^{-(\nu+1)/2}, \qquad -\infty<t<\infty. \]
The distribution is symmetric about zero but has heavier tails than \(N(0,1)\) because its denominator is random. Its moments require care:
\[ E(T)=0\quad\text{only when }\nu>1, \]
\[ \operatorname{Var}(T)=\frac{\nu}{\nu-2} \quad\text{only when }\nu>2. \]
For \(\nu=1\), the t distribution is Cauchy and has neither a finite mean nor variance. As \(\nu\to\infty\), \(U/\nu\) concentrates near 1 and \(t_\nu\) converges in distribution to \(N(0,1)\).
The independence in the construction is essential. Dividing a normal variable by an arbitrary correlated scale estimate does not generally produce a t distribution.
4 The F Distribution
Let independent variables satisfy
\[ U_1\sim\chi^2_{\nu_1}, \qquad U_2\sim\chi^2_{\nu_2}. \]
Then
\[ \boxed{ F=\frac{U_1/\nu_1}{U_2/\nu_2} \sim F_{\nu_1,\nu_2}. } \]
The density is
\[ f_F(f)= \frac{1}{B(\nu_1/2,\nu_2/2)} \left(\frac{\nu_1}{\nu_2}\right)^{\nu_1/2} f^{\nu_1/2-1} \left(1+\frac{\nu_1}{\nu_2}f\right)^{-(\nu_1+\nu_2)/2}, \qquad f>0. \]
It is usually right-skewed. Its mean exists only when \(\nu_2>2\) and equals
\[ E(F)=\frac{\nu_2}{\nu_2-2}. \]
Its variance exists only when \(\nu_2>4\):
\[ \operatorname{Var}(F)= \frac{2\nu_2^2(\nu_1+\nu_2-2)} {\nu_1(\nu_2-2)^2(\nu_2-4)}. \]
Two transformations are especially useful. If \(F\sim F_{\nu_1,\nu_2}\), then
\[ \frac{1}{F}\sim F_{\nu_2,\nu_1}, \]
and, if \(T\sim t_\nu\),
\[ \boxed{T^2\sim F_{1,\nu}.} \]
The latter identity explains why a two-sided t test and a one-degree-of-freedom F test give the same p-value when applied to the same model contrast.
5 Exact Normal-sample Results
Suppose \(X_1,\ldots,X_n\) are independent \(N(\mu,\sigma^2)\) observations, and define
\[ \bar X=\frac1n\sum_{i=1}^nX_i, \qquad S^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar X)^2. \]
Normal-sample geometry gives three linked facts:
\[ \bar X\sim N\left(\mu,\frac{\sigma^2}{n}\right), \]
\[ \frac{(n-1)S^2}{\sigma^2}\sim\chi^2_{n-1}, \]
and \(\bar X\) is independent of \(S^2\). Therefore
\[ \boxed{ \frac{\bar X-\mu}{S/\sqrt n}\sim t_{n-1}. } \]
The loss of one degree of freedom comes from the constraint
\[ \sum_{i=1}^n(X_i-\bar X)=0. \]
Only \(n-1\) residual directions can vary independently after the mean is estimated.
For two independent normal samples,
\[ \frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2} \sim F_{n_1-1,n_2-1}. \]
Under \(\sigma_1^2=\sigma_2^2\), the raw ratio \(S_1^2/S_2^2\) has this F distribution. These are exact finite-sample statements under independence and normality. Their use as approximations under nonnormal sampling requires separate asymptotic or robustness arguments.
6 Quadratic Forms and Geometry
Let \(\mathbf Z\sim N_p(\mathbf0,\mathbf I)\) and let \(\mathbf P\) be a symmetric idempotent matrix: \(\mathbf P^2=\mathbf P\). If \(\operatorname{rank}(\mathbf P)=r\), then
\[ \boxed{\mathbf Z^\mathsf T\mathbf P\mathbf Z\sim\chi^2_r.} \]
An orthogonal rotation diagonalizes \(\mathbf P\) into \(r\) ones and \(p-r\) zeros, leaving a sum of \(r\) independent squared standard normals. This geometric result underlies residual sums of squares, analysis of variance, and many likelihood-ratio approximations.
If two projection matrices select orthogonal subspaces, their quadratic forms are independent. Ratios of the corresponding mean squares then have F distributions. The numerator degrees of freedom count directions attributed to a model term; the denominator degrees of freedom count directions left for estimating residual variation.
7 Worked Example: Uncertainty in a Mean
Suppose \(n=10\) independent biological specimens are modeled as normal with unknown \(\mu\) and \(\sigma^2\). If the observed mean is \(\bar x=2.4\) and sample standard deviation is \(s=0.6\), then the pivot for a proposed mean \(\mu_0\) is
\[ t=\frac{2.4-\mu_0}{0.6/\sqrt{10}}, \]
and its exact reference distribution under the model is \(t_9\). Replacing the t reference by a standard normal reference would ignore uncertainty in the estimated scale and produce narrower finite-sample intervals.
The ten independently sampled specimens determine the degrees of freedom. Measuring thousands of genes per specimen or sequencing each specimen more deeply may reduce technical error, but it does not change the biological sample size from 10. If wells, reads, or cells are nested within specimens, treating them as independent observations inflates degrees of freedom and understates uncertainty.
8 Assumptions and Sensitivity
| Requirement | Exact result affected | Failure mechanism | Response |
|---|---|---|---|
| Independent observations | All three normal-sample pivots | Related subjects, repeated measures, batches | Model clustering or dependence at the experimental-unit level |
| Normal observations | Exact \(\chi^2\), t, and F laws | Skewness, heavy tails, mixtures, outliers | Inspect data-generating process; use robust, permutation, bootstrap, or suitable model |
| Correct common-variance assumption | Pooled two-sample t and some F tests | Group-specific variance | Welch-type inference or heteroscedastic model |
| Numerator–denominator independence | t and F constructions | Reusing dependent scale information | Derive the joint law or use a justified alternative |
| Correct degrees of freedom | Tail areas and critical values | Ignoring fitted parameters or hierarchy | Derive degrees of freedom from the model and design |
Student’s one-sample pivot is often reasonably robust to mild nonnormality when independent data are not strongly skewed or heavy-tailed, especially as \(n\) grows. Variance tests based directly on chi-squared or F distributions are much more sensitive to nonnormality. Robustness is not permission to ignore a visibly incompatible design or distribution.
9 Common Pitfalls
- Treating degrees of freedom as an arbitrary software detail: they encode constraints and independent variation in the model.
- Using a t distribution merely because the sample is small: the exact result also requires a normal model and an independent chi-squared scale estimate.
- Assuming the t variance always exists: it is infinite for \(1<\nu\leq2\) and undefined for \(\nu\leq1\).
- Using the classical F variance-ratio test without checking normality: it is especially sensitive to heavy tails and outliers.
- Confusing a reference distribution with the observed statistic: the statistic is a realized value; the distribution describes its repeated-sampling behavior under assumptions.
- Inflating degrees of freedom with technical measurements: independent biological units, not the number of features or reads, govern biological inference.
10 Connections to the Knowledge Map
- Normal and multivariate normal distributions supplies the normal variables and Gaussian geometry.
- Expectation, variance, and moments provides moment-generating functions and variance identities.
- Independence and conditional independence explains the factorization required in ratios and sums.
- Transformations of random variables supplies the tools behind squared and ratio distributions.
- Random variables and probability distributions distinguishes densities, support, and realized values.
11 Take-home Messages
- A chi-squared variable is a sum of independent squared standard normals and is a gamma-family distribution.
- A t variable divides a standard normal by an independent chi-squared scale estimate, producing heavier tails.
- An F variable is a ratio of independent mean squares, and \(t_\nu^2\) has an \(F_{1,\nu}\) distribution.
- Exact normal-sample inference depends on independence, normality, and correctly counted degrees of freedom.
- Variance-based chi-squared and F procedures are particularly sensitive to nonnormal observations.
- Technical measurements cannot supply the biological degrees of freedom missing from an experiment.
12 References
- Student. “The Probable Error of a Mean.” Biometrika 6(1), 1908, 1–25. https://doi.org/10.2307/2331554.
- Fisher, R. A. “On a Distribution Yielding the Error Functions of Several Well Known Statistics.” Proceedings of the International Congress of Mathematicians, Toronto, Volume 2, 1924, 805–813. https://digital.library.adelaide.edu.au/items/40fa0ae6-b451-4f9a-b63d-531001bc8d0d.
- Casella, G. and Berger, R. L. Statistical Inference, 2nd ed. Duxbury, 2002.
- Johnson, N. L., Kotz, S., and Balakrishnan, N. Continuous Univariate Distributions, Volume 2, 2nd ed. Wiley, 1995.
- NIST/SEMATECH. “Gallery of Distributions.” e-Handbook of Statistical Methods. https://www.itl.nist.gov/div898/handbook/eda/section3/eda366.htm.
- R Core Team. “The Student t Distribution.” R Statistical Functions Documentation. https://stat.ethz.ch/R-manual/R-devel/library/stats/html/TDist.html.
- R Core Team. “The F Distribution.” R Statistical Functions Documentation. https://stat.ethz.ch/R-manual/R-devel/library/stats/html/FDist.html.
- R Core Team. “The Chi-squared Distribution.” R Statistical Functions Documentation. https://stat.ethz.ch/R-manual/R-devel/library/stats/html/Chisquare.html.