Dirichlet Distribution

A probability model for compositions and uncertain categorical probabilities

Statistics
Probability
Distributions
Compositional Data
The Dirichlet distribution on the simplex, including normalization, moments, gamma construction, aggregation, conjugacy, and compositional interpretation.
Author

Jiahong Sun

Published

September 13, 2026

Modified

September 13, 2026

1 Topic at a Glance

One-sentence summary: The Dirichlet distribution describes uncertainty about a vector of nonnegative proportions constrained to sum to one.

Item Description
Domain Probability & Randomness; multivariate continuous distributions
Support The \((K-1)\)-dimensional simplex \(p_k>0\), \(\sum_kp_k=1\)
Parameters Concentrations \(\boldsymbol\alpha=(\alpha_1,\ldots,\alpha_K)\) with every \(\alpha_k>0\)
Core uses Priors for categorical probabilities, random compositions, Dirichlet–multinomial mixtures
Typical applications Cell-type proportions, taxonomic compositions, ancestry fractions, uncertain category probabilities
Prerequisites Categorical and multinomial distributions; gamma and beta distributions

2 Distribution on the Simplex

Define the open simplex

\[ \mathcal S_{K-1} =\left\{\boldsymbol p\in(0,1)^K:\sum_{k=1}^Kp_k=1\right\}. \]

A random vector \(\boldsymbol P=(P_1,\ldots,P_K)\) has a Dirichlet distribution with concentration vector \(\boldsymbol\alpha\) when

\[ \boxed{ f(\boldsymbol p) =\frac{\Gamma(\alpha_0)}{\prod_{k=1}^K\Gamma(\alpha_k)} \prod_{k=1}^Kp_k^{\alpha_k-1}, \qquad \boldsymbol p\in\mathcal S_{K-1}, } \]

where

\[ \alpha_0=\sum_{k=1}^K\alpha_k. \]

The density is with respect to \((K-1)\)-dimensional volume on the simplex. Only \(K-1\) components are free because \(P_K=1-\sum_{k=1}^{K-1}P_k\).

The normalizing constant is the multivariate beta function

\[ B(\boldsymbol\alpha) =\frac{\prod_k\Gamma(\alpha_k)}{\Gamma(\alpha_0)}. \]

For \(K=2\), \(P_1\sim\operatorname{Beta}(\alpha_1,\alpha_2)\) and \(P_2=1-P_1\). The Dirichlet is therefore the multivariate extension of the beta distribution.

3 Mean, Variance, and Dependence

Each component has marginal distribution

\[ P_k\sim\operatorname{Beta}(\alpha_k,\alpha_0-\alpha_k), \]

and hence

\[ \boxed{E(P_k)=\frac{\alpha_k}{\alpha_0}.} \]

The variances and covariances are

\[ \boxed{ \operatorname{Var}(P_k) =\frac{\alpha_k(\alpha_0-\alpha_k)} {\alpha_0^2(\alpha_0+1)}, } \]

\[ \boxed{ \operatorname{Cov}(P_j,P_k) =-\frac{\alpha_j\alpha_k} {\alpha_0^2(\alpha_0+1)}, \qquad j\neq k. } \]

Negative covariance is forced by closure: if one component rises while the total stays one, at least one other component must fall. It is not, by itself, evidence that two biological categories inhibit one another.

3.1 Mean–concentration parameterization

Let

\[ m_k=\frac{\alpha_k}{\alpha_0}, \qquad \sum_km_k=1. \]

Then \(\alpha_k=\alpha_0m_k\), and

\[ E(P_k)=m_k, \qquad \operatorname{Var}(P_k)=\frac{m_k(1-m_k)}{\alpha_0+1}. \]

The vector \(\boldsymbol m\) locates the distribution, while \(\alpha_0\) controls concentration around it. With \(\boldsymbol m\) fixed, larger \(\alpha_0\) means less variable compositions. If every \(\alpha_k=1\), density is uniform over the simplex. Values below one favor sparse, boundary-near compositions; values above one favor the interior.

When every \(\alpha_k>1\), the interior mode is

\[ p_k^{\text{mode}}=\frac{\alpha_k-1}{\alpha_0-K}. \]

If some concentrations are at most one, the mode can lie on a boundary and this interior formula is not valid.

4 Construction from Independent Gamma Variables

Let

\[ G_k\sim\operatorname{Gamma}(\alpha_k,b) \]

independently with one common rate \(b\), and define

\[ G_+=\sum_{k=1}^KG_k, \qquad P_k=\frac{G_k}{G_+}. \]

Then

\[ \boxed{ \boldsymbol P\sim\operatorname{Dirichlet}(\boldsymbol\alpha), \qquad G_+\sim\operatorname{Gamma}(\alpha_0,b), \qquad \boldsymbol P\perp\!\!\!\perp G_+. } \]

The common rate cancels when each \(G_k\) is divided by the total. A change of variables from \((G_1,\ldots,G_K)\) to \((P_1,\ldots,P_{K-1},G_+)\) has absolute Jacobian \(G_+^{K-1}\). The joint density then factorizes into a Dirichlet density for \(\boldsymbol P\) and a gamma density for \(G_+\).

This construction gives a direct simulation algorithm: draw independent gamma variates and normalize them. It also clarifies the separation between composition and total abundance. A Dirichlet model alone describes only the normalized vector; it cannot recover information discarded by conditioning on or dividing by the total.

5 Aggregation and Subcompositions

The Dirichlet family is closed under category aggregation. If categories 1 and 2 are merged, then

\[ (P_1+P_2,P_3,\ldots,P_K) \sim \operatorname{Dirichlet}(\alpha_1+\alpha_2,\alpha_3,\ldots,\alpha_K). \]

More generally, summing components within any disjoint grouping sums their concentration parameters. In particular,

\[ \sum_{k\in A}P_k \sim\operatorname{Beta}\left( \sum_{k\in A}\alpha_k, \sum_{k\notin A}\alpha_k \right). \]

For two components, the internal split

\[ \frac{P_j}{P_j+P_k} \sim\operatorname{Beta}(\alpha_j,\alpha_k) \]

is independent of \(P_j+P_k\). These properties follow naturally by grouping the independent gamma variables before normalization.

6 Dirichlet–Multinomial Updating

Suppose a probability vector has prior

\[ \boldsymbol P\sim\operatorname{Dirichlet}(\alpha_1,\ldots,\alpha_K), \]

and, conditional on \(\boldsymbol P\), observed category counts satisfy

\[ \boldsymbol Y\mid\boldsymbol P \sim\operatorname{Multinomial}(n,\boldsymbol P). \]

Multiplying the likelihood by the prior gives

\[ \begin{aligned} p(\boldsymbol p\mid\boldsymbol y) &\propto \prod_{k=1}^Kp_k^{y_k} \prod_{k=1}^Kp_k^{\alpha_k-1}\\ &=\prod_{k=1}^Kp_k^{\alpha_k+y_k-1}, \end{aligned} \]

so

\[ \boxed{ \boldsymbol P\mid\boldsymbol Y=\boldsymbol y \sim\operatorname{Dirichlet}(\alpha_1+y_1,\ldots,\alpha_K+y_K). } \]

The posterior mean is

\[ E(P_k\mid\boldsymbol y) =\frac{\alpha_k+y_k}{\alpha_0+n} =\frac{\alpha_0}{\alpha_0+n}\frac{\alpha_k}{\alpha_0} +\frac{n}{\alpha_0+n}\frac{y_k}{n}. \]

It is a weighted average of the prior mean and observed proportion. Posterior predictive probabilities for the next category are the same posterior means.

If instead \(\boldsymbol P\) varies across experimental units and counts are multinomial conditional on each unit’s vector, integrating out \(\boldsymbol P\) gives a Dirichlet–multinomial distribution. Its counts are more variable than a single fixed-probability multinomial model. That mixture is useful for heterogeneity, but it imposes a specific covariance pattern and cannot reveal the source of heterogeneity by itself.

7 Worked Example: Cell-type Composition

Suppose a prior for three cell-type proportions is

\[ \boldsymbol P\sim\operatorname{Dirichlet}(2,2,2), \]

and an idealized classification sample yields counts

\[ \boldsymbol y=(12,5,3), \qquad n=20. \]

Under conditionally independent, correctly classified multinomial draws,

\[ \boldsymbol P\mid\boldsymbol y \sim\operatorname{Dirichlet}(14,7,5). \]

The posterior mean composition is

\[ E(\boldsymbol P\mid\boldsymbol y) =\left(\frac{14}{26},\frac7{26},\frac5{26}\right) \approx(0.538,0.269,0.192). \]

Statistically, this result concerns the latent category probabilities for the modeled sampling unit. Biologically, cells from one specimen are subsamples, not independent specimens. In a treatment comparison, the experimental units are independently assigned organisms or specimens; inference should preserve sample-level replication and represent classification uncertainty, batch, and between-sample variation.

8 When a Dirichlet Model Is Too Restrictive

Assumption or implication Why it matters Diagnostic question Alternative or sensitivity check
One composition with fixed \(\boldsymbol\alpha\) all units share one mean and concentration do batches or biological groups have different compositions? regression, hierarchical Dirichlet, or logistic-normal model
Negative pairwise covariance dependence has a restricted closure-driven form do log-ratios show richer covariance? logistic-normal or other compositional model
No point mass on a simplex boundary exact zeros have probability zero are zeros structural, rounded, or sampling zeros? explicit zero process, count model, or hurdle construction
Exchangeable multinomial trials given \(\boldsymbol P\) labels are conditionally independent with common probabilities are cells clustered, related, or differentially detected? preserve clusters and model detection or sampling
Composition is the target total abundance is irrelevant or modeled elsewhere does library size or total cell abundance carry biology? joint model for total and composition

The Dirichlet distribution is not a universal default for compositional data. Its covariance structure is determined by the mean and one concentration scale, whereas real log-ratio relationships can be more flexible. Closure also makes raw correlations difficult to interpret: an increase in one component necessarily changes the others even without direct biological interaction.

9 Common Pitfalls

  1. Treating the \(K\) components as unconstrained: the simplex has only \(K-1\) free dimensions and a singular raw covariance matrix.
  2. Interpreting negative covariance as antagonism: part of the association is mechanically induced by the sum-to-one constraint.
  3. Using a continuous Dirichlet likelihood for observed counts: counts have a multinomial or other discrete observation process; the Dirichlet usually models their latent probabilities.
  4. Ignoring zeros: the ordinary Dirichlet has no point mass on boundaries, so exact zeros need an observation or structural-zero explanation.
  5. Discarding totals without checking the estimand: normalization can remove biologically meaningful abundance information.
  6. Pooling cells across specimens: a large cell count does not replace independent biological replication.
  7. Assuming conjugacy guarantees fit: convenient posterior algebra does not validate classification, independence, or sampling assumptions.

10 Connections to the Knowledge Map

11 Take-home Messages

  1. The Dirichlet distribution is a continuous law for a probability vector on the simplex.
  2. Its mean vector and total concentration separate location from variability.
  3. Normalized independent gamma variables generate a Dirichlet composition independent of their total.
  4. Aggregation preserves the family, and a Dirichlet prior is conjugate to multinomial sampling.
  5. Negative covariance, boundary zeros, discarded totals, and restricted dependence require careful scientific interpretation.
  6. Compositional inference must respect independent biological samples rather than count cells or reads as replication.

12 References

Back to top