Populations, Samples, Parameters, and Estimands

Defining what a study observes, what it targets, and where its conclusions apply

Statistics

Inference

Study Design

A design-first guide to populations, samples, parameters, estimands, experimental units, and the scope of statistical inference.

Author

Jiahong Sun

Published

September 21, 2026

Modified

September 21, 2026

1 The Target Comes Before the Calculation

Statistical inference uses observed data to learn about a target that is not fully observed. A scientifically meaningful analysis therefore needs two definitions before it needs an estimator:

Which units and outcomes constitute the population of interest?
Which precise feature of that population is the estimand?

These choices are not bookkeeping. A perfectly calculated estimate can answer the wrong question if the sample excludes relevant units, the experimental unit is misidentified, or the estimand does not match the biological claim.

Term	Role	Example in an RNA-seq study
Target population	Units to which the conclusion is intended to apply	Adults with a specified disease under defined eligibility criteria
Sampling frame	Operational list or process from which units can be recruited	Patients attending three participating hospitals
Sample	Units actually selected and observed	The 80 enrolled patients
Experimental unit	Smallest unit independently assigned or sampled for the target comparison	A patient, not a read or transcript
Parameter	A fixed, usually unknown feature of a probability model or population	The population mean log expression in each condition
Estimand	The fully specified quantity the scientific question asks for	The difference in population mean log expression between treatment strategies
Estimator	A rule mapping possible data to a value	The difference between two sample means
Estimate	The estimator’s realized value for the observed data	An observed difference of $0.42$ log units

2 Populations and Samples

A finite population is a specified collection of units, such as all 2,430 specimens in a biobank on a given date. A finite-population quantity can be defined without a probability model. For values $Y_1,\ldots,Y_N$, the population mean is

\[ \bar Y_N=\frac{1}{N}\sum_{i=1}^N Y_i. \]

Randomness can then arise from the sampling design: which units enter the sample. Design-based inference evaluates an estimator over repeated samples generated by that design while treating the finite population values as fixed.

A superpopulation is a conceptual data-generating distribution from which observed units are regarded as draws. Its mean is

\[ \mu=E_F(Y)=\int y\,dF(y), \]

where $F$ is the population distribution. Model-based inference treats the observations as random under assumptions about $F$ or a parameterized family. The distinction matters: the mean of the specimens currently stored in one biobank is not automatically the mean for all future patients.

The target population states the intended scope of the conclusion. The source population is the population accessible to the study, and the sampling frame is its operational representation. Differences among these sets create selection and transportability questions that a larger sample from the same narrow frame cannot solve.

3 Parameters and Estimands

A parameter indexes or summarizes a population or probability model. Examples include a Bernoulli probability $p$, a mean $\mu$, a variance $\sigma^2$, or a regression coefficient $\beta$. Parameters are treated as fixed but unknown in frequentist inference; observed data vary across hypothetical repetitions.

An estimand is the specific population-level quantity an analysis seeks to learn. It includes enough detail to make the scientific question unambiguous. The same dataset may support several estimands:

\[ \theta_{\text{mean}}=E(Y), \qquad \theta_{\text{median}}=F^{-1}(0.5), \qquad \theta_{\text{tail}}=P(Y>c). \]

These describe different features of the same outcome distribution. None is a generic “effect.” For a comparison, the scale also matters:

\[ \theta_D=E(Y\mid A=1)-E(Y\mid A=0), \]

\[ \theta_R=\frac{E(Y\mid A=1)}{E(Y\mid A=0)}, \qquad \theta_{\log R}=\log \theta_R. \]

The difference, ratio, and log ratio can lead to different notions of biological importance. Moreover, these conditional associations are not causal effects unless the design and assumptions justify a causal interpretation.

For a treatment question, a useful estimand specification identifies:

the population of units;
the treatment conditions or strategies being compared;
the outcome and measurement time;
the population-level summary, such as a mean difference or risk ratio; and
how post-assignment events, missing outcomes, rescue treatment, or treatment discontinuation enter the question.

This is the central idea of the ICH E9(R1) estimand framework: align the scientific objective, design, analysis, sensitivity analysis, and interpretation rather than allowing the available data or software default to define the target after the fact.

4 From an Estimand to Observed Data

An estimand may be well defined but not identified from the observed-data distribution. Identification asks whether the target is uniquely determined under the design and stated assumptions. Estimation asks how to calculate it efficiently and quantify uncertainty once identification is established.

Consider the population mean $\mu=E(Y)$ and an observed sample $Y_1,\ldots,Y_n$. Under independent sampling from the target population,

\[ \hat\mu=\bar Y=\frac{1}{n}\sum_{i=1}^nY_i \]

is an estimator of $\mu$. If inclusion depends on $Y$ and that mechanism is ignored, however, $\bar Y$ may estimate the mean among selected units rather than the target-population mean. Algebra cannot replace a defensible link between sample and target.

Likewise, random treatment assignment can identify a treatment effect in the randomized population, but generalizing it to a broader population requires additional sampling or transportability assumptions. “Randomized” describes treatment allocation, not necessarily population sampling.

5 Worked Bioinformatics Example

Suppose a study profiles peripheral-blood RNA from 40 treated and 40 control patients. Each library produces 30 million reads, and several aliquots are sequenced twice.

Biological experimental unit: the patient, if treatment is assigned to patients.
Technical units: aliquots, libraries, lanes, and reads nested within patients.
Target population: patients satisfying the stated clinical and laboratory criteria in the settings to which inference is intended to apply.
Outcome: for example, normalized log expression of a prespecified gene at day 14.
Estimand: the day-14 difference in mean expression under the two treatment strategies in that target population.
Estimator: a design-appropriate adjusted difference in sample means or regression coefficient.
Estimate: the numerical coefficient fitted to these 80 patients.

Replicate reads reduce measurement error for a patient’s expression value, but they do not turn 80 patients into millions of independent biological units. If samples come only from responders, the estimand concerns responders unless assumptions and additional data support a wider target. If the analysis selects the most changed gene after examining all genes, the target and uncertainty must also account for that selection.

6 Assumptions and Scope Checks

Question	Why it matters	Useful check
Who could enter the study?	Defines the source population and possible selection bias	Compare eligibility, recruitment, and exclusions with the target population
What was independently sampled or assigned?	Determines the experimental unit and effective replication	Draw the nesting and assignment structure before modeling
Is the target descriptive, associational, predictive, or causal?	Each goal requires different assumptions and validation	Write the scientific question without statistical method names
Is the estimand identified?	A unique target may not follow from observed data alone	State randomization, measurement, missing-data, and confounding assumptions
Does the estimator target the stated estimand?	Software can fit a different population summary or scale	Express the estimator’s probability limit under the proposed model
Where should the conclusion generalize?	Precision within a sample does not establish external validity	Describe differences between sample, source, and target populations

Sensitivity analysis should vary scientifically doubtful assumptions while holding the estimand clear. Changing both the assumptions and target at once makes discrepancies difficult to interpret.

7 Common Category Errors

Calling an estimate a parameter: $\mu$ is the unknown population mean; $\bar y=4.7$ is an observed estimate.
Letting the available sample define the target silently: convenience recruitment may provide a useful sample without representing the intended population.
Confusing measurement count with sample size: reads and cells can be subsamples of the same biological unit.
Treating model coefficients as self-explanatory: a coefficient’s estimand depends on outcome scale, conditioning variables, link function, and model specification.
Equating association with intervention: $E(Y\mid A=a)$ and the mean outcome under an intervention setting $A=a$ are different objects unless causal assumptions connect them.
Choosing the method before the estimand: a familiar estimator may be precise for a target that does not answer the scientific question.

8 Connections and Take-home Messages

Random variables and probability distributions separates a random variable from its realized value.
Expectation, variance, and moments defines common population summaries.
Law of large numbers explains when sample averages approach population expectations.
Asymptotic bias, consistency, and efficiency evaluates estimator sequences after the target has been defined.
Point estimators: bias, variance, and mean squared error compares rules for estimating a fixed estimand.

The population fixes the scope of inference; the estimand fixes the question; the estimator fixes the rule applied to possible data; and the estimate is the observed answer. Good inference requires alignment among all four, plus a design that supplies genuine biological replication and assumptions that identify the target.

9 References

Neyman, J. “On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection.” Journal of the Royal Statistical Society 97 (1934): 558–606. Article.
Lohr, S. L. Sampling: Design and Analysis, 3rd ed. Chapman and Hall/CRC, 2021. Publisher page.
International Council for Harmonisation. ICH E9(R1) Addendum on Estimands and Sensitivity Analysis in Clinical Trials. 2019. Official guideline.
Hernán, M. A. and Robins, J. M. Causal Inference: What If. Chapman & Hall/CRC, 2020. Official online book.
Lehmann, E. L. and Casella, G. Theory of Point Estimation, 2nd ed. Springer, 1998. Publisher page.

--- title: "Populations, Samples, Parameters, and Estimands" subtitle: "Defining what a study observes, what it targets, and where its conclusions apply" description: "A design-first guide to populations, samples, parameters, estimands, experimental units, and the scope of statistical inference." author: "Jiahong Sun" date: "2026-09-21" date-modified: "2026-09-21" categories: [Statistics, Inference, Study Design] toc: true number-sections: true draft: false --- # The Target Comes Before the Calculation Statistical inference uses observed data to learn about a target that is not fully observed. A scientifically meaningful analysis therefore needs two definitions before it needs an estimator: 1. **Which units and outcomes constitute the population of interest?** 2. **Which precise feature of that population is the estimand?** These choices are not bookkeeping. A perfectly calculated estimate can answer the wrong question if the sample excludes relevant units, the experimental unit is misidentified, or the estimand does not match the biological claim. | Term | Role | Example in an RNA-seq study | |---|---|---| | Target population | Units to which the conclusion is intended to apply | Adults with a specified disease under defined eligibility criteria | | Sampling frame | Operational list or process from which units can be recruited | Patients attending three participating hospitals | | Sample | Units actually selected and observed | The 80 enrolled patients | | Experimental unit | Smallest unit independently assigned or sampled for the target comparison | A patient, not a read or transcript | | Parameter | A fixed, usually unknown feature of a probability model or population | The population mean log expression in each condition | | Estimand | The fully specified quantity the scientific question asks for | The difference in population mean log expression between treatment strategies | | Estimator | A rule mapping possible data to a value | The difference between two sample means | | Estimate | The estimator's realized value for the observed data | An observed difference of $0.42$ log units | # Populations and Samples A **finite population** is a specified collection of units, such as all 2,430 specimens in a biobank on a given date. A finite-population quantity can be defined without a probability model. For values $Y_1,\ldots,Y_N$, the population mean is $$ \bar Y_N=\frac{1}{N}\sum_{i=1}^N Y_i. $$ Randomness can then arise from the sampling design: which units enter the sample. Design-based inference evaluates an estimator over repeated samples generated by that design while treating the finite population values as fixed. A **superpopulation** is a conceptual data-generating distribution from which observed units are regarded as draws. Its mean is $$ \mu=E_F(Y)=\int y\,dF(y), $$ where $F$ is the population distribution. Model-based inference treats the observations as random under assumptions about $F$ or a parameterized family. The distinction matters: the mean of the specimens currently stored in one biobank is not automatically the mean for all future patients. The **target population** states the intended scope of the conclusion. The **source population** is the population accessible to the study, and the **sampling frame** is its operational representation. Differences among these sets create selection and transportability questions that a larger sample from the same narrow frame cannot solve. # Parameters and Estimands A **parameter** indexes or summarizes a population or probability model. Examples include a Bernoulli probability $p$, a mean $\mu$, a variance $\sigma^2$, or a regression coefficient $\beta$. Parameters are treated as fixed but unknown in frequentist inference; observed data vary across hypothetical repetitions. An **estimand** is the specific population-level quantity an analysis seeks to learn. It includes enough detail to make the scientific question unambiguous. The same dataset may support several estimands: $$ \theta_{\text{mean}}=E(Y), \qquad \theta_{\text{median}}=F^{-1}(0.5), \qquad \theta_{\text{tail}}=P(Y>c). $$ These describe different features of the same outcome distribution. None is a generic “effect.” For a comparison, the scale also matters: $$ \theta_D=E(Y\mid A=1)-E(Y\mid A=0), $$ $$ \theta_R=\frac{E(Y\mid A=1)}{E(Y\mid A=0)}, \qquad \theta_{\log R}=\log \theta_R. $$ The difference, ratio, and log ratio can lead to different notions of biological importance. Moreover, these conditional associations are not causal effects unless the design and assumptions justify a causal interpretation. For a treatment question, a useful estimand specification identifies: - the population of units; - the treatment conditions or strategies being compared; - the outcome and measurement time; - the population-level summary, such as a mean difference or risk ratio; and - how post-assignment events, missing outcomes, rescue treatment, or treatment discontinuation enter the question. This is the central idea of the ICH E9(R1) estimand framework: align the scientific objective, design, analysis, sensitivity analysis, and interpretation rather than allowing the available data or software default to define the target after the fact. # From an Estimand to Observed Data An estimand may be well defined but not **identified** from the observed-data distribution. Identification asks whether the target is uniquely determined under the design and stated assumptions. Estimation asks how to calculate it efficiently and quantify uncertainty once identification is established. Consider the population mean $\mu=E(Y)$ and an observed sample $Y_1,\ldots,Y_n$. Under independent sampling from the target population, $$ \hat\mu=\bar Y=\frac{1}{n}\sum_{i=1}^nY_i $$ is an estimator of $\mu$. If inclusion depends on $Y$ and that mechanism is ignored, however, $\bar Y$ may estimate the mean among selected units rather than the target-population mean. Algebra cannot replace a defensible link between sample and target. Likewise, random treatment assignment can identify a treatment effect in the randomized population, but generalizing it to a broader population requires additional sampling or transportability assumptions. “Randomized” describes treatment allocation, not necessarily population sampling. # Worked Bioinformatics Example Suppose a study profiles peripheral-blood RNA from 40 treated and 40 control patients. Each library produces 30 million reads, and several aliquots are sequenced twice. - **Biological experimental unit:** the patient, if treatment is assigned to patients. - **Technical units:** aliquots, libraries, lanes, and reads nested within patients. - **Target population:** patients satisfying the stated clinical and laboratory criteria in the settings to which inference is intended to apply. - **Outcome:** for example, normalized log expression of a prespecified gene at day 14. - **Estimand:** the day-14 difference in mean expression under the two treatment strategies in that target population. - **Estimator:** a design-appropriate adjusted difference in sample means or regression coefficient. - **Estimate:** the numerical coefficient fitted to these 80 patients. Replicate reads reduce measurement error for a patient's expression value, but they do not turn 80 patients into millions of independent biological units. If samples come only from responders, the estimand concerns responders unless assumptions and additional data support a wider target. If the analysis selects the most changed gene after examining all genes, the target and uncertainty must also account for that selection. # Assumptions and Scope Checks | Question | Why it matters | Useful check | |---|---|---| | Who could enter the study? | Defines the source population and possible selection bias | Compare eligibility, recruitment, and exclusions with the target population | | What was independently sampled or assigned? | Determines the experimental unit and effective replication | Draw the nesting and assignment structure before modeling | | Is the target descriptive, associational, predictive, or causal? | Each goal requires different assumptions and validation | Write the scientific question without statistical method names | | Is the estimand identified? | A unique target may not follow from observed data alone | State randomization, measurement, missing-data, and confounding assumptions | | Does the estimator target the stated estimand? | Software can fit a different population summary or scale | Express the estimator's probability limit under the proposed model | | Where should the conclusion generalize? | Precision within a sample does not establish external validity | Describe differences between sample, source, and target populations | Sensitivity analysis should vary scientifically doubtful assumptions while holding the estimand clear. Changing both the assumptions and target at once makes discrepancies difficult to interpret. # Common Category Errors - **Calling an estimate a parameter:** $\mu$ is the unknown population mean; $\bar y=4.7$ is an observed estimate. - **Letting the available sample define the target silently:** convenience recruitment may provide a useful sample without representing the intended population. - **Confusing measurement count with sample size:** reads and cells can be subsamples of the same biological unit. - **Treating model coefficients as self-explanatory:** a coefficient's estimand depends on outcome scale, conditioning variables, link function, and model specification. - **Equating association with intervention:** $E(Y\mid A=a)$ and the mean outcome under an intervention setting $A=a$ are different objects unless causal assumptions connect them. - **Choosing the method before the estimand:** a familiar estimator may be precise for a target that does not answer the scientific question. # Connections and Take-home Messages - [Random variables and probability distributions](../probability-randomness/random-variables-and-probability-distributions.qmd) separates a random variable from its realized value. - [Expectation, variance, and moments](../probability-randomness/expectation-variance-and-moments.qmd) defines common population summaries. - [Law of large numbers](../asymptotic-thinking/law-of-large-numbers.qmd) explains when sample averages approach population expectations. - [Asymptotic bias, consistency, and efficiency](../asymptotic-thinking/asymptotic-bias-consistency-and-efficiency.qmd) evaluates estimator sequences after the target has been defined. - [Point estimators: bias, variance, and mean squared error](point-estimators-bias-variance-and-mean-squared-error.qmd) compares rules for estimating a fixed estimand. The population fixes the scope of inference; the estimand fixes the question; the estimator fixes the rule applied to possible data; and the estimate is the observed answer. Good inference requires alignment among all four, plus a design that supplies genuine biological replication and assumptions that identify the target. # References - Neyman, J. “On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection.” *Journal of the Royal Statistical Society* 97 (1934): 558--606. [Article](https://doi.org/10.1111/j.2397-2335.1934.tb04184.x). - Lohr, S. L. *Sampling: Design and Analysis*, 3rd ed. Chapman and Hall/CRC, 2021. [Publisher page](https://doi.org/10.1201/9780429296284). - International Council for Harmonisation. *ICH E9(R1) Addendum on Estimands and Sensitivity Analysis in Clinical Trials*. 2019. [Official guideline](https://database.ich.org/sites/default/files/E9-R1_Step4_Guideline_2019_1203.pdf). - Hernán, M. A. and Robins, J. M. *Causal Inference: What If*. Chapman & Hall/CRC, 2020. [Official online book](https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/). - Lehmann, E. L. and Casella, G. *Theory of Point Estimation*, 2nd ed. Springer, 1998. [Publisher page](https://doi.org/10.1007/b98854).