Skip to content

FDR vs p-value: what a 5% false discovery rate controls

Oak
A lab instrument pulls a tray of glowing amber sample tubes from a huge wall of dim blue ones, with a few blue tubes mixed into the selection.

A p-value answers a question about one test: if the null hypothesis were true for this gene, how often would I see a statistic this extreme or more? An FDR of 0.05 answers a question about a set: among the features I called significant, roughly 5% are expected to be null. The two numbers live at different levels. The p-value is a property of a single feature, and the false discovery rate is a property of the selection rule applied to the whole experiment. An “FDR-adjusted p-value” (often labeled padj, adj.P.Val, or q-value) is the smallest FDR threshold at which that particular feature would still make the cut.

That last definition is the one people get wrong most often. An adjusted p-value of 0.03 means that if you draw the line here and take every feature with an adjusted value at or below 0.03, the expected proportion of false positives in the resulting list is about 3%. The value does not mean a 3% chance that this gene is null.

Why raw p-values fail at omics scale

Run one test at α = 0.05 and you accept a 5% chance of a false positive. Run 20,000 tests on features that are all null and you expect about 1,000 p-values below 0.05 by construction, because p-values under the null are uniform on [0, 1]. Nothing is broken. The threshold simply was never designed to survive that many repetitions.

Bonferroni fixes this by controlling the family-wise error rate, the probability of even one false positive anywhere in the experiment. For 20,000 genes the per-test threshold becomes 0.05 / 20000 = 2.5 × 10⁻⁶. That is the right criterion for genome-wide association studies, where the conventional 5 × 10⁻⁸ line reflects roughly a million independent common-variant tests. It is far too strict for discovery-stage transcriptomics or proteomics, where you are willing to tolerate some false positives in exchange for finding real signal. Standard treatments of omics analysis make the same split: control the family-wise rate when a single false claim is costly, and control the false discovery rate when you are generating a ranked list for follow-up.1

What Benjamini-Hochberg does, mechanically

Sort your m p-values ascending: p₍₁₎ ≤ p₍₂₎ ≤ … ≤ p₍ₘ₎. Find the largest k such that p₍ₖ₎ ≤ (k / m) · α. Reject the first k hypotheses. That is the whole procedure. The adjusted value reported by software is the step-up monotone version, padj(i) = min over j ≥ i of (m/j) · p(j), capped at 1.

In R this is one call:

padj <- p.adjust(pvals, method = "BH")   # also "BY", "holm", "bonferroni"
sum(padj < 0.05, na.rm = TRUE)

DESeq2 and limma do it for you. results(dds, alpha = 0.05) returns padj, and topTable(fit, adjust.method = "BH") returns adj.P.Val. Both are Benjamini-Hochberg unless you override the method.

The interpretation follows directly. If 500 of 20,000 genes come back with padj < 0.05, you expect roughly 25 of those 500 to be null. You do not know which 25. FDR gives you no per-gene guarantee at all, which is why ranking within the significant set still matters when you pick candidates for orthogonal confirmation.2

q-values, π₀, and why they differ from BH

Benjamini-Hochberg implicitly assumes every hypothesis could be null, so it uses m in the denominator. Storey’s q-value estimates π₀, the actual proportion of true nulls, and scales by π₀ · m instead. When a large fraction of your features are genuinely changing, π₀ might be 0.7, and the q-values come out about 30% smaller than BH-adjusted p-values. That means more discoveries at the same nominal FDR.

library(qvalue)
qobj <- qvalue(pvals)          # default smoother fit for pi0
qobj$pi0                       # estimated null proportion
qobj$qvalues                   # q-values
qobj$lfdr                      # local FDR, per-feature posterior null probability

The lfdr column is the one people usually want when they say “probability this gene is null.” Local FDR is a per-feature quantity, unlike the tail-based q-value. It is also noisier and depends on how well the density of the p-value distribution is estimated.

Before trusting either, plot hist(pvals, breaks = 50). A healthy result is a flat block near 1 with a spike near 0. A U-shape or a hump in the middle means your test is misspecified, and no correction will rescue it. A π₀ estimate near 1 with no spike means you have no signal, and that is a real answer.

The failure modes that bite in practice

Independent filtering is the first. DESeq2 removes low-count genes before adjustment to raise power, and those genes get padj = NA rather than a number. Genes flagged as Cook’s distance outliers also get NA. If you subset with padj < 0.05 and forget na.rm, your counts silently change. Check metadata(res)$filterThreshold to see where the cut landed.

Discreteness is the second. Permutation tests with only a few hundred shuffles produce a granular p-value grid, and BH then produces long runs of identical adjusted values. Ties at the threshold are an artifact of the resolution. They are not a biological plateau. Increase permutations or use an analytic test.

Correlation is the third. Benjamini-Hochberg is valid under independence and under positive regression dependence, which covers most co-expressed gene sets. Mass-spectrometry data violates this harder than transcriptomics does, because a single protein generates many correlated peptide and isotope features, so the effective number of independent tests is much smaller than the feature count.3 Collapse features to the protein or compound level before testing when you can. If you cannot, method = "BY" controls FDR under arbitrary dependence at the cost of a log(m) power penalty.

The fourth is the one that silently invalidates everything. Applying an FDR correction after selecting features by inspection, or after trying several normalizations and keeping the one with the most hits, gives you a number with no error-rate guarantee. Fix the pipeline first, then test.

What this means for one person’s longitudinal data

With a single subject, your replicates are timepoints, and the null you are testing is about within-person variation rather than between-person differences. That changes what a false discovery means. Some molecular features are stable within a person across months while others fluctuate widely, and multi-omics cohort data shows that this stability varies substantially by feature and by omics layer.4 A gene whose within-person coefficient of variation is high will produce large apparent fold-changes that are entirely null.

The practical consequence is to estimate that variability from your own repeated baseline draws before you test against it. Feature-level variance from your own series is a far better null than an assumption of independence across timepoints. Integrated multi-omics studies routinely combine layers under a shared FDR threshold to find signals that any single layer would miss.5 The same logic applies at n = 1. The caveat is that concordance across layers is evidence and not proof, because the layers are measured on the same biological sample and share technical sources of correlation.

Any result you plan to act on medically belongs in front of a physician, and clinically actionable variants need confirmation in a CLIA-certified laboratory. FDR governs a discovery list. It does not make a list clinical.

Choosing a threshold

We use BH at 0.05 as the default for transcriptomics and untargeted proteomics, tightened to 0.01 when the downstream cost of chasing a false lead is high. For candidate lists destined for targeted confirmation, we rank by q-value and add an effect-size floor. That floor is typically |log₂ fold change| > 0.5, because a statistically clean 3% change is rarely worth a confirmation assay. Formal frameworks exist for choosing how many targets to carry forward given an expected discovery rate.2

Report both numbers. Give the raw p-value, the adjusted value, the method, and the number of tests m. Any of those four missing makes the result unreproducible, and m in particular changes the adjusted values for every feature in the table.

Footnotes

  1. Daniela Dunkler, Fátima Sánchez-Cabo, Georg Heinze. Statistical Analysis Principles for Omics Data. Methods in Molecular Biology, 2011. https://doi.org/10.1007/978-1-61779-027-0_5 ↩

  2. Jeffrey T Leek, Margaret A Taub, Jason L Rasgon. A statistical approach to selecting and confirming validation targets in -omics experiments. BMC Bioinformatics, 2012. https://doi.org/10.1186/1471-2105-13-150 ↩ ↩2

  3. Pietro Franceschi, Marco Giordan, Ron Wehrens. Multiple comparisons in mass-spectrometry-based -omics technologies. TrAC Trends in Analytical Chemistry, 2013. https://doi.org/10.1016/j.trac.2013.04.011 ↩

  4. Marta Gallego-Paüls, Carles Hernández-Ferrer, Mariona Bustamante, et al. Variability of multi-omics profiles in a population-based child cohort. BMC Medicine, 2021. https://doi.org/10.1186/s12916-021-02027-z ↩

  5. Hongwei Zeng, Donge Tang, Yong Dai, et al. Integrated multi-omics profiling identifies aging-related molecular signatures and convergent interferon signaling in systemic lupus erythematosus. Frontiers in Immunology, 2026. https://doi.org/10.3389/fimmu.2026.1861904 ↩

Pothos builds longitudinal molecular profiles of individuals: whole-genome sequencing, single-cell RNA sequencing, and plasma proteomics, delivered as a detailed report, every data file, and stored cells. Build your profile.