Skip to content

How to Cluster Your Own RNA-seq Data and Interpret the Result

Oak
A feathered reptilian creature on a dark plinth, its plumage grouped into color-matched rosettes that glow faintly, head turned away.

By the end of this guide you will have two artifacts: a sample-by-sample distance structure that tells you how your own timepoints relate to each other, and a set of gene clusters whose members move together across those timepoints. You need a counts matrix from at least six RNA-seq samples (more if they span conditions or timepoints), R 4.3 or later with DESeq2, DEGreport, WGCNA, and clusterProfiler installed, and a metadata table with one row per sample. If you only have raw FASTQ files, step 1 covers quantification. Everything here is bulk RNA-seq from whole blood or another single tissue, which is the usual case for a personal profile. Single-cell data has different noise structure and I flag the differences where they matter.

1. Get a counts matrix you trust

Clustering amplifies quantification artifacts, so the upstream step deserves more care than it usually gets. For bulk RNA-seq I would use selective alignment with salmon against a decoy-aware transcriptome index rather than genome alignment with STAR, because it is roughly ten times faster and the transcript-level estimates are what you want for gene-level aggregation.

# Build once against GENCODE v45 transcripts + genome decoys
salmon index -t gentrome.fa.gz -d decoys.txt -i salmon_idx_v45 -k 31 -p 16

# Quantify each sample
salmon quant -i salmon_idx_v45 -l A \
  -1 sample01_R1.fastq.gz -2 sample01_R2.fastq.gz \
  --validateMappings --gcBias --seqBias --posBias \
  --numBootstraps 30 -p 16 -o quants/sample01

Turn on --gcBias unless you have a reason not to. Library GC bias varies between runs, and if your samples were prepped in two batches the uncorrected bias will show up as your first principal component. Check the mapping rate in quants/*/logs/salmon_quant.log and treat anything below 70 percent as a sample to investigate rather than cluster.

On read depth: for gene-level clustering and differential expression in bulk tissue, 20 to 30 million paired-end reads per sample is the standard target, and 10 million is usually enough if you only care about moderately and highly expressed genes. Depth matters more than read length for expression quantification. If you plan to look at isoform switching or lowly expressed transcription factors, go deeper. For a longitudinal design, equal depth across timepoints matters more than high depth, because depth differences translate directly into detection-rate differences that clustering will read as biology.

Aggregate to gene level with tximport, which also computes the average transcript length offsets that DESeq2 uses.

library(tximport); library(DESeq2)
files <- file.path("quants", meta$sample, "quant.sf")
names(files) <- meta$sample
txi <- tximport(files, type = "salmon", tx2gene = tx2gene)
dds <- DESeqDataSetFromTximport(txi, colData = meta, design = ~ timepoint)

2. Filter low-expression genes before you do anything else

Most of the 60,000 rows in an unfiltered gene matrix are noise, and noise dominates Euclidean distance. Filtering is the single most influential decision in the whole pipeline, and it is worth being systematic about it rather than copying a threshold from a tutorial.

I use a presence filter on counts per million rather than raw counts, so that depth differences do not decide which genes survive.

library(edgeR)
cpm_mat <- cpm(counts(dds))
# keep genes at >=1 CPM in at least the size of your smallest group
keep <- rowSums(cpm_mat >= 1) >= 3
dds <- dds[keep, ]
nrow(dds)   # expect ~12,000-16,000 for whole blood

Whole blood is a special case. Globin transcripts such as HBB, HBA1, and HBA2 can consume 50 to 70 percent of reads in non-globin-depleted libraries. That compresses the dynamic range for everything else. If you see this, remove the globin genes after quantification and before filtering, then note that your effective depth was lower than the nominal figure.

Published pipelines formalize these choices so that results are reproducible across labs. The R-ODAF framework, built for regulatory toxicogenomics, specifies filtering and normalization steps explicitly for exactly this reason. It is a reasonable template if you want your personal analysis to be repeatable a year from now.1

3. Transform the counts before computing any distance

Raw or CPM-normalized counts have variance that grows with the mean. If you feed them to a Euclidean distance, high-expression genes dominate the geometry and your clusters mostly reflect expression level rather than co-regulation. The fix is a variance-stabilizing transformation.

vsd <- vst(dds, blind = TRUE)        # blind=TRUE: ignore the design
mat <- assay(vsd)                    # log2-like scale, homoskedastic

Use blind = TRUE for exploratory clustering. It stops the design formula from influencing the dispersion estimates, which keeps the clustering an independent check on your assumptions. Use rlog instead of vst only if you have fewer than about 30 samples and highly variable library sizes. It is slower but shrinks low-count genes more aggressively.

For gene-level clustering you then need one more step: center and scale each gene to z-scores. Without this, clustering groups genes by mean expression. With it, clustering groups genes by shape of trajectory, which is what you want.

z <- t(scale(t(mat)))   # rows = genes, mean 0, sd 1 per gene

4. Cluster samples before you cluster genes

Sample clustering is quality control that happens to be informative. Run it first, because a single outlier sample will reshape every gene cluster downstream.

# PCA on the 2000 most variable genes
rv <- rowVars(mat)
sel <- order(rv, decreasing = TRUE)[1:2000]
pca <- prcomp(t(mat[sel, ]), scale. = FALSE)
summary(pca)$importance[, 1:4]

# Sample-sample correlation, Spearman on all filtered genes
cors <- cor(mat, method = "spearman")
library(pheatmap)
pheatmap(cors, annotation_col = meta[, c("timepoint", "batch")])

What you are looking for is whether the dominant structure lines up with something biological or with something technical. In personal longitudinal data the most common finding is that PC1 tracks library prep batch or RNA integrity number. It does not track time. Spearman correlations between technical replicates of whole blood typically sit above 0.97 on filtered, transformed data, so a pair below 0.90 that should be similar points at a sample quality problem.

Two confounders deserve a specific check. Blood cell composition drives a large share of whole-blood expression variance, so a shift in neutrophil fraction between draws will look like a global transcriptional change. Sex-linked expression is the other one, and it is relevant even in a single-person profile because X-inactivation escape and sex-hormone-responsive genes behave differently from autosomal genes. Tools built to test for sex-dependent effects in omics data make this explicit as an analysis step rather than a covariate to regress away.2

5. Choose a distance metric on purpose

You will encounter four families of clustering. Hierarchical methods are agglomerative or divisive. Partitioning methods include k-means and k-medoids. Model-based methods use finite mixtures, including negative binomial mixtures fit directly to counts. Graph or density based methods include Leiden, Louvain, and DBSCAN. Each one is a distance metric plus a rule for combining points. The metric usually matters more than the rule.

My defaults:

  • Samples: Spearman correlation distance, as.dist(1 - cor(mat, method = "spearman")). Rank-based, so it tolerates a few extreme genes.
  • Gene trajectories: Pearson correlation distance on z-scored data. You care about shape, and Pearson is shape.
  • Hierarchical linkage: ward.D2, which produces compact clusters of comparable size. Average linkage is the alternative and tends to produce one giant cluster plus singletons.

Systematic comparisons of similarity measures in spectral clustering on omics data show the metric choice changes cluster assignments substantially, which is a good argument for testing two or three rather than trusting one.3 Run your analysis with both Spearman and Euclidean and see whether the conclusion survives. If it does not, you have learned something about how fragile the structure is.

d_samples <- as.dist(1 - cor(mat, method = "spearman"))
hc <- hclust(d_samples, method = "ward.D2")
plot(hc, labels = meta$sample)

6. Cluster genes into trajectory patterns

For a longitudinal personal profile, the useful question is which genes move together over time. The cleanest route is to restrict to genes with evidence of change, then cluster only those. Clustering all 14,000 filtered genes gives you clusters dominated by genes that do nothing.

# Genes changing across timepoints (likelihood ratio test)
dds_lrt <- DESeq(dds, test = "LRT", reduced = ~ 1)
res <- results(dds_lrt)
sig <- rownames(res)[which(res$padj < 0.05)]
length(sig)

library(DEGreport)
pat <- degPatterns(mat[sig, ], metadata = as.data.frame(colData(dds)),
                   time = "timepoint", minc = 15)
head(pat$df)      # gene -> cluster assignment

degPatterns computes pairwise correlation distance, cuts the dendrogram, and merges clusters whose mean profiles are more than a threshold similar. The minc = 15 argument drops clusters with fewer than 15 genes, which is a sensible floor because small clusters are rarely interpretable and rarely reproducible.

Time-course RNA-seq has a literature of its own, and the methods differ from case-versus-control analysis in ways that matter. Sampling density changes the answer, as does the choice between clustering on fitted splines versus raw values and the handling of autocorrelation between adjacent timepoints.4 If your timepoints are irregularly spaced, fit a smooth trajectory per gene before clustering rather than treating the observations as evenly spaced.

The alternative for larger sample counts is WGCNA, which builds a signed co-expression network and calls modules on the topological overlap matrix. It needs roughly 15 samples minimum and works better with 30 or more.

library(WGCNA)
datExpr <- t(mat)
sft <- pickSoftThreshold(datExpr, powerVector = 1:20, networkType = "signed")
net <- blockwiseModules(datExpr, power = sft$powerEstimate,
                        networkType = "signed", TOMType = "signed",
                        minModuleSize = 30, mergeCutHeight = 0.25,
                        numericLabels = TRUE, verbose = 3)
table(net$colors)
MEs <- net$MEs   # module eigengenes: one value per module per sample

Module eigengenes are the payoff. Each is the first principal component of a module, so you get a handful of numbers per sample instead of 14,000. Correlating those against your biomarker panel or glucose summary statistics is a far better-powered test than 14,000 individual gene correlations.

7. Pick k and then check whether it means anything

Do not choose the number of clusters by eye. Use two criteria and prefer the smaller k when they disagree.

library(cluster)
sil <- sapply(2:12, function(k) {
  cl <- cutree(hclust(d_genes, method = "ward.D2"), k = k)
  mean(silhouette(cl, d_genes)[, "sil_width"])
})
plot(2:12, sil, type = "b", xlab = "k", ylab = "mean silhouette")

Then test stability by resampling. fpc::clusterboot recomputes the clustering on bootstrap samples and reports the Jaccard similarity of each cluster to its closest match.

library(fpc)
cb <- clusterboot(d_genes, B = 100, distances = TRUE,
                  clustermethod = disthclustCBI, k = 6, method = "ward.D2")
cb$bootmean    # >0.75 = stable, <0.6 = do not interpret

Treat a mean Jaccard below 0.6 as a cluster you should not build a story on. Most analyses skip this step, and it is the reason two people can cluster the same dataset and report different module counts. On personal data with ten or fifteen samples, expect that only two or three clusters clear the stability bar.

8. Annotate the clusters, then read them conservatively

A cluster is a list of gene IDs until you attach biology to it. Run enrichment per cluster against the filtered gene set as the background, never against the whole genome, because your filter already removed the genes that are not expressed in your tissue.

library(clusterProfiler); library(org.Hs.eg.db)
universe <- rownames(mat)
for (k in unique(pat$df$cluster)) {
  genes <- pat$df$genes[pat$df$cluster == k]
  ego <- enrichGO(genes, OrgDb = org.Hs.eg.db, keyType = "SYMBOL",
                  ont = "BP", universe = universe,
                  pAdjustMethod = "BH", qvalueCutoff = 0.05)
  print(head(as.data.frame(ego)[, c("Description", "qvalue", "Count")], 8))
}

Interpretation runs in one direction only. A cluster enriched for interferon-stimulated genes that rises at one timepoint tells you that an interferon-responsive program was more active in that blood draw. It does not tell you why, and it does not identify a condition. Blood transcriptomes respond to a recent viral exposure, a poor night of sleep, and a hard training session. They also respond to time of day. Without a repeated design you cannot separate those. If a pattern looks clinically meaningful to you, that is a conversation for a physician who can order the appropriate confirmatory testing. Expression clusters cannot resolve it.

Two structural cautions. First, cluster membership is not causal direction. Co-expression is consistent with shared upstream regulation, shared cell-type-of-origin, or coincidence. Second, clustering is sensitive to everything upstream, so report your filter threshold, transformation, metric, linkage, and k alongside any result you intend to revisit.

9. Cross-check clusters against your other data layers

The strongest validation available to a personal profile is agreement across measurement types. If an expression module tracks a protein in your proteomics panel, or if a cluster’s eigengene correlates with a blood marker measured on the same day, the module is more likely to reflect real biology than pipeline artifact.

# Correlate module eigengenes with biomarker panel
cor_mat <- cor(MEs, biomarkers, use = "pairwise.complete.obs",
               method = "spearman")
p_mat <- corPvalueStudent(cor_mat, nSamples = nrow(MEs))

Multi-omic clustering studies find that patients whose subtype assignments agree across data types have more consistent outcomes than those whose assignments conflict, which is a general argument for treating cross-layer agreement as the evidence standard.5 The tradeoffs are real too: adding layers adds batch effects, missing data, and multiple-testing burden, and integration can dilute a strong single-layer signal.6 With a dozen samples, correct your correlations for multiple testing and expect to be underpowered for anything but large effects.

If you are working with single-cell rather than bulk data, the clustering methods diverge. Zero inflation and dropout break correlation-based distances, and current approaches learn a low-dimensional embedding before clustering. Model-based deep embedding with soft constraints from known marker genes performs better than raw-space clustering on scRNA-seq.7 Self-supervised contrastive learning is the other line of work, and it handles integration across batches within the same framework.8

Common problems

PC1 reflects batch. Biology is not what it shows. Check meta$batch against PC1 loadings first. If they align, either regress the batch out with limma::removeBatchEffect on the VST matrix for visualization only, or include batch in the design and cluster on residuals. Never regress out a variable that is collinear with your variable of interest.

Every gene lands in one giant cluster. Almost always a missing z-score step, or average linkage instead of Ward. Confirm that rowMeans(z) is approximately zero and rowSds(z) is approximately one.

Clusters change completely when you change k by one. The structure is weak. Run clusterboot, and if Jaccard values sit below 0.6, report the ordination (PCA or UMAP) instead of discrete clusters.

Enrichment returns only ribosome and translation terms. Usually a 3’ bias or degradation signal. Check the transcript integrity number or the coverage uniformity in your salmon or RSeQC output before interpreting.

A single timepoint drives a whole module. Fit the module eigengene with that sample dropped. If the module dissolves, it was one sample. It was not a pattern.

Gene symbols do not map. Strip Ensembl version suffixes with sub("\\..*$", "", ids) before annotation, and expect 5 to 10 percent of IDs to fail mapping regardless.

Footnotes

  1. M.C. Verheijen, T.W. Gant, W. Tong, et al. R-ODAF: Omics data analysis framework for regulatory application. Toxicology Letters, 2021. https://doi.org/10.1016/s0378-4274(21)00539-7 ↩

  2. Sophie Le Bars, Mohamed Soudy, Enrico Glaab. XYomics: detecting sex-dependent molecular mechanisms in omics data. Nucleic Acids Research, 2026. https://doi.org/10.1093/nar/gkag759 ↩

  3. Tianyi Shi, Xiucai Ye, Zeng Zou, et al. PySimi: a unified framework for similarity measure evaluation in spectral clustering with applications to omics data. Frontiers in Genetics, 2026. https://doi.org/10.3389/fgene.2026.1913487 ↩

  4. Daniel Spies, Constance Ciaudo. Dynamics in Transcriptomics: Advancements in RNA-seq Time Course and Downstream Analysis. Computational and Structural Biotechnology Journal, 2015. https://doi.org/10.1016/j.csbj.2015.08.004 ↩

  5. Shuting Lin, Peng Qiu. Clustering Analysis of Multiple Omics Data Types Identifies Cancer Patients With Consistent Survival Outcomes. Cancer Informatics, 2025. https://doi.org/10.1177/11769351251394107 ↩

  6. C. Nelson Hayes, Hikaru Nakahara, Atsushi Ono, et al. From Omics to Multi-Omics: A Review of Advantages and Tradeoffs. Genes, 2024. https://doi.org/10.3390/genes15121551 ↩

  7. Tian Tian, Jie Zhang, Xiang Lin, et al. Model-based deep embedding for constrained clustering analysis of single cell RNA-seq data. Nature Communications, 2021. https://doi.org/10.1038/s41467-021-22008-3 ↩

  8. Wenkai Han, Yuqi Cheng, Jiayang Chen, et al. Self-supervised contrastive learning for integrative single cell RNA-seq data analysis. Briefings in Bioinformatics, 2022. https://doi.org/10.1093/bib/bbac377 ↩

Pothos builds longitudinal molecular profiles of individuals: whole-genome sequencing, single-cell RNA sequencing, and plasma proteomics, delivered as a detailed report, every data file, and stored cells. Build your profile.