Reproducible Pipelines

Source-checked on 2026-06-04. This page describes educational workflow patterns. It does not certify that Hack-Cancer maintains ready-to-run clinical or research-grade pipelines, and it should not be used as a clinical analysis protocol.

Cancer genomics pipelines are useful only when the biological question, cohort design, reference files, software versions, quality control thresholds, and validation strategy are all documented. Workflow managers such as Nextflow, Snakemake, and CWL can improve reproducibility, but they do not make an analysis scientifically valid by themselves.

How to Read This Page

• Treat each section as a template for what a pipeline must include, not as a published Hack-Cancer workflow.
• Prefer maintained community workflows, such as nf-core pipelines, when they fit the assay and research question.
• Pin exact versions for workflow code, containers, reference genomes, annotation sets, and statistical packages.
• Run small positive and negative controls before analyzing a real cohort.
• Keep clinical interpretation separate from computational output unless the workflow has been validated under the appropriate governance process.

Variant Calling Pattern

Variant calling can mean very different things depending on whether the study is germline, tumor-only, tumor-normal, targeted panel, whole exome, or whole genome.

Core Decisions

Decision	Why it matters
Tumor-only vs tumor-normal	Tumor-only calls have more uncertainty around germline filtering and clonality.
Panel vs exome vs genome	Capture design changes sensitivity, coverage, and false-negative behavior.
SNV/indel vs CNV/SV	Different variant classes need different callers and validation checks.
Reference genome and annotation	hg19/GRCh37 and GRCh38 coordinates are not interchangeable.

Typical Components

• Alignment or preprocessing appropriate to the assay.
• Duplicate marking, base quality handling, and sample-level QC.
• Variant calling with a documented caller and parameter set.
• Annotation with a versioned gene model and knowledgebase.
• Orthogonal or benchmark validation where the result will support a high-stakes claim.

For real work, start from maintained resources such as GATK Best Practices or nf-core/sarek rather than assuming the example commands in this site are validated.

RNA-seq Pattern

RNA-seq analysis should begin with the endpoint: expression quantification, differential expression, fusion detection, immune deconvolution, pathway analysis, or subtype classification. These are not interchangeable tasks.

Minimum Scientific Checks

• Confirm library type, read length, strandedness, and batch structure.
• Inspect mapping rates, duplication, insert size, rRNA content, and sample swaps.
• Separate exploratory plots from statistical tests.
• Use appropriate multiple-testing control for gene-level and pathway-level claims.
• Avoid interpreting fold changes without sample design, normalization, and variance estimates.

For production-quality research workflows, compare the planned analysis against maintained pipelines such as nf-core/rnaseq and against the statistical assumptions of tools such as DESeq2 or edgeR.

Single-cell Pattern

Single-cell RNA-seq pipelines are especially sensitive to cell capture chemistry, tissue dissociation, doublets, ambient RNA, batch effects, and cell-type annotation.

Common Failure Modes

• Treating clusters as biological cell types without marker review.
• Removing "low-quality" cells in a way that removes rare tumor or immune populations.
• Overcorrecting batch effects and erasing real biological signal.
• Inferring trajectories or ligand-receptor networks without independent support.
• Comparing proportions without accounting for sampling and dissociation bias.

Cell Ranger, Seurat, Scanpy, and nf-core/scrnaseq can be useful starting points, but study-specific QC and biological review remain mandatory.

Machine Learning Pipelines

Machine learning workflows in oncology need more than model training. They need a documented cohort, endpoint definition, leakage audit, calibration, external validation, and subgroup performance review.

Do not treat cross-validation on a small or synthetic dataset as clinical evidence. A model that performs well on the wrong split can still fail on a new hospital, scanner, sequencing platform, or population.

Cloud Execution

Cloud infrastructure can make pipelines easier to scale, but it adds requirements:

• Data access controls and audit logging.
• Region and storage decisions aligned with data governance.
• Reproducible container registries and workflow revisions.
• Cost monitoring for large sequencing or imaging workloads.
• Clear separation between educational examples and regulated data processing.

What to Document for Any Real Pipeline

• Research question and endpoint.
• Cohort inclusion and exclusion criteria.
• Sample identifiers, assay type, and data provenance.
• Workflow manager and exact workflow revision.
• Container image digests or environment lock files.
• Reference genome, annotation files, and database versions.
• QC thresholds and what happens when a sample fails QC.
• Statistical methods and multiple-testing correction.
• Validation dataset or benchmark truth set.
• Limitations and known failure modes.

Maintained Starting Points

Area	Public resource	Use carefully because
Germline and somatic variant calling	GATK Best Practices	The right workflow depends on assay design and sample pairing.
Tumor-normal and panel/WES/WGS workflows	nf-core/sarek	Parameters, references, and caller choices still need review.
Bulk RNA-seq	nf-core/rnaseq	Differential expression is downstream of the pipeline and depends on study design.
Single-cell RNA-seq	nf-core/scrnaseq, Cell Ranger, Seurat, Scanpy	Cell-type claims require biological validation, not just clustering.
Workflow execution	Nextflow, Snakemake, CWL	Orchestration improves reproducibility but does not validate the science.

References

1. GATK Best Practices Workflows. Broad Institute. https://gatk.broadinstitute.org/hc/en-us/sections/360007226651-Best-Practices-Workflows
2. nf-core/sarek. https://nf-co.re/sarek
3. nf-core/rnaseq. https://nf-co.re/rnaseq
4. nf-core/scrnaseq. https://nf-co.re/scrnaseq
5. Nextflow documentation. https://www.nextflow.io/docs/latest/
6. Snakemake documentation. https://snakemake.readthedocs.io/