AI and Machine Learning in Oncology

Under scientific review

This page discusses software algorithms, foundation models, and regulated clinical decision support. Use it for educational and orientation purposes, not for medical diagnosis or treatment selection.

For software engineers and data scientists entering oncology, Artificial Intelligence and Machine Learning (AI/ML) represent powerful tools for pattern recognition across gigabytes of multimodal data — digital pathology gigapixel slides, 3D radiologic volumes, and high-dimensional omics matrices.

However, clinical AI is fundamentally different from consumer AI: a high AUC on a retrospective dataset does not imply clinical utility. Flaws in data splits, shortcut learning, or distribution shifts can turn an apparently accurate model into a dangerous clinical liability.

---

1. Primary Workflows & Architectures in Oncology

A. Digital Pathology & Computational Histology

Digital pathology turns glass biopsy slides into Whole Slide Images (WSIs) — gigapixel images typically $100,000 \times 100,000$ pixels at $40\times$ magnification.

• The Challenge: A single slide exceeds GPU VRAM limits.
• The Solution (Multiple Instance Learning - MIL): WSIs are broken into thousands of $256 \times 256$ pixel patches. A pre-trained feature extractor (e.g., ResNet, ViT, or specialized Pathology Foundation Models like CONCH or Prov-GigaPath) embeds each patch into a low-dimensional vector. An aggregation network (Attention-based MIL) predicts patient-level labels (e.g., gene mutation status, microsatellite instability, or survival).

B. Radiology & 3D Volumetric Imaging

• Segmentation: Identifying tumor boundaries on CT/MRI scans using 3D U-Net or Medical Vision Transformers (nnU-Net).
• Computer-Aided Triage: Flagging urgent incidental findings (e.g., pulmonary nodules) in radiology queues.

C. Foundation Models & Multimodal AI

Recent architectures (e.g., BioGPT, Med-PaLM, multimodal Transformers) align text (EHR notes, pathology reports), images (radiology/histology), and molecular data (genomics) into shared latent spaces, enabling cross-modal retrieval and automated diagnostic drafting.

---

2. The Clinical Evaluation Standard

To evaluate whether an oncology ML model is production-ready, engineers must enforce six strict validation criteria:

1. Patient-Level Data Splitting: Never split patches, slices, or longitudinal follow-up visits from the same patient across train and test sets. Doing so causes severe Data Leakage, resulting in artificial 0.99 AUCs that collapse in real-world deployment.
2. External & Out-of-Distribution (OOD) Validation: Test the model on independent data from different hospitals, scanners (e.g., GE vs. Siemens), staining protocols, and diverse patient demographics.
3. Calibration Over Discrimination: AUC only measures ranking order. Clinical decision-making requires Probability Calibration (Platt scaling or isotonic regression) so that a predicted risk of $30%$ corresponds to 30 actual events out of 100 patients.
4. Subgroup & Equity Auditing: Evaluate performance across sex, age, ancestry, and disease subtypes to prevent bias propagation.
5. Comparison Against Clinical Baselines: The model must outperform standard clinical risk scores (e.g., TNM staging or NCCN guidelines), not just a random baseline.
6. Prospective Clinical Trials: True clinical utility is proven only when using the AI model in a prospective trial improves patient outcomes or workflow efficiency.

---

3. Key Failure Modes & Technical Pitfalls

Failure Mode	Mechanism	Prevention / Mitigation
Shortcut Learning (Site Leakage)	Model learns scanner noise, hospital watermarks, or surgical pen marks instead of tumor biology	Intensity normalization, color normalization (Macenko method), stain jittering, OOD testing
Label Leakage	Target variable includes post-baseline information (e.g., predicting survival using post-treatment imaging)	Strict temporal scoping of input features
Class Imbalance Distortion	Rare cancer subtypes (VAF < 1% or rare tumors) are ignored by standard cross-entropy loss	Focal loss, balanced sampling, precision-recall AUC (PR-AUC)
Covariate Shift / Drift	Hospital updates scanner software or treatment protocols, invalidating model weights	Continuous monitoring, drift detection (KS-tests on embeddings), versioned deployments

---

4. Regulatory Frameworks (FDA, EMA, ANVISA)

Deploying oncology AI requires navigating Software as a Medical Device (SaMD) regulations:

• FDA (United States): Regulates AI/ML devices via 510(k), De Novo, or PMA pathways. Released guidance on Predetermined Change Control Plans (PCCPs), allowing pre-approved model retraining algorithms without requiring a new marketing submission for every weight update.
• EMA & AI Act (European Union): Classifies diagnostic/prognostic medical AI as high-risk under the EU AI Act, mandating strict data governance, risk management, and technical auditability.
• ANVISA (Brazil): Regulated under RDC 657/2022 and RDC 751/2022 for SaMD, alongside LGPD compliance for patient health data.

---

References

1. Campanella, G., et al. (2019). Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine, 25(8), 1301-1309.
2. Collins, G. S., et al. (2024). TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 385, e078378.
3. Xu, H., et al. (2024). A whole-slide foundation model for digital pathology from real-world data. Nature, 630(7900), 181-188.
4. U.S. FDA (2025). Marketing Submission Recommendations for a Predetermined Change Control Plan for AI-Enabled Device Software Functions.