{ "protocol": { "lockedAtUTC": "2026-10-13T17:11:59.629816+00:00", "candidate": "M4 v7: nonnegative unit-sum blend of absolute-error boosted trees, ExtraTrees and ridge. No intercept or test calibration.", "selection": "Same v6 52 features, weights, gene-disjoint inner split, seed and 12 configurations per component. Select each base model on equal-study inner validation MAE; optimize convex weights on the same inner validation predictions. Refit bases on training studies, apply frozen weights to held study.", "budget": "108 base tuning fits plus three convex optimizations. Ensemble combines all three families and has greater aggregate computation than a single baseline. Not a novel architecture claim.", "sourceHashes": { "docs/validation/2026-10-13-response-reconciliation-v4/m4-aligned.parquet": "08527f163c57d28fc248a78eb7f9c262dafed686300727a4257df2e90d2ff68b", "docs/validation/2026-10-13-public-only-benchmarks-v8/evaluated-predictions.csv": "a199a705ce31e8bb27e0f4e9c9fe207dbbac9251990cf94041cefd846b55b357" }, "families": [ "M4_v6", "ExtraTrees", "ridge" ], "configurations": { "M4_v6": [ { "max_depth": 2, "min_child_weight": 20, "reg_lambda": 10 }, { "max_depth": 2, "min_child_weight": 20, "reg_lambda": 100 }, { "max_depth": 2, "min_child_weight": 80, "reg_lambda": 10 }, { "max_depth": 2, "min_child_weight": 80, "reg_lambda": 100 }, { "max_depth": 4, "min_child_weight": 20, "reg_lambda": 10 }, { "max_depth": 4, "min_child_weight": 20, "reg_lambda": 100 }, { "max_depth": 4, "min_child_weight": 80, "reg_lambda": 10 }, { "max_depth": 4, "min_child_weight": 80, "reg_lambda": 100 }, { "max_depth": 6, "min_child_weight": 20, "reg_lambda": 10 }, { "max_depth": 6, "min_child_weight": 20, "reg_lambda": 100 }, { "max_depth": 6, "min_child_weight": 80, "reg_lambda": 10 }, { "max_depth": 6, "min_child_weight": 80, "reg_lambda": 100 } ], "ExtraTrees": [ { "min_samples_leaf": 2, "max_features": 0.5, "max_depth": 18 }, { "min_samples_leaf": 2, "max_features": 0.5, "max_depth": null }, { "min_samples_leaf": 2, "max_features": 1.0, "max_depth": 18 }, { "min_samples_leaf": 2, "max_features": 1.0, "max_depth": null }, { "min_samples_leaf": 5, "max_features": 0.5, "max_depth": 18 }, { "min_samples_leaf": 5, "max_features": 0.5, "max_depth": null }, { "min_samples_leaf": 5, "max_features": 1.0, "max_depth": 18 }, { "min_samples_leaf": 5, "max_features": 1.0, "max_depth": null }, { "min_samples_leaf": 10, "max_features": 0.5, "max_depth": 18 }, { "min_samples_leaf": 10, "max_features": 0.5, "max_depth": null }, { "min_samples_leaf": 10, "max_features": 1.0, "max_depth": 18 }, { "min_samples_leaf": 10, "max_features": 1.0, "max_depth": null } ], "ridge": [ { "alpha": 0.01 }, { "alpha": 0.1 }, { "alpha": 1.0 }, { "alpha": 10.0 }, { "alpha": 100.0 }, { "alpha": 1000.0 }, { "alpha": 10000.0 }, { "alpha": 100000.0 }, { "alpha": 1000000.0 }, { "alpha": 10000000.0 }, { "alpha": 100000000.0 }, { "alpha": 1000000000.0 } ] }, "rows": 29703, "genes": 11277, "freshHoldout": false, "primaryMetric": "MAE; secondary Spearman and R2", "finalFit": "Choose each component by mean inner MAE across folds; average the three inner-selected convex weight vectors; refit components on all development data.", "limitations": [ "All three studies are repeatedly used development data; no confirmatory superiority claim.", "Weights and component configurations share an inner validation set, creating selection optimism; outer outcomes do not select weights.", "Gene overlap across studies; inner validation genes are disjoint from inner training.", "The ensemble contains the ExtraTrees baseline itself; any gain is an incremental ensemble effect, not independent biological novelty.", "Different assays, perturbations and cell types remain. Gene bootstrap does not model laboratory/replicate uncertainty." ] }, "folds": [ { "study": "NMDRHT", "genes": 10990, "overlappingGenes": 10813, "weights": { "M4_v6": 0.8387065443600793, "ExtraTrees": 0.1612934556399207, "ridge": 0.0 }, "metrics": { "M4_v7": { "spearman": 0.5679820521477302, "mae": 0.4980065885071933, "r2": 0.12429365487825161 }, "M4_v6": { "spearman": 0.5449911919932277, "mae": 0.5047684174371222, "r2": 0.09955716303896855 }, "ExtraTrees": { "spearman": 0.5959547409063429, "mae": 0.4768180392190265, "r2": 0.2129883714588905 }, "ridge": { "spearman": 0.4785093673131193, "mae": 0.5128689886365398, "r2": 0.07413050161577561 }, "training_median": { "spearman": null, "mae": 0.5675473432855375, "r2": -0.10189963958838 } }, "comparisons": [ { "baseline": "ExtraTrees", "baselineMinusM4MAE": -0.02118854928816678, "adjustedPairedCi": [ -0.024332657561000174, -0.018402333979790405 ] }, { "baseline": "ridge", "baselineMinusM4MAE": 0.0148624001293465, "adjustedPairedCi": [ 0.01224432811219291, 0.017364891217738862 ] } ], "selected": { "M4_v6": 5, "ExtraTrees": 8, "ridge": 6 } }, { "study": "SMG1_HEK293", "genes": 11026, "overlappingGenes": 11026, "weights": { "M4_v6": 0.7544087266328959, "ExtraTrees": 0.24559127336710404, "ridge": 0.0 }, "metrics": { "M4_v7": { "spearman": 0.5163826388596757, "mae": 0.30448280888948, "r2": 0.05369730483566615 }, "M4_v6": { "spearman": 0.47822452371907853, "mae": 0.3152805587954828, "r2": -0.02752815878337067 }, "ExtraTrees": { "spearman": 0.5799844713366191, "mae": 0.29401334552029457, "r2": 0.16520040797102054 }, "ridge": { "spearman": 0.41404510466266237, "mae": 0.32589989456892, "r2": -0.07730212523684488 }, "training_median": { "spearman": null, "mae": 0.30850374408515713, "r2": -0.0719493901606727 } }, "comparisons": [ { "baseline": "ExtraTrees", "baselineMinusM4MAE": -0.0104694633691855, "adjustedPairedCi": [ -0.013474677248514106, -0.007251764442265149 ] }, { "baseline": "ridge", "baselineMinusM4MAE": 0.02141708567943998, "adjustedPairedCi": [ 0.018078886499321375, 0.024933961749467017 ] } ], "selected": { "M4_v6": 5, "ExtraTrees": 8, "ridge": 5 } }, { "study": "Tani2012", "genes": 7687, "overlappingGenes": 7643, "weights": { "M4_v6": 0.8255673767252742, "ExtraTrees": 0.1544336129635905, "ridge": 0.01999901031113524 }, "metrics": { "M4_v7": { "spearman": 0.4669791674181943, "mae": 0.6166839088958349, "r2": 0.03549723129597415 }, "M4_v6": { "spearman": 0.4535900512104125, "mae": 0.6162641094906888, "r2": 0.03260251909473166 }, "ExtraTrees": { "spearman": 0.4963067837862868, "mae": 0.6235053269034547, "r2": 0.03609311693973649 }, "ridge": { "spearman": 0.3939429105211656, "mae": 0.6648507723026432, "r2": -0.05557351723207127 }, "training_median": { "spearman": null, "mae": 0.6450195364304797, "r2": -0.07579048338070993 } }, "comparisons": [ { "baseline": "ExtraTrees", "baselineMinusM4MAE": 0.006821418007620088, "adjustedPairedCi": [ 0.0033214625493272233, 0.010307818959245948 ] }, { "baseline": "ridge", "baselineMinusM4MAE": 0.048166863406808515, "adjustedPairedCi": [ 0.04410625433502463, 0.05223281010837128 ] } ], "selected": { "M4_v6": 10, "ExtraTrees": 9, "ridge": 4 } } ], "finalModel": { "path": "backend/models/m4-decay-response-v7/M4_v7.joblib", "sha256": "2517178177d8784206d9acf6f9bce5b86213adc0e46c62e9032708ed1c34323a", "weights": { "M4_v6": 0.8062275492394164, "ExtraTrees": 0.18710611399020508, "ridge": 0.006666336770378413 }, "componentSelection": { "M4_v6": { "configurationId": 5 }, "ExtraTrees": { "configurationId": 8 }, "ridge": { "configurationId": 4 } } }, "independentSuperiority": false, "search": [ { "name": "GSE56748 / GSE56751 \u00b7 MOV10", "url": "https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE56748", "decision": "Six libraries are total, labeled and flow-through RNA under mock versus MOV10 depletion. The series mentions UPF1 binding, but measured decay perturbation is MOV10, not UPF1/SMG1. Excluded from the requested NMD holdout; numerical half-life table unopened." }, { "name": "Alternative UPF1 isoform study", "url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC9108617/", "decision": "Genome-wide stability changes are inferred from RNA-seq with REMBRANDTS; direct decay measurements are selected-gene qPCR. Not a broad measured-decay M4 test or matched RNA/RPF M5 test." }, { "name": "GSE143301 / GSM4407916 \u00b7 ABCE1", "url": "https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM4407916", "decision": "Rechecked sample list: UPF1 samples are total RNA only. Shared Ribo-seq extraction text does not establish UPF1 RPF libraries. No qualifying matched UPF1 endpoint." }, { "name": "GSE303542 / GSM9130088", "url": "https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM9130088", "decision": "UPF1 sample is Neurospora crassa, despite human Ribo-seq text in shared processing metadata. It does not supply a fresh human UPF1 RNA/RPF cohort." }, { "name": "GSE186795 \u00b7 ribosome-bound UPF1", "url": "https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE186795", "decision": "Ribosome-associated UPF1 investigation in Saccharomyces cerevisiae. Wrong species for the requested human validation." }, { "name": "MDC SLAM-seq catalogue", "url": "https://edoc.mdc-berlin.de/id/eprint/25702/", "decision": "Boehm/Landthaler HCT116 AID-UPF1, 0/12/24h: the already-used NMDRHT study. A catalogue mirror is not independent data." } ], "scope": "Public GEO sample metadata, primary literature and repository searches including EBI, Figshare and Zenodo-linked results. Targeted review, not an exhaustive absence claim. No author contact.", "overallMAE": { "M4_v7": 0.47305776876416944, "M4_v6": 0.47877102857443127, "ExtraTrees": 0.4647789038809253, "ridge": 0.5012065518360344 }, "winsAgainstExtraTrees": 1, "improvedAgainstV6": 2, "newFreshStudyQualified": false }