Recorded 2026-10-13. Accuracy, prediction runtime and artifact readiness are measured separately. Comparisons apply to their stated endpoint and cohort.
M3 v3 · multi-study development and fresh SCN5A test
M3 improves over v2, but still loses to the strongest ExtraTrees and Pangolin calibrators.
On the fresh iPSC-CM cohort, M3 Brier error falls from 0.15405 to 0.10776. Two-score ExtraTrees reaches 0.08381; uncalibrated SpliceAI has the lowest error, 0.06130. No clear superiority or serving-model promotion.
The earlier MFASS and four external minigene cohorts are now development data: 8,031 variants, five whole-study folds and 1,080 tuning fits. The new revision combines the two neural scores with variant location and substitution features. Pangolin remains an explicit model dependency.
Exploratory qualification only. Fresh public ParSE-seq data provide 176 eligible iPSC-CM variants (primary) and 190 HEK variants (secondary). They share one gene, SCN5A, and overlapping variants. The deposited Pangolin aggregation method is incomplete, so this is not a matched-protocol confirmation or evidence of superiority across independent genes.
Brier error ↓ · iPSC-CM is the locked primary endpoint; HEK is related secondary evidence
Model
iPSC-CM ↓
HEK ↓
M3 v3 · scores + variant context
0.10776
0.09419
ExtraTrees · same scores + context
0.08427
0.07465
ExtraTrees · two scores only
0.08381
0.06968
Pangolin + context · boosted trees
0.09762
0.08311
Pangolin · retuned spline
0.09038
0.07815
Pangolin · retuned Platt
0.08974
0.07651
Pangolin · retuned isotonic
0.09098
0.07838
Ridge · scores + context
0.12177
0.11722
M3 v2 · frozen
0.15405
0.16227
ExtraTrees joint v2 · frozen
0.14311
0.15400
ExtraTrees SpliceAI v2 · frozen
0.10298
0.11663
Pangolin spline v2 · frozen
0.14563
0.15220
Pangolin Platt v2 · frozen
0.17691
0.18621
Pangolin isotonic v2 · frozen
0.15564
0.16392
M3 v1 · frozen
0.11867
0.13313
ExtraTrees v1 · frozen
0.10904
0.12365
Pangolin Platt v1 · frozen
0.18003
0.18968
Pangolin isotonic v1 · frozen
0.16423
0.17449
Published Pangolin score · uncalibrated
0.16257
0.16958
SpliceAI max score · uncalibrated
0.06130
0.06173
Mean development-study prevalence
0.25819
0.27030
Retuned and earlier frozen baselines are all retained. The same variants are scored by every model within each assay. The strongest baseline is not replaced with a weaker retuning to create a win.
Each fold held out an entire assay cohort. Model selection minimized mean study Brier, with worst-study error breaking ties. Every model received the same study-weighting grid. ExtraTrees received the same features as M3, and a Pangolin-only model with context tested whether SpliceAI added useful information.
Location features distinguish near-junction exon, deep exon, essential splice, near-junction intron and proximal intron variants. Additional features encode transition status and reference/alternate GC membership. No gene identity, assay identity, clinical label or measured response was used as an input.
Development selection results only · these studies no longer provide independent validation
Model
Mean study Brier ↓
Worst study Brier ↓
M3 candidate · boosted trees
0.18452
0.45101
M3 candidate · spline (selected)
0.18256
0.42121
ExtraTrees · same scores + context
0.18376
0.45937
ExtraTrees · two scores only
0.18579
0.42493
Pangolin + context · boosted trees
0.17211
0.39219
Pangolin · retuned spline
0.16993
0.38135
Pangolin · retuned Platt
0.17079
0.39337
Pangolin · retuned isotonic
0.17074
0.38409
Ridge · scores + context
0.20438
0.38428
The selected M3 spline was the better M3 candidate, but did not outperform every baseline during development either. Both that limitation and the fresh-test losses are retained. No model was changed after the fresh results were computed.
Development study rows: MFASS 5,801; POU1F1 941; MST1R 598; WT1 502; FAS 189. SCN5A was reserved. The original MFASS v1 test partition remains outside training.
Fresh-data provenance and exclusions
Published Supplementary Data 2 supplies variant coordinates and predictor scores; Supplementary Data 3 supplies experimental Outcome labels. Clinical classifications are not targets. SpliceAI max is reconstructed from AG, AL, DG and DL; its aggregate score is not substituted. The published Pangolin column is used unchanged.
Author analysis code refers to Pangolin_Aggregate, but the deposited code does not show how that aggregate was constructed. Neural inference was not rerun. This unresolved contract difference prevents a definitive calibration-transfer claim.
From 468 assay rows, 366 were eligible across both cell types, representing 224 unique variants. All eligible reference alleles matched hg38. No exact variant overlap with the full historical qualified MFASS cohort or SCN5A gene overlap with v3 training was found.
non SNV: 18 assay rows
indeterminate or missing assay: 87 assay rows
missing or invalid predictor: 6 assay rows
missing location: 0 assay rows
missing coordinate: 0 assay rows
reference mismatch or nonSNV: 13 assay rows
historical MFASS variant overlap: 0 assay rows
Exclusion counts overlap; they must not be added. Only complete common-input SNVs with normal/abnormal assay labels were tested. Indeterminate outcomes remain excluded, not relabelled benign.
The primary rule required M3 to beat every baseline with an adjusted interval wholly above zero. It failed. Positive baseline-minus-M3 Brier differences favor M3. These exploratory intervals use 10,000 paired whole-exon resamples across 19 SCN5A exons, with 99.75% intervals adjusting for 20 comparisons.
Exon resampling describes uncertainty within this selected gene. It does not establish performance across genes, resolve predictor provenance, or make the two cell types independent.
ExtraTrees · same scores + context: -0.023487 [-0.044142, -0.007075].
ExtraTrees · two scores only: -0.023948 [-0.050201, -0.005185].
Published Pangolin score · uncalibrated: 0.054813 [-0.019852, 0.134514].
SpliceAI max score · uncalibrated: -0.046457 [-0.095492, -0.004855].
Mean development-study prevalence: 0.150430 [0.070557, 0.223507].
Limitations, secondary metrics and model status
Previously tested cohorts now development, not independent validation.
Development assay labels have heterogeneous biological definitions.
Fresh SCN5A selection includes clinically ascertained variants and variants chosen using SpliceAI; not a population sample.
Pangolin score aggregation is incompletely specified in deposited code.
Pretrained SpliceAI/Pangolin training overlap with SCN5A is not ruled out.
Research model requires Pangolin; not a standalone sequence model or clinical/patent validation.
Public article summaries and five example source rows were inspected while qualifying columns; fresh performance metrics did not guide model selection. Model and prediction hashes were locked before numerical evaluation. The complete protocol and audit are downloadable.
Primary iPSC-CM: 176 variants, 75 abnormal. Secondary HEK: 190 variants, 87 abnormal. Raw neural scores are not calibrated clinical probabilities.
M3 v3 · scores + variant context: log loss 0.35745, AUROC 0.91228, average precision 0.85195, ECE 0.05421.
ExtraTrees · same scores + context: log loss 0.30499, AUROC 0.94673, average precision 0.90430, ECE 0.08013.
ExtraTrees · two scores only: log loss 0.30326, AUROC 0.95479, average precision 0.91818, ECE 0.12002.
Pangolin + context · boosted trees: log loss 0.32963, AUROC 0.93386, average precision 0.89263, ECE 0.08300.
Pangolin · retuned spline: log loss 0.31365, AUROC 0.94191, average precision 0.90598, ECE 0.04481.
Pangolin · retuned Platt: log loss 0.34824, AUROC 0.94191, average precision 0.90598, ECE 0.06648.
Pangolin · retuned isotonic: log loss 0.32272, AUROC 0.94033, average precision 0.88478, ECE 0.06117.
Ridge · scores + context: log loss 0.44431, AUROC 0.95036, average precision 0.93543, ECE 0.15575.
M3 v2 · frozen: log loss 0.45430, AUROC 0.94449, average precision 0.90418, ECE 0.19250.
ExtraTrees joint v2 · frozen: log loss 0.42327, AUROC 0.94581, average precision 0.91199, ECE 0.18645.
ExtraTrees SpliceAI v2 · frozen: log loss 0.33731, AUROC 0.96152, average precision 0.93873, ECE 0.16119.
Pangolin spline v2 · frozen: log loss 0.43359, AUROC 0.94191, average precision 0.90598, ECE 0.18078.
Pangolin Platt v2 · frozen: log loss 0.53718, AUROC 0.94191, average precision 0.90598, ECE 0.22001.
Pangolin isotonic v2 · frozen: log loss 0.46093, AUROC 0.93875, average precision 0.88369, ECE 0.19176.
M3 v1 · frozen: log loss 0.37327, AUROC 0.96073, average precision 0.93664, ECE 0.16946.
ExtraTrees v1 · frozen: log loss 0.34971, AUROC 0.96442, average precision 0.94937, ECE 0.16090.
Pangolin Platt v1 · frozen: log loss 0.54403, AUROC 0.94191, average precision 0.90598, ECE 0.22385.
Pangolin isotonic v1 · frozen: log loss 0.47912, AUROC 0.93644, average precision 0.88370, ECE 0.20412.
Published Pangolin score · uncalibrated: log loss 0.52277, AUROC 0.94191, average precision 0.90598, ECE 0.20243.
SpliceAI max score · uncalibrated: log loss 0.28246, AUROC 0.96442, average precision 0.94937, ECE 0.04852.
Mean development-study prevalence: log loss 0.71240, AUROC 0.50000, average precision 0.42614, ECE 0.11682.
Calibration only. Neural scoring not rerun; no full-runtime claim. Separate research checkpoint; no serving-model replacement. Public data only, no author contact. ParSE-seq is now evaluated and cannot be reused as an untouched holdout for another revision.
M3 v2 · independent-source transfer test
Development improved. Clear superiority over ExtraTrees and Pangolin is not established.
M3 v2 beats joint-input ExtraTrees in 3 of 4 external cohorts, but calibrated Pangolin has lower equal-study Brier error. The new M3 explicitly uses Pangolin as an input. This is a research fusion calibrator, and no serving model was replaced.
Frozen models were tested on 2,230 public variants from four minigene cohorts separate from MFASS. Development used 5,801 MFASS variants in 324 groups and 420 tuning fits. Only the original development partition was used; neither the previous MFASS test partition nor the external outcomes were used for tuning.
Qualification: cross-study transfer with assay and scoring differences. Complete base-model training independence is unverified.
Brier error ↓ · equal-study mean is the locked primary metric; pooled error weights larger cohorts more heavily
Model
Equal-study Brier ↓
Pooled Brier ↓
M3 v2 · SpliceAI + Pangolin fusion
0.25284
0.20014
ExtraTrees · same two inputs
0.25551
0.20357
ExtraTrees · SpliceAI only
0.29705
0.23807
Pangolin · spline calibration
0.25061
0.20132
Pangolin · Platt calibration
0.26585
0.21194
Pangolin · isotonic calibration
0.25306
0.20546
SpliceAI · spline calibration
0.29224
0.23462
Ridge · same two inputs
0.30804
0.24947
Earlier ExtraTrees v1 · frozen
0.30527
0.24380
Earlier Pangolin Platt v1 · frozen
0.26734
0.21309
Earlier Pangolin isotonic v1 · frozen
0.25513
0.20732
Uncalibrated Pangolin score
0.26989
0.21433
Uncalibrated SpliceAI score
0.27719
0.22607
MFASS development prevalence
0.34937
0.28163
M3 has the lowest pooled Brier error among the evaluated models, but not the lowest primary equal-study error. That pooled result does not establish superiority. The sizeable errors on FAS and MST1R show poor transfer to those assays.
Results for every external cohort
Each cohort is one gene/exon; lower Brier error is better
Five-fold gene/exon-grouped development Brier selected a regularized additive spline over the logit-transformed SpliceAI and Pangolin scores. Joint-input ExtraTrees and ridge received the same underlying scores. Single-input calibrators and the earlier frozen baselines remain in the external comparison.
Development Brier: M3 fusion 0.02792, joint ExtraTrees 0.02811, Pangolin spline 0.02806. These are selection scores, not independent validation results.
314 development rows without gene symbols and 13 BRCA1 rows were excluded before the successful training run. External-study genes were reserved in advance. A first attempt failed while saving gene metadata; its inputs and log were preserved. Models and external predictions were frozen before numerical external evaluation.
Recorded runtime covers calibration fitting and prediction on published scores only. Neural inference, sequence retrieval and annotation are not included; no full-pipeline speed claim is made.
Uncertainty and the superiority decision
The locked success rule required improvement over every baseline and consistent wins against joint ExtraTrees and calibrated Pangolin across all four cohorts. It failed.
The following paired whole-gene bootstrap intervals are descriptive: only four independent genes are available, which is too few for reliable extreme-tail inference. An additional exact study sign-flip audit has a minimum attainable one-sided p-value of 0.0625, even before correcting for 13 comparisons. No confirmatory superiority claim is supported.
Positive differences favor M3. 10,000 resamples; 99.615% multiplicity-adjusted percentile intervals. The inference audit was added after evaluation without changing any model, endpoint or result.
ExtraTrees · same two inputs: baseline minus M3 0.002674 [-0.002677, 0.010077]; M3 wins 3/4 cohorts.
MFASS development prevalence: baseline minus M3 0.096534 [0.043572, 0.138587]; M3 wins 4/4 cohorts.
Public-data qualification, overlap and limitations
The publisher checksum matched. All 2,230 reference alleles matched hg19, all selected rows had usable scores and author-defined binary assay labels, and no rows were discarded. No gene or genomic-position overlap was found with the successful local v2 development set. The coordinate audit also checked adjacent positions conservatively.
FAS, MST1R, POU1F1 and WT1 were selected before model development because they have measured minigene outcomes and both scores. MLH1 clinical curation, BRCA1 combined RNA/protein-function labels, unlabelled random variants and unmatched FGFR2 data were excluded on assay grounds.
Different assay definitions: MFASS large exon-inclusion loss vs author-defined disruption including alternate isoform gains.
External hg19 MANE/exon-length scoring differs from development hg38 GENCODE44/50nt. Unmasked score columns used for both. Neural scores are published, not rerun.
Only four genes/exons and selected disease-associated loci; whole-gene confidence intervals have limited resolution.
SpliceAI/Pangolin training exposure to external genes was not ruled out. New local calibrators have no external gene or outcome training.
Published study and example rows were inspected for qualification. No external numerical metrics were used to tune models.
M3 v2 depends on Pangolin; this cannot demonstrate a standalone new sequence model is superior to Pangolin.
Not a clinical pathogenicity test or patentability determination.
Public data only; no author contact. These external cohorts have now been evaluated and must not be described as untouched holdouts for future revisions.
These pooled secondary metrics do not replace the prespecified equal-study Brier endpoint. Raw neural disruption scores are not calibrated probabilities.
M3 v2 · SpliceAI + Pangolin fusion: log loss 0.62823, AUROC 0.84586, average precision 0.72617, ECE 0.19665.
ExtraTrees · same two inputs: log loss 0.65307, AUROC 0.83538, average precision 0.71143, ECE 0.19969.
ExtraTrees · SpliceAI only: log loss 0.82469, AUROC 0.69831, average precision 0.57921, ECE 0.22828.
Pangolin · spline calibration: log loss 0.64345, AUROC 0.83039, average precision 0.69781, ECE 0.19606.
Pangolin · Platt calibration: log loss 0.66207, AUROC 0.83039, average precision 0.69781, ECE 0.20725.
Pangolin · isotonic calibration: log loss 0.66294, AUROC 0.82919, average precision 0.68280, ECE 0.19693.
SpliceAI · spline calibration: log loss 0.78852, AUROC 0.69845, average precision 0.58069, ECE 0.22077.
Ridge · same two inputs: log loss 0.75506, AUROC 0.77542, average precision 0.64958, ECE 0.21588.
Earlier ExtraTrees v1 · frozen: log loss 0.85708, AUROC 0.69852, average precision 0.57953, ECE 0.23720.
Earlier Pangolin Platt v1 · frozen: log loss 0.66598, AUROC 0.83039, average precision 0.69781, ECE 0.20874.
Earlier Pangolin isotonic v1 · frozen: log loss 0.66780, AUROC 0.82042, average precision 0.65878, ECE 0.19940.
Uncalibrated Pangolin score: log loss 0.70751, AUROC 0.83039, average precision 0.69781, ECE 0.21526.
Uncalibrated SpliceAI score: log loss 0.83723, AUROC 0.69853, average precision 0.58071, ECE 0.21095.
MFASS development prevalence: log loss 1.01534, AUROC 0.50000, average precision 0.30359, ECE 0.26497.
M3 · MFASS calibration benchmark
Calibration improved over old M3 on this assay. Superiority over strong baselines is not established.
The new spline calibrator has the lowest point-estimate Brier error among tested SpliceAI-only calibrators, but the paired intervals do not establish those advantages. Calibrated Pangolin has lower Brier error. No model was promoted to serving.
A frozen, research-only experiment on public MFASS reporter data: 6,128 development variants and 2,134 held-out variants in 133 separate gene/exon groups, including 79 measured splice-disrupting variants. This is a within-study holdout using published neural scores, not a new independent study or a clinical pathogenicity test.
Same 2,134 test variants · Brier ↓ primary · lower probability error is better
Model
Brier ↓
Log loss ↓
AUROC ↑
M3 MFASS v1 · spline calibration
0.03037
0.12707
0.80002
SpliceAI · Platt (raw score)
0.03104
0.13489
0.79997
SpliceAI · Platt (logit)
0.03212
0.13498
0.79997
SpliceAI · beta calibration
0.03136
0.13362
0.79997
SpliceAI · isotonic
0.03075
0.12811
0.79871
SpliceAI · ExtraTrees calibration
0.03077
0.13126
0.80011
Pangolin · Platt
0.02889
0.12151
0.87101
Pangolin · isotonic
0.02913
0.11522
0.86905
Old M3 · transfer diagnostic
0.07240
0.30486
0.79997
Uncalibrated SpliceAI
0.03186
0.18868
0.79997
Development prevalence
0.03565
0.15837
0.50000
Old M3 was calibrated on ClinVar-derived labels with a different score-extraction method. Its improvement here demonstrates assay-specific recalibration, not improved clinical performance. The new model does not beat calibrated Pangolin on the primary metric.
Positive baseline-minus-M3 Brier differences favor M3. 5,000 whole-group bootstrap resamples; 99.5% intervals adjust for ten comparisons. The success criterion required an interval above zero against every baseline, and was not met.
SpliceAI · Platt (logit): 0.001747 [-0.000482, 0.004391]. Superiority not established.
SpliceAI · beta calibration: 0.000989 [-0.000715, 0.002898]. Superiority not established.
SpliceAI · isotonic: 0.000386 [-0.000818, 0.001572]. Superiority not established.
SpliceAI · ExtraTrees calibration: 0.000397 [-0.000459, 0.001370]. Superiority not established.
Pangolin · Platt: -0.001482 [-0.004334, 0.000293]. Superiority not established.
Pangolin · isotonic: -0.001236 [-0.005054, 0.001196]. Superiority not established.
Old M3 · transfer diagnostic: 0.042036 [0.035187, 0.047395]. Supported improvement within this cohort.
Uncalibrated SpliceAI: 0.001491 [-0.000560, 0.003900]. Superiority not established.
Development prevalence: 0.005282 [0.000382, 0.011420]. Supported improvement within this cohort.
Data qualification and training overlap
Reconciled 27,733 primary MFASS variants and verified each reporter reference/alternate sequence pair. The published score subset contains 8,324 variants; 27 lack usable predictions and remain missing. A further 35 scored rows were excluded for missing gene annotations or membership in the existing ClinVar calibration gene set, leaving 8,262.
All 137 local ClinVar calibration variants were audited: zero exact-variant overlaps and zero chromosome/position overlaps. DMD is shared at gene level; its rows were excluded. Connected gene, exon and identical-variant groups cannot cross development/test boundaries. The hash-based partition was not retried or balanced using outcomes.
Base SpliceAI and Pangolin training exposure to these genes/exons is not ruled out. Local splice-site calibration also uses GENCODE chr22; absence of overlap with the delta calibration variants does not establish complete model-training independence.
Public score files matched their published SHA-256 records. They use GENCODE v44 canonical annotations, unmasked scores and a 50-nt delta search distance. Neural inference was not rerun locally. An alternative score release that converts missing predictions to zero was rejected.
M3 MFASS v1: logistic regression on cubic spline of clipped-logit SpliceAI score. n_knots [3,5,7], C [.01,.1,1,10,100], 15 configurations selected by development grouped-CV Brier.
215 development tuning fits used five grouped folds and Brier selection. Candidate, baseline configurations and test predictions were locked before numerical test evaluation. MFASS labels were checked for source consistency during qualification; they did not select the split or test predictions.
One previously published reporter study, subset determined by published score availability; no independent study claim.
Base SpliceAI/Pangolin training exposure to these genes/exons is not ruled out; only local calibrator separation can be audited.
Published score provenance/hashes can be checked; neural computations are not reproduced in this run.
Current M3 sequence-window deltas differ from annotation-based published variant scores, so research calibration is not a drop-in serving replacement.
Experimental exon-disruption labels do not mean clinical pathogenicity.
Qualification excluded missing gene annotations before any calibration training, as recorded in a protocol addendum. The research checkpoint uses a different input contract from the app’s current sequence-window M3 and is not a drop-in replacement.
Secondary metrics and reliability bins
ECE uses ten fixed bins including probability 1. A near-constant prevalence predictor can have low ECE while offering no useful ranking; ECE does not replace Brier as the primary metric.
SpliceAI · Platt (raw score): ECE 0.00254; average precision 0.26562.
SpliceAI · Platt (logit): ECE 0.00872; average precision 0.26562.
SpliceAI · beta calibration: ECE 0.00445; average precision 0.26562.
SpliceAI · isotonic: ECE 0.00750; average precision 0.24835.
SpliceAI · ExtraTrees calibration: ECE 0.00291; average precision 0.26915.
Pangolin · Platt: ECE 0.00514; average precision 0.35588.
Pangolin · isotonic: ECE 0.00905; average precision 0.33575.
Old M3 · transfer diagnostic: ECE 0.20128; average precision 0.26562.
Uncalibrated SpliceAI: ECE 0.01951; average precision 0.26562.
Development prevalence: ECE 0.00100; average precision 0.03702.
Model
Count
Mean probability
Observed disruption
M3 MFASS v1 · spline calibration
2022
0.02304
0.02522
M3 MFASS v1 · spline calibration
70
0.13302
0.12857
M3 MFASS v1 · spline calibration
18
0.25162
0.27778
M3 MFASS v1 · spline calibration
4
0.33618
0.25000
M3 MFASS v1 · spline calibration
6
0.43567
0.33333
M3 MFASS v1 · spline calibration
2
0.55809
0.50000
M3 MFASS v1 · spline calibration
6
0.65288
1.00000
M3 MFASS v1 · spline calibration
4
0.75164
0.50000
M3 MFASS v1 · spline calibration
2
0.80395
1.00000
Uncalibrated SpliceAI
2002
0.01076
0.02448
Uncalibrated SpliceAI
64
0.13656
0.14063
Uncalibrated SpliceAI
24
0.23417
0.04167
Uncalibrated SpliceAI
7
0.34714
0.28571
Uncalibrated SpliceAI
8
0.44125
0.37500
Uncalibrated SpliceAI
8
0.54125
0.25000
Uncalibrated SpliceAI
1
0.69000
0.00000
Uncalibrated SpliceAI
3
0.74333
0.33333
Uncalibrated SpliceAI
3
0.86000
0.33333
Uncalibrated SpliceAI
14
0.96571
0.78571
SpliceAI · Platt (logit)
2027
0.02849
0.02516
SpliceAI · Platt (logit)
88
0.12341
0.17045
SpliceAI · Platt (logit)
10
0.25429
0.70000
SpliceAI · Platt (logit)
5
0.34139
0.60000
SpliceAI · Platt (logit)
2
0.40078
1.00000
SpliceAI · Platt (logit)
2
0.89450
0.50000
Pangolin · Platt
2024
0.02006
0.02026
Pangolin · Platt
62
0.14687
0.24194
Pangolin · Platt
11
0.22941
0.36364
Pangolin · Platt
8
0.33412
0.25000
Pangolin · Platt
8
0.43469
0.37500
Pangolin · Platt
10
0.55560
0.50000
Pangolin · Platt
3
0.64258
1.00000
Pangolin · Platt
7
0.74937
0.71429
Pangolin · Platt
1
0.82332
1.00000
M4 v9 · comparison with the strongest ExtraTrees results
0/3 MAE wins against the strongest checked ExtraTrees result per study.
V9 improves MAE over v8 in two studies, worsens on SMG1, and does not improve the equal-study mean. These are development results on previously examined studies. They do not establish independent superiority. No fresh M4 or M5 holdout qualified in this search, and the serving models have not changed.
V9 adds 34 transcript-distribution features (medians and interquartile ranges) to the original 52 annotations. M4, ExtraTrees and ridge receive identical inputs and study-transfer selection. All 29,703 observations and 11,277 genes are retained. The historical ExtraTrees predictions were locked before this run.
MAE ↓ · lower is better · all evaluation genes included
Study
M4 v9
M4 v8
Strongest ExtraTrees
New ridge
NMDRHT
0.5032
0.5070
0.4724
0.5244
SMG1_HEK293
0.2902
0.2858
0.2826
0.3185
Tani2012
0.6146
0.6149
0.6140
0.6669
Equal-study mean
0.4693
0.4692
0.4563
0.5033
The primary comparator is the better of the locked historical ExtraTrees result and this run’s ExtraTrees result on each identical gene/target cohort. Historical recipes differ in training and tuning. This descriptive comparison is deliberately demanding; it is not an independent confirmatory test.
NMDRHT: frozen historical MAE 0.4750 (v8 study-transfer ExtraTrees); new 86-feature ExtraTrees MAE 0.4724. 10,990 identical evaluation genes.
SMG1_HEK293: frozen historical MAE 0.2826 (historical v5 frozen ExtraTrees); new 86-feature ExtraTrees MAE 0.2951. 11,026 identical evaluation genes.
Tani2012: frozen historical MAE 0.6142 (v8 study-transfer ExtraTrees); new 86-feature ExtraTrees MAE 0.6140. 7,687 identical evaluation genes.
All metrics, paired uncertainty and blend weights
NMDRHT
10,813 evaluation genes overlap the training studies. Weights: Retuned boosted component 6.1%; New ExtraTrees 24.9%; Ridge 69.0%.
Model
MAE ↓
Spearman ↑
R² ↑
M4 v9
0.5032
0.5722
0.1082
Retuned boosted component
0.4939
0.5847
0.1375
New ExtraTrees
0.4724
0.6056
0.2321
Ridge
0.5244
0.4770
0.0330
Training median
0.5675
Undefined
-0.1019
Frozen historical ExtraTrees MAE minus M4: -0.028202; adjusted 99.44% interval [-0.031746, -0.024795]. Positive favors M4.
New ExtraTrees MAE minus M4: -0.030786; adjusted 99.44% interval [-0.034561, -0.026649]. Positive favors M4.
Ridge MAE minus M4: 0.021240; adjusted 99.44% interval [0.019767, 0.022748]. Positive favors M4.
SMG1_HEK293
11,026 evaluation genes overlap the training studies. Weights: Retuned boosted component 52.0%; New ExtraTrees 38.1%; Ridge 9.9%.
Model
MAE ↓
Spearman ↑
R² ↑
M4 v9
0.2902
0.5683
0.1487
Retuned boosted component
0.3018
0.5347
0.0576
New ExtraTrees
0.2951
0.5998
0.1661
Ridge
0.3185
0.4092
-0.0476
Training median
0.3085
Undefined
-0.0719
Frozen historical ExtraTrees MAE minus M4: -0.007594; adjusted 99.44% interval [-0.010153, -0.005151]. Positive favors M4.
New ExtraTrees MAE minus M4: 0.004872; adjusted 99.44% interval [0.002002, 0.007658]. Positive favors M4.
Ridge MAE minus M4: 0.028247; adjusted 99.44% interval [0.023777, 0.032621]. Positive favors M4.
Tani2012
7,643 evaluation genes overlap the training studies. Weights: Retuned boosted component 0.0%; New ExtraTrees 92.7%; Ridge 7.3%.
Model
MAE ↓
Spearman ↑
R² ↑
M4 v9
0.6146
0.5026
0.0509
Retuned boosted component
0.6127
0.4632
0.0418
New ExtraTrees
0.6140
0.5009
0.0513
Ridge
0.6669
0.3900
-0.0595
Training median
0.6450
Undefined
-0.0758
Frozen historical ExtraTrees MAE minus M4: -0.000419; adjusted 99.44% interval [-0.001241, 0.000465]. Positive favors M4.
New ExtraTrees MAE minus M4: -0.000654; adjusted 99.44% interval [-0.001270, -0.000023]. Positive favors M4.
Ridge MAE minus M4: 0.052318; adjusted 99.44% interval [0.045025, 0.059798]. Positive favors M4.
2,000 paired gene bootstrap resamples; adjustment covers nine comparisons in this run, not cumulative model development. Intervals condition on published estimates and exclude laboratory/replicate uncertainty.
Selection protocol and limitations
86 annotation features: the original 52 plus median and IQR for each of 17 transcript features. Identical inputs, weights, seed and 12 configurations per component for all model families. Within each outer training pair, train each configuration on one study and validate on the other, in both directions. Choose lowest equal-study transfer MAE; optimize convex weights on the selected cross-study predictions. Refit bases on the two training studies and apply frozen weights to the third. No held-outer-study outcomes used in selection.
216 base tuning fits plus three convex optimizations. Ensemble combines all three families and has greater aggregate computation than a single baseline. Not a novel architecture claim.
All three studies are repeatedly used development data; no confirmatory superiority claim.
Weights and component configurations share an inner validation set, creating selection optimism; outer outcomes do not select weights.
Inner splits separate studies, not genes; gene overlap is allowed to match the cross-study deployment question. This does not establish unseen-gene generalization.
The ensemble contains the ExtraTrees baseline itself; any gain is an incremental ensemble effect, not independent biological novelty.
Different assays, perturbations and cell types remain. Gene bootstrap does not model laboratory/replicate uncertainty.
The annotation summaries are not weighted by each study’s expressed isoform abundance. The final all-study research checkpoint has no fresh accuracy estimate. 216 tuning fits are recorded; model and input checksums accompany the evidence.
Fresh public M4 / M5 data search
Public GEO metadata, primary articles and repository mirrors checked on 2026-10-13. No author contact. This bounded search found no new qualifying cohort; it does not establish that none exists.
ICE1 / GSE105436: Broad stability changes are inferred using REMBRANDTS; directly measured 5-EU half-lives use selected-gene qPCR. Does not qualify as a broad measured-decay holdout for M4. No supplementary numerical outcomes used.
Human RNA structural switches (2024): UPF1 perturbation is assessed by endogenous RORC RT-qPCR. This does not provide a matched genome-wide UPF1 RNA/RPF response or broad measured-decay endpoint.
GSE305934 / GSE305933 (2026 paper): Matched human RNA/RPF data exist, but the experiment measures stem-cell differentiation, not UPF1 perturbation. Not a qualifying M5 response holdout.
Ribosome-associated lncRNAs (2018): NMD sensitivity uses separate GSE86148 UPF1 RNA-seq, alongside collected baseline ribosome profiles. It is not matched perturbation RNA/RPF data for M5.
MDC Ribo-seq deposit 25705: Public human UPF1 Ribo-seq, but a mirror of the Boehm NMDRHT study already used. Not a fresh independent cohort.
M5 remains limited by poor transfer between the existing UPF1 studies. These excluded sources cannot resolve it or support a fresh superiority claim.
M4 v8 · selection for study transfer
0/3 MAE wins against the strongest checked ExtraTrees results.
Against the newly retuned v8 baseline alone, M4 wins 1/3. The goal of beating ExtraTrees across studies is not met. All results here are development diagnostics. Another fresh human M4 study and matched human M5 RNA/RPF study remain unqualified. No serving-model promotion.
M4 v8 selects configurations and blend weights by transfer between the two training studies, in both directions. The third study stays outside that selection. ExtraTrees and ridge receive the same study-transfer selection. This tests whether selection on study shifts works better than v7’s random gene split; all three studies remain previously examined development data.
MAE ↓ · lower is better · development only
Study
M4 v8
V7 reference
ExtraTrees
Ridge
NMDRHT
0.507
0.498
0.475
0.530
SMG1_HEK293
0.286
0.304
0.302
0.315
Tani2012
0.615
0.617
0.614
0.668
Equal-study mean
0.469
0.473
0.464
0.504
V7 is shown for context on the identical evaluation genes; its configurations and weights were selected differently. The ExtraTrees and ridge columns are retuned v8 baselines. All 216 base tuning fits and three weight optimizations are recorded. No fresh test or model promotion.
Check against the strongest observed ExtraTrees results
M4 v8 wins 0/3 on MAE against the best checked ExtraTrees recipe per study among v5, v6/v7 and v8. This retrospective comparison guards against claiming progress by weakening a baseline; it is not a preselected independent test. Historical recipes differ in tuning and training data, but gene membership and measured targets match.
NMDRHT: M4 v8 0.507 versus ExtraTrees 0.475 (v8 study-transfer selection).
Study metrics, confidence intervals and learned weights
NMDRHT · 10,990 genes
10,813 genes also appear in the training studies. Blend weights: Retuned boosted component 3.2%; ExtraTrees 28.2%; Ridge 68.6%.
Model
Spearman ↑
R² ↑
M4 v8 ensemble
0.573
0.096
Retuned boosted component
0.584
0.138
ExtraTrees
0.601
0.220
Ridge
0.475
0.014
Training median
Undefined
-0.102
ExtraTrees MAE minus M4 v8: -0.032; adjusted 99.17% interval [-0.035, -0.029]. Positive favors M4.
Ridge MAE minus M4 v8: 0.023; adjusted 99.17% interval [0.021, 0.024]. Positive favors M4.
SMG1_HEK293 · 11,026 genes
11,026 genes also appear in the training studies. Blend weights: Retuned boosted component 31.8%; ExtraTrees 47.8%; Ridge 20.4%.
Model
Spearman ↑
R² ↑
M4 v8 ensemble
0.587
0.190
Retuned boosted component
0.532
0.053
ExtraTrees
0.618
0.137
Ridge
0.406
-0.035
Training median
Undefined
-0.072
ExtraTrees MAE minus M4 v8: 0.016; adjusted 99.17% interval [0.013, 0.019]. Positive favors M4.
Ridge MAE minus M4 v8: 0.029; adjusted 99.17% interval [0.025, 0.034]. Positive favors M4.
Tani2012 · 7,687 genes
7,643 genes also appear in the training studies. Blend weights: Retuned boosted component 0.0%; ExtraTrees 93.7%; Ridge 6.3%.
Model
Spearman ↑
R² ↑
M4 v8 ensemble
0.503
0.052
Retuned boosted component
0.459
0.038
ExtraTrees
0.502
0.052
Ridge
0.388
-0.061
Training median
Undefined
-0.076
ExtraTrees MAE minus M4 v8: -0.001; adjusted 99.17% interval [-0.001, -0.000]. Positive favors M4.
Ridge MAE minus M4 v8: 0.053; adjusted 99.17% interval [0.046, 0.060]. Positive favors M4.
2,000 paired gene bootstrap resamples; intervals adjust for six comparisons in this run, not cumulative model development. They condition on published estimates, without laboratory or replicate uncertainty.
Protocol and limitations
Same v6 52 features, weights, seed and 12 configurations per component. Within each outer training pair, train each configuration on one study and validate on the other, in both directions. Choose lowest equal-study transfer MAE; optimize convex weights on the selected cross-study predictions. Refit bases on the two training studies and apply frozen weights to the third. No held-outer-study outcomes used in selection.
216 base tuning fits plus three convex optimizations. Ensemble combines all three families and has greater aggregate computation than a single baseline. Not a novel architecture claim.
Choose each component by mean inner MAE across folds; average the three inner-selected convex weight vectors; refit components on all development data.
All three studies are repeatedly used development data; no confirmatory superiority claim.
Weights and component configurations share an inner validation set, creating selection optimism; outer outcomes do not select weights.
Inner splits separate studies, not genes; gene overlap is allowed to match the cross-study deployment question. This does not establish unseen-gene generalization.
The ensemble contains the ExtraTrees baseline itself; any gain is an incremental ensemble effect, not independent biological novelty.
Different assays, perturbations and cell types remain. Gene bootstrap does not model laboratory/replicate uncertainty.
Research checkpoint and source hashes are recorded. No subset was selected to manufacture a win. The final all-study checkpoint has no fresh accuracy estimate.
Public M4 / M5 dataset search
Public GEO sample metadata, primary literature and repository searches including EBI, Figshare and Zenodo-linked results. Targeted review, not an exhaustive absence claim. No author contact.
GSE56748 / GSE56751 · MOV10: Six libraries are total, labeled and flow-through RNA under mock versus MOV10 depletion. The series mentions UPF1 binding, but measured decay perturbation is MOV10, not UPF1/SMG1. Excluded from the requested NMD holdout; numerical half-life table unopened.
Alternative UPF1 isoform study: Genome-wide stability changes are inferred from RNA-seq with REMBRANDTS; direct decay measurements are selected-gene qPCR. Not a broad measured-decay M4 test or matched RNA/RPF M5 test.
GSE143301 / GSM4407916 · ABCE1: Rechecked sample list: UPF1 samples are total RNA only. Shared Ribo-seq extraction text does not establish UPF1 RPF libraries. No qualifying matched UPF1 endpoint.
GSE303542 / GSM9130088: UPF1 sample is Neurospora crassa, despite human Ribo-seq text in shared processing metadata. It does not supply a fresh human UPF1 RNA/RPF cohort.
GSE186795 · ribosome-bound UPF1: Ribosome-associated UPF1 investigation in Saccharomyces cerevisiae. Wrong species for the requested human validation.
MDC SLAM-seq catalogue: Boehm/Landthaler HCT116 AID-UPF1, 0/12/24h: the already-used NMDRHT study. A catalogue mirror is not independent data.
No new qualifying independent human cohort found. The next fresh test requires measured matched outcomes, clear sample provenance and a frozen evaluation protocol before numerical outcome access.
M4 v7 · ensemble development
2/3 studies improve over M4 v6; 1/3 beat ExtraTrees on MAE.
The goal of beating ExtraTrees across studies is not met. All results here are development diagnostics. Another fresh human M4 study and matched human M5 RNA/RPF study remain unqualified. No serving-model promotion.
M4 v7 learns a nonnegative weighted blend of boosted trees, ExtraTrees and ridge using inner validation data. Each component uses the same features, study weights and 12-configuration search as v6. Held-study outcomes do not select blend weights. This ensemble includes ExtraTrees itself and costs more to train than a standalone baseline.
Study metrics, confidence intervals and learned weights
NMDRHT · 10,990 genes
10,813 genes also appear in the training studies. Blend weights: M4 v6 83.9%; ExtraTrees 16.1%; Ridge 0.0%.
Model
Spearman ↑
R² ↑
M4 v7 ensemble
0.568
0.124
M4 v6
0.545
0.100
ExtraTrees
0.596
0.213
Ridge
0.479
0.074
Training median
Undefined
-0.102
ExtraTrees MAE minus M4 v7: -0.021; adjusted 99.17% interval [-0.024, -0.018]. Positive favors M4.
Ridge MAE minus M4 v7: 0.015; adjusted 99.17% interval [0.012, 0.017]. Positive favors M4.
SMG1_HEK293 · 11,026 genes
11,026 genes also appear in the training studies. Blend weights: M4 v6 75.4%; ExtraTrees 24.6%; Ridge 0.0%.
Model
Spearman ↑
R² ↑
M4 v7 ensemble
0.516
0.054
M4 v6
0.478
-0.028
ExtraTrees
0.580
0.165
Ridge
0.414
-0.077
Training median
Undefined
-0.072
ExtraTrees MAE minus M4 v7: -0.010; adjusted 99.17% interval [-0.013, -0.007]. Positive favors M4.
Ridge MAE minus M4 v7: 0.021; adjusted 99.17% interval [0.018, 0.025]. Positive favors M4.
Tani2012 · 7,687 genes
7,643 genes also appear in the training studies. Blend weights: M4 v6 82.6%; ExtraTrees 15.4%; Ridge 2.0%.
Model
Spearman ↑
R² ↑
M4 v7 ensemble
0.467
0.035
M4 v6
0.454
0.033
ExtraTrees
0.496
0.036
Ridge
0.394
-0.056
Training median
Undefined
-0.076
ExtraTrees MAE minus M4 v7: 0.007; adjusted 99.17% interval [0.003, 0.010]. Positive favors M4.
Ridge MAE minus M4 v7: 0.048; adjusted 99.17% interval [0.044, 0.052]. Positive favors M4.
2,000 paired gene bootstrap resamples; intervals adjust for six comparisons in this run, not cumulative model development. They condition on published estimates, without laboratory or replicate uncertainty.
Protocol and limitations
Same v6 52 features, weights, gene-disjoint inner split, seed and 12 configurations per component. Select each base model on equal-study inner validation MAE; optimize convex weights on the same inner validation predictions. Refit bases on training studies, apply frozen weights to held study.
108 base tuning fits plus three convex optimizations. Ensemble combines all three families and has greater aggregate computation than a single baseline. Not a novel architecture claim.
Choose each component by mean inner MAE across folds; average the three inner-selected convex weight vectors; refit components on all development data.
All three studies are repeatedly used development data; no confirmatory superiority claim.
Weights and component configurations share an inner validation set, creating selection optimism; outer outcomes do not select weights.
Gene overlap across studies; inner validation genes are disjoint from inner training.
The ensemble contains the ExtraTrees baseline itself; any gain is an incremental ensemble effect, not independent biological novelty.
Different assays, perturbations and cell types remain. Gene bootstrap does not model laboratory/replicate uncertainty.
Research checkpoint and source hashes are recorded. No subset was selected to manufacture a win. The final all-study checkpoint has no fresh accuracy estimate.
Public M4 / M5 dataset search
Public GEO sample metadata, primary literature and repository searches including EBI, Figshare and Zenodo-linked results. Targeted review, not an exhaustive absence claim. No author contact.
GSE56748 / GSE56751 · MOV10: Six libraries are total, labeled and flow-through RNA under mock versus MOV10 depletion. The series mentions UPF1 binding, but measured decay perturbation is MOV10, not UPF1/SMG1. Excluded from the requested NMD holdout; numerical half-life table unopened.
Alternative UPF1 isoform study: Genome-wide stability changes are inferred from RNA-seq with REMBRANDTS; direct decay measurements are selected-gene qPCR. Not a broad measured-decay M4 test or matched RNA/RPF M5 test.
GSE143301 / GSM4407916 · ABCE1: Rechecked sample list: UPF1 samples are total RNA only. Shared Ribo-seq extraction text does not establish UPF1 RPF libraries. No qualifying matched UPF1 endpoint.
GSE303542 / GSM9130088: UPF1 sample is Neurospora crassa, despite human Ribo-seq text in shared processing metadata. It does not supply a fresh human UPF1 RNA/RPF cohort.
GSE186795 · ribosome-bound UPF1: Ribosome-associated UPF1 investigation in Saccharomyces cerevisiae. Wrong species for the requested human validation.
MDC SLAM-seq catalogue: Boehm/Landthaler HCT116 AID-UPF1, 0/12/24h: the already-used NMDRHT study. A catalogue mirror is not independent data.
No new qualifying independent human cohort found. The next fresh test requires measured matched outcomes, clear sample provenance and a frozen evaluation protocol before numerical outcome access.
M4 v6 · development comparison
One development MAE win over ExtraTrees; fresh validation pending.
M4 v6 beats ExtraTrees on Tani MAE only, and beats ridge on all three development folds. ExtraTrees wins MAE on NMDRHT and SMG1 and rank correlation on all three. No independent superiority established. Research checkpoints only; no serving promotion. Another fresh human M4 study and a fresh matched human M5 RNA/RPF study remain unqualified.
Changed M4 to absolute-error boosted trees, removed its fixed ridge blend, and weighted the three development studies equally. All families receive the same 52 features, study weights, gene splits and 12-configuration tuning budget. Selection uses equal-study validation MAE.
29,703 observations from 11,277 genes; 108 tuning fits. Each outer diagnostic trains on two studies and evaluates the third. These studies have all been inspected previously. The prior SMG1 result remains a historical v5 test; it is now development data for v6.
10,990 evaluated genes; 10,813 also occur in the training studies.
Model
MAE ↓ primary
Spearman ↑
R² ↑
M4 v6
0.505
0.545
0.100
ExtraTrees
0.477
0.596
0.213
Ridge
0.513
0.479
0.074
Training median
0.568
Undefined
-0.102
ExtraTrees MAE minus M4: -0.028; adjusted interval [-0.032, -0.025]. Positive favors M4.
Ridge MAE minus M4: 0.008; adjusted interval [0.005, 0.011]. Positive favors M4.
SMG1_HEK293 · development only
11,026 evaluated genes; 11,026 also occur in the training studies.
Model
MAE ↓ primary
Spearman ↑
R² ↑
M4 v6
0.315
0.478
-0.028
ExtraTrees
0.294
0.580
0.165
Ridge
0.326
0.414
-0.077
Training median
0.309
Undefined
-0.072
ExtraTrees MAE minus M4: -0.021; adjusted interval [-0.025, -0.017]. Positive favors M4.
Ridge MAE minus M4: 0.011; adjusted interval [0.007, 0.014]. Positive favors M4.
Tani2012 · development only
7,687 evaluated genes; 7,643 also occur in the training studies.
Model
MAE ↓ primary
Spearman ↑
R² ↑
M4 v6
0.616
0.454
0.033
ExtraTrees
0.624
0.496
0.036
Ridge
0.665
0.394
-0.056
Training median
0.645
Undefined
-0.076
ExtraTrees MAE minus M4: 0.007; adjusted interval [0.003, 0.011]. Positive favors M4.
Ridge MAE minus M4: 0.049; adjusted interval [0.044, 0.053]. Positive favors M4.
Compared with v5, M4 MAE improves from 0.564 to 0.505 on NMDRHT and from 0.679 to 0.616 on Tani, but worsens from 0.307 to 0.315 on SMG1. Training data and weighting changed, so this does not isolate the algorithm’s effect. ExtraTrees also benefits from the added development data.
Validation limits and next-test rules
GSE289489 was already evaluated for v5 and is now explicitly development data.
Study-balanced weighting is used by all families; validation scores equally weight studies.
Outer folds have gene overlap. Inner splits separate genes across all training studies.
Repeated development on known studies cannot establish independent superiority.
Different UPF1/SMG1 perturbations, cells and decay assays remain; no assay offset learned from held-study outcomes.
Gene bootstrap intervals condition on measured estimates and do not capture laboratory or replicate uncertainty.
Two thousand paired gene bootstrap resamples; 99.17% intervals adjust for six within-run comparisons, not cumulative model development. All eligible genes are retained, without significance or winning-subgroup filtering. Final research checkpoints fit all three studies; their hashes are in the evidence download. No fresh accuracy is assigned to these final checkpoints.
A new study must supply public, measured human NMD decay responses with matched controls, clear sample labels and independent experimental lineage. Freeze candidate and baseline predictions and the primary metric before opening its numerical outcomes. M5 additionally requires matched UPF1 RNA/RPF measurements.
Fresh M4 / M5 public-data search
Targeted primary-study, supplementary lineage and public repository review, not an exhaustive proof that no qualifying dataset exists. No author contact.
2015 BRIC study: UPF1 RNA/BRIC measurements explicitly reuse DRA000591. New EXOSC5/STAU1 perturbations are not an independent NMD response endpoint.
2026 EZbakR final publication: GSE289489 is the evaluated SMG1 cohort, now development. GSE291379 measures cleavage sites, not decay-rate responses or RPF.
GSE207924: Subcellular RNA kinetics in K562 and mouse cells; no matched NMD perturbation response established.
GSE143662: Human DCP2 knockout stability assay; different perturbation from the requested NMD response test.
TDP43 endogenous stability study: Selected SRSF2/TARDBP isoform qPCR decay measurements. Does not qualify as a broad gene-level M4 benchmark.
MDC human UPF1 Ribo-seq deposit: External deposit resolves to E-MTAB-13837, already-used NMDRHT. A separate catalogue entry is not a fresh study.
GSE308938: Public September 2026, matched UPF1 RNA/RPF in yeast. Wrong species for the requested human M5 holdout.
GSE134059: Human UPF1 isoform RNA-seq and RIP-seq. RIP is an RNA-binding assay, not RPF; no matched M5 endpoint.
Fresh independent testing remains pending. No authors were contacted.
Historical M4 v5 · independent SMG1 test
M4 beats ridge; ExtraTrees still wins.
This historical human SMG1 decay test evaluated the improved M4 v5 on 11,026 genes. Superiority over both baselines is not established. M5 still lacks a qualifying fresh matched human UPF1 RNA/RPF holdout.
Public sources only. No author contact is required or planned.
GSE289489: HEK293, SMG1 inhibition versus DMSO, three labeled biological replicates per condition. Endpoint: log₂ decay-rate ratio. This is a separate NMD-pathway transfer test, not direct UPF1 validation. Public processed estimates and study methods.
Model
MAE ↓ primary
Spearman ↑
R² ↑
M4 v5
0.307
0.484
0.065
ExtraTrees
0.283
0.597
0.226
Ridge
0.322
0.427
-0.044
Training median
0.309
Undefined
-0.072
ExtraTrees MAE minus M4: -0.024; adjusted 97.5% interval [-0.027, -0.021]. Positive favors M4.
Ridge MAE minus M4: 0.015; adjusted 97.5% interval [0.013, 0.017]. Positive favors M4.
This cohort is now development data for M4 v6. For this historical v5 test, all three trained models, predictions, gene mapping and MAE-first protocol were frozen before numerical outcomes were opened. No fitting or calibration used this study. Baselines use their existing development-tuned configurations. Two thousand paired gene bootstrap resamples adjust for the two primary comparisons; intervals do not include replicate or laboratory uncertainty.
Coverage, independence and limitations
19,093 source genes: 2,754 lack an unambiguous reference mapping, 5,313 mapped genes lack frozen features, and 11,026 are evaluated. All evaluated genes appeared in development studies: study-level independence does not establish unseen-gene generalization.
Human SMG1 pathway transfer, not direct UPF1 depletion or therapeutic efficacy.
Previously published aggregate findings were inspected for qualification; numerical gene outcomes were not opened before prediction lock.
Public independent source study, no third-party blinded custody or raw-read duplicate audit.
Existing input bank derives from development response availability and limits coverage; many genes may be previously seen.
Uses published GRAND-SLAM/EZbakR gene estimates, not raw-read re-quantification; approximation and annotation differences remain.
The 21.7 MB published gene-level estimator was selected before outcomes; the alternate 821 MB object exceeded workspace capacity. No winning estimator or significant-gene subset was selected. This study has now been evaluated and must not be reused as a fresh holdout after further tuning. No model promotion or patent-superiority conclusion follows.
No new qualifying matched human UPF1 RNA/RPF cohort found. No new M5 accuracy claim or retraining on these incompatible endpoints. Existing v6 and assay-sensitivity failures remain visible.
GSE304548: Human UPF1 knockdown iPSC RNA-seq; no matched RPF in this deposit.
GSE304550: Human motor-neuron RNA-seq; associated RIP-seq measures RNA binding, not ribosome footprints. Not a qualifying TE holdout.
GSE289489: Human metabolic-labeling decay and long-read RNA measurements; qualifies only for the separate M4 SMG1 decay test, not M5 translation efficiency.
Next: qualify a public UPF1 study with matched RNA/RPF, replicated control and perturbation, consistent timing and an independent cohort before another accuracy test. GSE276918 remains excluded for M4 because public metadata conflict. Further M4 development should address its gap to ExtraTrees on development data, then use another untouched study.
M4 / M5 · metadata and assay investigation
No independent superiority established.
M4 v5 slightly improves NMDRHT absolute error, but its advantage over ExtraTrees is inconclusive and Tani performance worsens. GSE276918 still fails qualification. M5’s estimator sensitivity test does not fix UPF1 transfer. Direct UPF1 holdouts remain pending; the newer M4 SMG1 transfer test is shown above.
Do not fit decay rates or benchmark GSE276918 until an authoritative sample map resolves these issues. New outcome counts remain unopened.
Current GEO series and all six conflicting UPF1 sample records still have the 4h/8h title-versus-filename mismatch; records list May 31 2025 as last update.
ENA experiment records SRX26053390/SRX26053393 and BioSample SAMN43580456 repeat GEO titles rather than provide an independent corrected sample map.
Actinomycin-D is supported by the BioSample treatment and ENA library protocol, but the GEO summary still says alpha-amanitin. This strengthens one interpretation without resolving the provenance conflict.
ENA submitted-file reports expose run IDs but no original submitted filenames or checksums for the two checked experiments. They do not resolve the time mapping.
No publication identifier is linked in the current GEO series. A related author/title preprint was located through Europe PMC, but full text retrieval returned 403; its methods/supplement were not verified.
Exact depletion duration, paired control timing and whether spike-ins were used for sequencing normalization remain unconfirmed.
A separate 50/50 ridge and gradient-boosting ensemble uses the existing 52 geometry features. Each family receives 12 configurations and identical gene splits. MAE was chosen as the primary metric before this run to address absolute prediction error; historical v4 selected Spearman. All metrics are shown, and these repeatedly examined studies remain development data.
NMDRHT · 10,990 genes
7,400 genes also occur in the training study. Lower MAE is better; higher Spearman/R² is better.
Model
MAE · primary
Spearman
R²
M4 v5 · ridge/boost blend
0.564
0.454
-0.090
ExtraTrees
0.565
0.459
-0.090
Ridge
0.566
0.447
-0.103
Training median
0.619
Undefined
-0.320
ExtraTrees MAE minus M4 MAE: 0.001; adjusted interval [-0.002, 0.003]. Positive favors M4.
Ridge MAE minus M4 MAE: 0.002; adjusted interval [-0.001, 0.004]. Positive favors M4.
Tani2012 · 7,687 genes
7,400 genes also occur in the training study. Lower MAE is better; higher Spearman/R² is better.
Model
MAE · primary
Spearman
R²
M4 v5 · ridge/boost blend
0.679
0.431
-0.077
ExtraTrees
0.652
0.480
-0.028
Ridge
0.691
0.403
-0.102
Training median
0.671
Undefined
-0.117
ExtraTrees MAE minus M4 MAE: -0.027; adjusted interval [-0.033, -0.020]. Positive favors M4.
Ridge MAE minus M4 MAE: 0.012; adjusted interval [0.010, 0.015]. Positive favors M4.
NMDRHT MAE improves from v4’s 0.582 to 0.564, while Tani MAE worsens from 0.664 to 0.679. ExtraTrees retains higher Spearman in both studies. All M4 v5 R² values remain negative. No model promotion.
M4 protocol and uncertainty
12 configurations per family; inner gene-disjoint 80/20 split using unchanged aligned-v3 hash ordering; select lowest validation MAE. Refit training study; evaluate all genes of other study. Final fit selects mean inner MAE across folds, then fits both studies.
1,000 paired gene bootstrap resamples per fold; 98.75% intervals adjust for four comparisons within this experiment, not the cumulative model-development history. Intervals do not include uncertainty across laboratories or biological replicates.
Two repeatedly examined studies; development diagnostics only.
Study and gene overlap differ: 7400 genes shared between studies.
SLAM and BRIC assays, time/perturbation and feature-ascertainment differences persist.
No test-based calibration or outcome filtering.
Fixed 50/50 ridge/boost blend is explicitly compared with standalone ridge and tuned ExtraTrees.
M5 · assay estimator sensitivity
Separate RNA/RPF DESeq2 fits do not resolve negative UPF1 transfer. Approximate target ranks remain nearly identical to the joint estimator. This sensitivity analysis cannot identify a causal mechanism or repair missing time-matched controls. Frozen v6 predictions were rescored without retraining or changing targets in the serving model.
Same original source count matrices, assay-specific median-ratio normalization, and common count-filter rules. Fit depleted-versus-control separately within RNA and RPF; TE = RPF log2FC minus RNA log2FC. No significance filter; evaluation membership is the unchanged v6 cohort. This omits cross-assay pairing, so it is a sensitivity diagnostic, not a replacement for the paired Aliouat estimator.
All correlations below are Spearman. Model columns rescore the same frozen v6 predictions on the alternative targets; they are not fresh-test results.
Study
Genes
Old/new target correlation
M5 v6
ExtraTrees
Ridge
Aliouat2019
5,268
0.994
-0.221
-0.216
-0.179
NMDRHT
4,886
0.995
-0.313
-0.319
-0.275
Across 4,628 shared genes, RNA-response correlation is 0.229, RPF-response correlation is 0.316, but TE-response correlation remains -0.186 (joint estimator: -0.189). This narrows the diagnosis without establishing its cause.
Missing time-matched RPF controls, different depletion durations, source annotations and replicate pairing remain unresolved. No sign reversal, winning-subgroup selection or new M5 superiority claim is justified.
Fresh human M5 search · no new qualifying cohort
Targeted human UPF1 RNA/RPF searches and primary full-text lineage review; not exhaustive. Rechecked recent papers and the eRF1/UPF1 study trail.
Feedback from the Nascent Chain (2026): UPF1 abundance analysis cites Boehm/NMDRHT in reference 37; no new matched human UPF1 RNA/RPF study established.
Hidden stop codons / GSE303542: Previously reviewed human RPF arm is eRF1 depletion; it does not supply matched human UPF1 RNA/RPF.
Public data only: GSE276918 remains excluded until deposited metadata resolve its time and treatment conflicts.
M5 v6 · UPF1 transfer investigation
Transfer is not fixed. No superiority established.
Negative correlation and negative R² in both UPF1 transfer diagnostics; ridge has higher Spearman on both; no fresh study has qualified. Transfer is not fixed. Separate research artifacts were saved; this revision is not promoted to the serving model.
The two UPF1 studies disagree on their measured responses across 4,628 shared genes: Spearman -0.189. Both estimators encode RPF change minus RNA change. This disagreement does not establish a sign error or its cause. Acute versus chronic depletion, time controls and quantification remain unresolved differences.
Tested a UPF1-only recipe: remove SMG1 and added geometry, retain 103 sequence features, regularize M5 more strongly, and downweight uncertain training responses. ExtraTrees and ridge use the same features, precision weights and gene splits, with 12 configurations each. All eligible evaluation genes remain included.
Compared with the previous v5 recipe, M5 improves Aliouat correlation from −0.312 to −0.219 but worsens NMDRHT from −0.270 to −0.321. These are different development recipes on reused studies, not independent confirmation. Ridge outperforms v6 on both UPF1 folds. Negative R² means worse squared error than predicting the test mean.
Protocol, uncertainty and limitations
12 configurations per family; deterministic gene-disjoint 80/20 inner split; maximize mean unweighted inner Spearman; outer leave-one-study-out diagnostics. No sign flipping, test subgroup selection or outer-score selection.
Per training study: 1/(lfcSE squared + training median SE squared); replace invalid SE with training median; cap at training 95th percentile; normalize equal study totals. Recomputed within inner training, never from validation/test uncertainty.
1,000 paired gene-bootstrap resamples per fold. 98.75% intervals adjust for four comparisons within this experiment, not the whole history of repeated model development. They condition on estimated gene responses and do not capture replicate or laboratory uncertainty. Final models fit both development studies and have no fresh accuracy estimate.
Both studies already used and repeatedly inspected; development only.
Acute 12h versus chronic 72h UPF1 depletion, unmatched NMDRHT time controls, source quantification differences persist.
Sequence-only input cannot identify perturbation duration or experimental context.
Precision weighting relies on approximate DESeq2 standard errors; it does not repair study confounding.
Gene overlap across studies is reported; all eligible outer genes are retained.
Fresh human dataset search · no qualifying holdout
Targeted web searches using UPF1/RPF/ribosome profiling and human/HeLa/HepG2 terms, followed by primary GEO metadata and two full-text method reviews. An extension of previous audits, not an exhaustive repository census. No numerical outcomes from new studies were scored.
GSE296644: UPF1 knockdown with Oxford Nanopore direct RNA sequencing; no matched RPF assay established.
GSE246483: Human UPF1 knockdown/mutant complementation, triplicate rRNA-depleted RNA-seq; no matched RPF assay established.
GSE228561 / GSE228565: Human renal-cell-carcinoma ribosome profiling follows fumarate-related genotype changes. The screened mutant RPF sample does not establish UPF1 perturbation.
GSE252369 / NSUN methylation: Human UPF1 knockdown is measured by RNA-seq in WT and NSUN2 knockout cells; UPF1 footprints are CLIP/RNA-binding measurements, not matched ribosome-protected fragments.
Human heart / fibroblast RBP analysis: RNA/Ribo-seq and RBP-target correlations are available, but this is not a direct UPF1-perturbation experiment. Fibroblast replication uses TGF-beta stimulation.
Next evidence needed: A public independent human study with UPF1 and control RNA plus RPF, matched conditions, biological replicates and unambiguous sample labels.
No fresh independent test was run. RNA-binding footprints, RNA abundance alone and unrelated perturbations cannot establish the requested UPF1 translation-efficiency superiority.
M5 v5 · SMG1-informed development
Higher SMG1 development score; superiority over ExtraTrees remains unproven.
M5 v5 scores 0.312 versus ExtraTrees 0.290 and ridge 0.258. Its +0.022 advantage over ExtraTrees has an adjusted interval of −0.010 to +0.056, which includes no advantage. Both UPF1 transfer folds have negative correlations. No fresh independent test has been run.
Separate research checkpoints trained on 13,616 study–gene rows (5,591 unique genes). Added annotation geometry and an explicit perturbation flag: 157 features, equal study weights, and 12 tuning configurations per family per fold. ExtraTrees and ridge receive the same features, splits and tuning budget. The serving model is unchanged.
All three studies have been examined previously and are now development data. These leave-one-study-out diagnostics fit two studies and score the third; tuning uses gene-disjoint inner validation only. Final research models are then fitted on all three studies and have no independent accuracy estimate.
Aliouat2019 · retrospective transfer diagnostic
5,268 evaluated genes; 4,742 also occur in the training studies. Higher Spearman/R² and lower MAE are better.
1,000 paired gene bootstrap resamples; 99.17% intervals adjusted for six comparisons. These are descriptive development intervals, conditional on each study, and do not capture uncertainty across laboratories or biological replicates. The SMG1 diagnostic is numerically better than historical v4 (0.194), but reuse of the study prevents a fresh validation claim. Negative UPF1 correlations and R² indicate poor transportability, including worse squared error than predicting the test mean.
Limitations and model selection
SMG1 is now development data and cannot be reused as a fresh final test.
Only three source studies, two UPF1 and one SMG1; perturbation and cell/study context are confounded.
A held-SMG1 fold trains on UPF1 only; its perturbation indicator is unseen in training.
Gene overlap between outer studies is reported; study holdout does not imply unseen genes.
Geometry comes from the prior NMDRHT annotation and inherits its discovery/response-availability ascertainment.
No fresh independent study has yet qualified; no model is promoted or superiority guaranteed.
Mean inner-validation Spearman across the three outer folds, then refit all development data with equal study weights. Final-fit inner scores: M5 0.284, ExtraTrees 0.320, ridge 0.228. These are selection scores, not independent performance.
Targets combine RNA and RPF estimates across different perturbations, cell contexts and quantification protocols. This predicts endogenous-gene translation-efficiency responses, not therapeutic efficacy. Historical v4 runtime measurements below do not measure v5 runtime.
Fresh human holdout · not yet qualified
No fresh qualified human study found; independent test not run. Search is not exhaustive.
Human, direct UPF1 perturbation (or separately labelled SMG1 transfer), control and treated RNA plus RPF, biological replication, compatible gene-level TE endpoint, accessible sample provenance, and no overlap with development experiments. Qualify before opening numerical outcomes.
Palbociclib / GSE297939: Not qualified: deposited RNA-seq and ATAC-seq are palbociclib versus DMSO. SMG1 appears in targeted RT-PCR validation, with no matched SMG1 RNA/RPF cohort.
Hidden stop codons / GSE303542: Not qualified: human Ribo-seq compares eRF1 depletion with WT. UPF1 knockout is in fungal SLAM-seq; this is not a human UPF1 RNA/RPF contrast.
NMDRHT / E-MTAB-13837: Already used: a newly indexed deposit or publication date does not make this existing NMDRHT experiment independent.
Obtain an unused study with matched RNA/RPF and sample metadata. Freeze these model hashes, cohort rules, primary metric and paired comparisons before testing once.
M5 v4 · historical SMG1-inhibition transfer test
Historical frozen-v4 result preserved below. This SMG1 dataset is now development data for M5 v5 above and cannot serve as a fresh test again.
M5 beats ridge on SMG1 transfer; ExtraTrees is stronger.
A supported ranking advantage over ridge on this cohort, not superiority over both baselines. This tests transfer from UPF1-trained models to SMG1 inhibition, not an independent direct UPF1 response.
3,462 evaluated human genes · K562 WT cells · SMG1 inhibitor versus DMSO · two biological replicates per condition. Existing models and predictions were frozen before opening the numerical counts. No retraining or holdout tuning.
Same genes and response estimator for every method. Higher Spearman and R², lower MAE are better.
Model
Spearman
MAE
R²
M5 · frozen v4
0.194
0.267
0.017
ExtraTrees · frozen v4
0.256
0.262
0.045
Ridge · frozen v4
0.110
0.271
0.003
Training median
Undefined · constant
0.271
-0.001
M5 minus ExtraTrees: -0.062 Spearman; adjusted paired interval [-0.096, -0.026]. ExtraTrees has higher correlation on this SMG1 cohort.
M5 minus ridge: 0.084 Spearman; adjusted paired interval [0.051, 0.116]. Supported M5 ranking advantage on this SMG1 cohort.
2,000 paired gene-bootstrap resamples. The two primary comparisons use Bonferroni-adjusted 97.5% intervals. These measure uncertainty across genes conditional on this experiment, not across laboratories or biological replicates. M5 explains only 1.671% of target variance; absolute accuracy remains limited.
Cohort, mapping and estimation
24,244 transcript rows → 12,922 mapped genes → 9,539 finite count-filtered targets → 3,462 genes with frozen features and predictions. 6,077 target genes lacked features; 2,697 feature-bank genes lacked an eligible target.
3,397 evaluated genes were seen in development; 65 were not. This is study transfer, predominantly on previously seen genes.
Transcript counts sum to supplied stable gene IDs. The paired DESeq2 design uses a lysate intercept, assay term and SMG1-treated-RPF interaction, with normalization performed separately within each assay. Zero fits failed convergence. No significance or response-magnitude filter.
New K562 SMG1 study distinct from Aliouat HCT116 and Boehm NMDRHT; publication/accession/sample metadata checked. Raw read identity not deduplicated. Gene overlap expected and reported. No independent third-party custody. Supplement header/methods inspected previously, not numeric gene responses.
Measured runtime · precomputed features only
Method
Batch
Median ms
p95 ms
M5 · frozen v4
1
0.859
1.128
ExtraTrees · frozen v4
1
26.023
27.133
Ridge · frozen v4
1
1.014
1.231
M5 · frozen v4
32
0.889
1.192
ExtraTrees · frozen v4
32
26.469
26.863
Ridge · frozen v4
32
1.049
1.179
30 rotating-order warm rounds for batches 1/32 sorted by gene, same four-thread budget. Precomputed-feature input validation/copy, imputation, prediction and JSON serialization; no sequence folding, feature extraction, HTTP, or target estimation. Load and first all-bank inference recorded separately. Shared local machine; these timings do not establish full-workflow speed superiority or compensate for the accuracy difference.
What this result supports—and its limitations
Different perturbation and cell context from training; SMG1 blockade is not UPF1 depletion.
GENCODE35 source quantification versus existing representative transcript features; gene aggregation and annotation versions differ.
Feature bank was originally selected from NMDRHT response availability and limits coverage.
Two replicates per condition; gene bootstrap does not quantify cross-study or biological-replicate uncertainty.
No retraining, calibration, sign flipping, winning subgroup selection or serving-model promotion.
Most evaluated genes were seen during model development: study independence does not mean gene independence. The 65 unseen genes are not used to select or claim a winning subgroup.
The original study’s selected transcript annotation used pooled ribosome data. Aggregation uses all supplied transcript rows for each gene, with no outcome-based selection here.
The paired count-model coefficient is an estimate with two biological replicates per condition and two residual degrees of freedom; gene-bootstrap intervals condition on those estimates.
Runtime covers precomputed-feature validation, preprocessing, inference and JSON serialization only. It is not sequence-to-result, HTTP, cold-process or production latency; thread configuration is fixed rather than optimized per method.
A constant training-median baseline has undefined Spearman correlation. An initial export failed on NaN; this was fixed without altering the cohort, targets or frozen predictions.
No model promotion, direct UPF1 superiority, therapeutic-efficacy or patentability conclusion follows.
Direct human UPF1 holdout search continues
Targeted Europe PMC search of 2025–2026 literature using UPF1/RENT1 and ribosome-profiling/Ribo-seq terms, plus targeted web queries. Metadata screening is not full qualification of every record.
TDP-43 loss / cryptic polyadenylation, 2025: Methods explicitly restrict the referenced dual TDP-43/UPF1 experiment’s RNA-seq to TDP-43/control and control/control arms. A matched UPF1 RNA/RPF experiment is not established.
ReLiC human RNA-linked CRISPR screen, 2025: Abstract describes barcoded reporters and polysome fractionation, not an established genome-wide matched UPF1 RNA/RPF contrast. Full-text API returned an access error; remains unqualified.
No new direct human UPF1 holdout qualified in this extension. This does not prove none exists; supplementary and unindexed sources may still be missed.
Supplementary files checked; no fresh human UPF1 RNA/RPF holdout qualified.
2026-10-13 · 0 qualified studies · Benchmark not run
The strongest new lead has real human RNA/RPF counts, but a different perturbation. M5, ExtraTrees and ridge have not been scored on these sources. No new superiority claim is supported.
Supplement S5 has 12 monosome/RPF and 12 RNA sample columns: WT, EIF4E2-loss and GIGYF2-loss, each with DMSO or SMG1 inhibitor and two replicates. All 36 archive libraries match these conditions (12 RNA, 12 monosome, 12 disome). No UPF1-depleted arm appears.
A possible separate SMG1-inhibition transfer study, subject to its own qualification. It cannot be relabeled as a UPF1-depletion holdout.
All eight deposited libraries are RNA-seq: control, UPF1, SMG6 and G3BP1 perturbations in two replicates. The paper’s ribosome-profiling analysis uses a previously published zebrafish dataset; it is not a human UPF1 RNA/RPF pair.
The human RNA response cannot alone define the M5 translation-efficiency response.
Reuses the previously audited GSE143301 sample family
The article links HeLa RPF samples GSM4256659–61 and GSM4256665–67, plus UPF1 total-RNA samples GSM4407914–16. These are the previously checked control-RPF and UPF1-RNA records; no matched UPF1 RPF arm is established. Supplementary source data do not make this a new study.
Article methods and supplementary inventory reviewed; a new matched experiment was not found.
Data availability points to cell-line RNA-seq/TCR-seq (PRJNA1263526), controlled-access patient RNA data, and a Zenodo figure-data deposit. Reviewed methods focus on SMG1 perturbation, RNA and immunopeptidomics; they do not establish the required UPF1 RNA/RPF contrast.
Article data availability and supplementary inventory were checked. The linked Zenodo record could not be retrieved (API timeout and web-access error). Figure-data archives were not opened and restricted patient data were not requested.
This is a targeted search, not proof that an eligible dataset does not exist. Repository indexing may miss supplementary content.
S5 was streamed to inspect and save its column header only. No gene-response rows were analyzed; supplementary captions and sample metadata were inspected.
BioStudies’ initial broad query returned an approximate 14,324 hits; only the first 100 metadata records were retrieved. The explicit Boolean query returned 10 records. These are not counts of independent eligible studies.
The initial Zenodo request exceeded its anonymous page-size limit; retrying with 25 succeeded with zero results. The paper-linked Zenodo record was attempted separately but could not be retrieved; this access gap remains.
No model fitting, new prediction run, accuracy benchmark or runtime benchmark was performed. Existing v4 evidence and checkpoints are unchanged.
Next evidence needed
Public deposits must supply matched outcomes and unambiguous sample metadata before qualification. No author contact is planned.
A qualifying study will be evaluated with frozen models and common cohort rules; results will include failures and uncertainty, regardless of which model wins.
M5 · expanded human holdout search
No fresh human M5 holdout qualified in this expanded search.
2026-10-13 · 0 qualified studies · New benchmark: not run
M5 versus ExtraTrees and ridge remains pending a qualifying independent dataset. No new accuracy or superiority result is available.
840 human SRA experiments 80 study accessions · UPF1 and aliases
Metadata-only screen of UPF1, RENT1 and HUPF1 in human SRA records, plus the KnockRBP multi-omics catalogue and a recent GEO lead. Counts describe catalogue records, not independent cohorts; sources overlap.
All 1,453 catalogue entries screened. The 24 Ribo-seq entries target other RBPs. UPF1 entries DataSet_212, 338, 975 and 1135 are Poly(A)-seq or Total RNA-seq in CHA15, HeLa, HepG2 and K562. They cannot supply the paired RNA/RPF endpoint.
GEO metadata lists Saccharomyces cerevisiae for every sample, including WT and upf1-deletion RNA-seq and 80S Ribo-seq. Suitable assay names do not establish a human holdout.
The human RPF matches resolve to Aliouat and NMDRHT data already used in development/evaluation. Another accession or archive release does not make them untouched.
Ribosomal RNA depletion (RiboMinus), RIP-seq and ribosome-bound decay intermediates are not automatically ribosome-protected-fragment translation-efficiency measurements. Keyword matches were not treated as assay qualification.
Obtain an independent human study with UPF1 perturbation and matched controls measured by both RNA-seq and RPF, biological replication, sample-level counts and a clear sample sheet. A lab-held or not-yet-indexed deposit remains a possible route; no author has been contacted.
Verify human origin, perturbation, assay identity, matched conditions, replication and study independence before reading outcome tables.
Freeze the cohort, feature mapping, target estimator and all three existing pooled v4 checkpoints before evaluation; no tuning on the holdout.
Compare M5 with ExtraTrees and ridge using identical evaluable genes, paired uncertainty intervals, error metrics and measured runtime. Report negative results and exclusions.
Search limitations
This is a dated search snapshot, not proof that no eligible dataset exists. Unindexed, private or poorly annotated data may be missed.
The GEO alias query returned 501 sample IDs; these IDs were saved but not independently adjudicated sample by sample in this expansion.
No new numerical outcome table was opened, no model was fitted and no new superiority or latency result is claimed. Existing v4 development scores remain unchanged.
Fresh human holdout search · earlier evidence
New M4 lead found; neither endpoint has a qualified fresh holdout yet.
Qualification audit complete for this search; no new accuracy benchmark run. New outcome tables remain unopened. The v4 accuracy results below are unchanged.
Metadata and methods search only. Catalogue totals overlap and are not counts of independent eligible studies. Search results are bounded by indexing and public availability; zero qualified datasets does not prove none exist.
The thesis repository contains one original attachment: the thesis PDF. Its metadata and file inventory expose no count matrix. Edinburgh DataShare returned no matches for the author/NMD query. The six GEO author-name matches include GSE152435/6/7 and three unrelated mouse studies; the relevant deposit lists total and subcellular RNA, not the missing nascent fraction.
Required next: Nascent and matched total RNA count matrices, sample-to-file map, labeling/normalization metadata and accession lineage. No restricted HeLa genomic variant data are needed.
A separate Yunnan University deposit lists human HCT116 control/UPF1 time courses at 0, 4, 8 and 12 hours with three replicates. Six UPF1 samples have conflicting time labels: 4h titles versus I8 file descriptions, and 8h titles versus I4 descriptions. The series summary names alpha-amanitin, whereas its design names actinomycin D. Exact depletion duration and matched control assignment also need clarification.
Required next: An authoritative corrected sample sheet, inhibitor identity, depletion/control timing and spike-in normalization information. Confirm sample lineage and read identities before qualification. Do not choose a time assignment by whichever gives a better model score.
All 21 sample metadata records were checked. The 12 RPF entries cover control and ABCE1 depletion; three UPF1-depleted entries are total RNA only. Sequencing lanes are not additional biological replicates.
Required next: A genuinely matched, replicated control/UPF1 RNA-plus-RPF experiment. ABCE1 depletion and ribosomal-RNA depletion (RiboMinus) are not substitutes.
The ENA project record explicitly maps this human UPF1 Ribo-Seq deposit to E-MTAB-13837_2. Its project description and publication lineage identify the Boehm NMDRHT experiments already used in model development. A different archive accession does not make a fresh study.
Required next: A separate study family with eligible RNA/RPF conditions; no recycled NMDRHT accession.
The project title matches the Tani 2012 BRIC-seq study already used through its reprocessed results. The archive contains 25 runs, including siUPF1 and control time courses. Its later archive release date does not establish a new experiment.
Required next: Independent biological measurements, not another deposit or reanalysis of the same experiment.
Primary sourceInspect the six conflicting M4 time labels
Source metadata only; no correction has been guessed.
GEO sample
Title
File description
GSM8508966
HCT116-UPF1,4h-1
I8-1_UPF1_1.fq.gz
GSM8508967
HCT116-UPF1,4h-2
I8-2_UPF1_1.fq.gz
GSM8508968
HCT116-UPF1,4h-3
I8-3_UPF1_1.fq.gz
GSM8508969
HCT116-UPF1,8h-1
I4-1_UPF1_1.fq.gz
GSM8508970
HCT116-UPF1,8h-2
I4-2_UPF1_1.fq.gz
GSM8508971
HCT116-UPF1,8h-3
I4-3_UPF1_1.fq.gz
What unlocks the next benchmark
Public deposits must supply matched outcomes and unambiguous sample metadata before qualification. No author contact is planned.
Resolve specimen/condition/time labels and normalization before estimating targets.
Verify independent study/sample lineage, eligible human response endpoint and adequate biological replication.
Freeze cohort rules, target estimator and model/preprocessing hashes before opening new numerical outcomes.
Evaluate the fixed candidate, ExtraTrees, ridge and training median on identical rows; report all results, coverage, paired intervals and limitations. No holdout tuning or selection of a winning subgroup.
Search limitations and checkpoint records
GSE276918 has the relevant assay in principle, but metadata conflicts prevent a defensible decay-rate fit. Its outcome count table remains unopened.
Metadata lineage checks are preliminary; no raw-read identity audit or fresh-test power assessment has been completed for GSE276918.
Public metadata can omit experiments or use unexpected terms. ENA and GEO results overlap; RiboMinus and RIP-seq must not be counted as RPF.
Elicit API access was unavailable for this account. Direct GEO, ENA and Edinburgh repository searches were used instead.
Hashes of all six existing pooled candidate/baseline checkpoints are in the download. No new model was fitted and no new superiority claim is made.
M4 and M5 v4 · corrected sources and stronger tuning
M5 reference corrected; superiority over both strong baselines is not established.
V4 research models trained and tested on reused development studies. Previously tested studies are now explicitly development data for this revision. No untouched final holdout has been qualified.
M4 retains gene-aligned decay targets and geometry summaries. M5 uses separately deposited RNA and ribosome counts with ambiguous gene mappings excluded; Aliouat now models paired libraries. NMDRHT remains unpaired because its RNA and ribosome libraries came from separate experiments.
Each direction holds out a whole study. The other study supplies an inner gene-disjoint training/validation split and 24 configurations per family. Final research bundles use both development studies; configuration selection uses inner-validation scores, not transfer scores.
M4 · m4-reconciled-response-v4
M4 improves its NMDRHT transfer correlation from 0.434 to 0.450, but falls from 0.442 to 0.435 on Tani. Tuned ExtraTrees still leads both directions.
Train on Tani2012 → evaluate on NMDRHT
7,687 training genes · 10,990 evaluation genes · 7,400 genes shared across studies. Study exclusion does not imply unseen genes.
Retrospective development metrics; not an independent final test.
Method
Spearman ↑
MAE ↓
R² ↑
Reconciled v4 · XGBoost
0.450
0.582
-0.143
Tuned ExtraTrees
0.463
0.572
-0.096
Tuned ridge
0.447
0.566
-0.104
Training median
Undefined · constant
0.619
-0.320
Candidate minus ExtraTrees: -0.013; adjusted paired interval [-0.022, -0.003]. No supported candidate advantage.
Candidate minus ridge: 0.003; adjusted paired interval [-0.009, 0.015]. No supported candidate advantage.
Train on NMDRHT → evaluate on Tani2012
10,990 training genes · 7,687 evaluation genes · 7,400 genes shared across studies. Study exclusion does not imply unseen genes.
Retrospective development metrics; not an independent final test.
Method
Spearman ↑
MAE ↓
R² ↑
Reconciled v4 · XGBoost
0.435
0.664
-0.053
Tuned ExtraTrees
0.480
0.652
-0.025
Tuned ridge
0.404
0.691
-0.103
Training median
Undefined · constant
0.671
-0.117
Candidate minus ExtraTrees: -0.045; adjusted paired interval [-0.058, -0.032]. No supported candidate advantage.
Candidate minus ridge: 0.031; adjusted paired interval [0.019, 0.043]. Positive development difference only.
Alignment, tuning and runtime limits
log2(kdeg depleted / control)
Native HCT116 transcript geometry summaries reused for both studies; not assay-specific isoform composition.
SLAM versus BRIC estimation, cell line and perturbation differences persist.
Same-time point estimate cancels time baseline, but no covariance-aware uncertainty or raw-read refitting was performed.
All source outcomes were previously used or belong to an already-used study.
Precomputed feature-table validation, imputation and inference; excludes sequence extraction, network and response-target estimation. One bundle-load measurement is not cold end-to-end latency. Fixed thread limit: 4. Thirty warm repeats per workload.
Method
Batch
Warm median ms
Warm p95 ms
Reconciled v4 · XGBoost
1
2.741
4.314
Reconciled v4 · XGBoost
32
2.798
4.994
Tuned ExtraTrees
1
31.706
44.183
Tuned ExtraTrees
32
42.294
44.879
Tuned ridge
1
3.017
5.407
Tuned ridge
32
2.951
5.323
1,000 paired gene bootstrap resamples; correlation intervals adjusted across eight comparisons. These intervals are conditional on each study and the selected model. They do not account for earlier study reuse, target-estimation uncertainty, or between-study variability. All search configurations, selected fits and checkpoint hashes are in the download.
The corrected, paired-count M5 candidate still transfers negatively in both directions and loses to ridge. The earlier supplement-derived M5 TE advantage is superseded.
Train on NMDRHT → evaluate on Aliouat2019
4,886 training genes · 5,268 evaluation genes · 4,628 genes shared across studies. Study exclusion does not imply unseen genes.
Retrospective development metrics; not an independent final test.
Method
Spearman ↑
MAE ↓
R² ↑
Reconciled v4 · XGBoost
-0.218
0.405
-0.347
Tuned ExtraTrees
-0.222
0.404
-0.334
Tuned ridge
-0.180
0.385
-0.252
Training median
Undefined · constant
0.338
-0.044
Candidate minus ExtraTrees: 0.004; adjusted paired interval [-0.014, 0.023]. No supported candidate advantage.
Candidate minus ridge: -0.038; adjusted paired interval [-0.065, -0.016]. No supported candidate advantage.
Train on Aliouat2019 → evaluate on NMDRHT
5,268 training genes · 4,886 evaluation genes · 4,628 genes shared across studies. Study exclusion does not imply unseen genes.
Retrospective development metrics; not an independent final test.
Method
Spearman ↑
MAE ↓
R² ↑
Reconciled v4 · XGBoost
-0.291
0.476
-0.160
Tuned ExtraTrees
-0.287
0.476
-0.156
Tuned ridge
-0.266
0.471
-0.145
Training median
Undefined · constant
0.448
-0.057
Candidate minus ExtraTrees: -0.004; adjusted paired interval [-0.030, 0.019]. No supported candidate advantage.
Candidate minus ridge: -0.025; adjusted paired interval [-0.046, -0.003]. No supported candidate advantage.
Alignment, tuning and runtime limits
Common DESeq2 assay-by-condition interaction log2FC, RPF change minus RNA change.
Count quantification and transcript annotation remain source-specific; Aliouat stable transcript IDs map to GENCODE42 then exclude every gene with multiple mapped transcript rows; no effective-length offsets.
NMDRHT acute 12h versus 0h and Aliouat 72h shRNA/control differ; no time-matched untreated NMDRHT RPF arm is established.
Representative transcript features are not condition-specific. Existing feature bank is biased toward prior NMDRHT response availability.
Joint negative-binomial estimator shares dispersion assumptions across assays. Aliouat libraries are paired by source lysate/replicate; NMDRHT remains unpaired because source RNA and RPF experiments differ. Only two Aliouat replicates per condition.
No source significance or response-magnitude filtering.
Precomputed feature-table validation, imputation and inference; excludes sequence extraction, network and response-target estimation. One bundle-load measurement is not cold end-to-end latency. Fixed thread limit: 4. Thirty warm repeats per workload.
Method
Batch
Warm median ms
Warm p95 ms
Reconciled v4 · XGBoost
1
3.563
3.838
Reconciled v4 · XGBoost
32
3.514
4.193
Tuned ExtraTrees
1
29.493
29.982
Tuned ExtraTrees
32
29.509
30.486
Tuned ridge
1
3.930
5.392
Tuned ridge
32
3.720
4.320
1,000 paired gene bootstrap resamples; correlation intervals adjusted across eight comparisons. These intervals are conditional on each study and the selected model. They do not account for earlier study reuse, target-estimation uncertainty, or between-study variability. All search configurations, selected fits and checkpoint hashes are in the download.
M5 source discrepancy · earlier TE claim superseded
Supplement S1 RNA-labelled contrasts closely match deposited RIBOSEQ-derived contrasts, not deposited RNASEQ contrasts. Prior S1 RPF-minus-RNA differences are therefore not a reliable TE reference.
Supersede the earlier M5 v2 TE superiority interpretation. Retain historical results unchanged. V4 uses separately deposited assay counts and paired Aliouat libraries.
Supplement RNA-labelled contrasts versus reconstructed deposited assays
Deposited assay
Contrast
Matched transcripts
Spearman
Median |LFC difference|
RNASEQ
UPF1
13,098
0.37185
0.175547
RNASEQ
eRF3
13,098
0.71525
0.148552
RIBOSEQ
UPF1
13,136
0.99808
0.000262
RIBOSEQ
eRF3
13,136
0.99922
0.000271
The ribosome-count UPF1 reconstruction is within the supplement’s half-rounding unit for 88.6% of matched rows, versus 0.16% for RNA counts. The separate eRF3 contrast corroborates the mismatch.
Strong numerical fingerprint is not author confirmation of a publication error. Raw reads were not re-quantified.
Deposited assay labels are supported by GEO sample metadata; transcript/CDS count regions and upstream quantification may still differ.
M5 cross-study biology and target transportability remain unresolved.
The thesis describes nascent/total RNA stability estimates after UPF1 depletion. Its deposited-data section (printed p94 / PDF p127) specifies only total, membrane and cytoplasmic RNA under GSE152437. All 39 GEO sample titles/metadata were checked; no nascent fraction is listed. Obtain the nascent count matrix and sample metadata, then audit lineage before qualification.
Deposited RNA abundance/fractionation does not supply the required nascent comparison. Longman2020 RNA libraries also appear in the upstream NMDRHT annotation metadata. Any later nascent release requires a sample-level overlap audit.
RNA and ribosome libraries are present, but all six sample metadata records describe control or SR-alpha depletion and immunoprecipitation; no UPF1-depleted RNA/RPF arm. This cannot test the fixed UPF1 TE-response endpoint.
Prioritize obtaining the Jackson-Jones nascent-RNA matrix and its sample lineage for M4. For M5, another matched human control/UPF1 RNA-plus-RPF study is required. Freeze a final model and predictions before reading any new holdout outcomes; do not relabel reused studies as fresh.
Primary-source methods and sample metadata were inspected; candidate numerical outcomes were not opened. This is a bounded search, not proof that no eligible dataset exists. No patentability or therapeutic-efficacy conclusion follows.
Alignment improves the comparison; overall superiority remains unestablished.
Retrospective development only · research checkpoints, not serving defaults. Previously tested studies are now explicitly development data for this revision. No untouched final holdout has been qualified.
M4 now learns directly from gene-level decay responses, with a same-time NMDRHT control contrast and gene summaries of validated geometry. M5 uses the same joint RNA/ribosome count estimator across both studies, with ambiguous transcript-to-gene mappings excluded.
Each direction holds out a whole study. The other study supplies an inner gene-disjoint training/validation split and 12 configurations per family. Final research bundles use both development studies; configuration selection uses inner-validation scores, not transfer scores.
M4 · m4-aligned-response-v3
ExtraTrees remains strongest in both transfer directions. The candidate beats ridge on Tani, but not on NMDRHT.
Train on Tani2012 → evaluate on NMDRHT
7,687 training genes · 10,990 evaluation genes · 7,400 genes shared across studies. Study exclusion does not imply unseen genes.
Retrospective development metrics; not an independent final test.
Method
Spearman ↑
MAE ↓
R² ↑
Aligned v3 candidate
0.434
0.588
-0.195
Tuned ExtraTrees
0.460
0.563
-0.088
Tuned ridge
0.447
0.566
-0.103
Training median
Undefined · constant
0.619
-0.320
Candidate minus ExtraTrees: -0.027; adjusted paired interval [-0.036, -0.019]. No supported candidate advantage.
Candidate minus ridge: -0.014; adjusted paired interval [-0.024, -0.003]. No supported candidate advantage.
Train on NMDRHT → evaluate on Tani2012
10,990 training genes · 7,687 evaluation genes · 7,400 genes shared across studies. Study exclusion does not imply unseen genes.
Retrospective development metrics; not an independent final test.
Method
Spearman ↑
MAE ↓
R² ↑
Aligned v3 candidate
0.442
0.670
-0.065
Tuned ExtraTrees
0.480
0.652
-0.027
Tuned ridge
0.404
0.691
-0.103
Training median
Undefined · constant
0.671
-0.117
Candidate minus ExtraTrees: -0.038; adjusted paired interval [-0.050, -0.027]. No supported candidate advantage.
Candidate minus ridge: 0.038; adjusted paired interval [0.024, 0.052]. Positive development difference only.
Alignment, tuning and runtime limits
log2(kdeg depleted / control)
Native HCT116 transcript geometry summaries reused for both studies; not assay-specific isoform composition.
SLAM versus BRIC estimation, cell line and perturbation differences persist.
Same-time point estimate cancels time baseline, but no covariance-aware uncertainty or raw-read refitting was performed.
All source outcomes were previously used or belong to an already-used study.
Precomputed feature-table validation, imputation and inference; excludes sequence extraction, network and response-target estimation. One bundle-load measurement is not cold end-to-end latency. Fixed thread limit: 4. Thirty warm repeats per workload.
Method
Batch
Warm median ms
Warm p95 ms
Aligned v3 candidate
1
2.440
2.679
Aligned v3 candidate
32
2.401
2.572
Tuned ExtraTrees
1
27.984
28.582
Tuned ExtraTrees
32
27.997
28.869
Tuned ridge
1
2.370
2.630
Tuned ridge
32
2.347
2.616
1,000 paired gene bootstrap resamples; correlation intervals adjusted across eight comparisons. These intervals are conditional on each study and the selected model. They do not account for earlier study reuse, target-estimation uncertainty, or between-study variability. All search configurations, selected fits and checkpoint hashes are in the download.
Negative transfer in both directions. The source discrepancy has since been traced in v4; see the correction above. Do not promote this model.
Train on NMDRHT → evaluate on Aliouat2019
4,886 training genes · 5,268 evaluation genes · 4,628 genes shared across studies. Study exclusion does not imply unseen genes.
Retrospective development metrics; not an independent final test.
Method
Spearman ↑
MAE ↓
R² ↑
Aligned v3 candidate
-0.223
0.416
-0.513
Tuned ExtraTrees
-0.225
0.397
-0.393
Tuned ridge
-0.181
0.383
-0.328
Training median
Undefined · constant
0.332
-0.061
Candidate minus ExtraTrees: 0.001; adjusted paired interval [-0.013, 0.015]. No supported candidate advantage.
Candidate minus ridge: -0.042; adjusted paired interval [-0.069, -0.017]. No supported candidate advantage.
Train on Aliouat2019 → evaluate on NMDRHT
5,268 training genes · 4,886 evaluation genes · 4,628 genes shared across studies. Study exclusion does not imply unseen genes.
Retrospective development metrics; not an independent final test.
Method
Spearman ↑
MAE ↓
R² ↑
Aligned v3 candidate
-0.324
0.481
-0.181
Tuned ExtraTrees
-0.294
0.480
-0.176
Tuned ridge
-0.269
0.471
-0.147
Training median
Undefined · constant
0.448
-0.061
Candidate minus ExtraTrees: -0.030; adjusted paired interval [-0.049, -0.009]. No supported candidate advantage.
Candidate minus ridge: -0.055; adjusted paired interval [-0.079, -0.030]. No supported candidate advantage.
Alignment, tuning and runtime limits
Common DESeq2 assay-by-condition interaction log2FC, RPF change minus RNA change.
Count quantification and transcript annotation remain source-specific; Aliouat stable transcript IDs map to GENCODE42 then exclude every gene with multiple mapped transcript rows; no effective-length offsets.
NMDRHT acute 12h versus 0h and Aliouat 72h shRNA/control differ; no time-matched untreated NMDRHT RPF arm is established.
Representative transcript features are not condition-specific. Existing feature bank is biased toward prior NMDRHT response availability.
Joint negative-binomial estimator shares dispersion assumptions across assays and treats libraries as unpaired; published Aliouat pairing is not modeled; NMDRHT cross-assay sample pairing has not been verified.
No source significance or response-magnitude filtering.
Precomputed feature-table validation, imputation and inference; excludes sequence extraction, network and response-target estimation. One bundle-load measurement is not cold end-to-end latency. Fixed thread limit: 4. Thirty warm repeats per workload.
Method
Batch
Warm median ms
Warm p95 ms
Aligned v3 candidate
1
3.745
4.004
Aligned v3 candidate
32
3.791
4.530
Tuned ExtraTrees
1
29.240
30.396
Tuned ExtraTrees
32
29.296
30.133
Tuned ridge
1
3.778
4.780
Tuned ridge
32
3.726
4.124
1,000 paired gene bootstrap resamples; correlation intervals adjusted across eight comparisons. These intervals are conditional on each study and the selected model. They do not account for earlier study reuse, target-estimation uncertainty, or between-study variability. All search configurations, selected fits and checkpoint hashes are in the download.
New versus published NMDRHT TE ranks: 0.99996 across 5,300 genes. New versus prior derived Aliouat TE ranks: -0.025 across 3,460 genes.
Different estimators and gene/transcript inclusion can change results. Mapping ambiguity was removed before the final diagnostic rerun, but disagreement remains. No sign flip, sample relabeling, response rescaling, or cohort selection was used to improve transfer scores.
Fresh human holdout search · continued
GSE303542
Excluded from primary UPF1 response holdout
Public GEO sample metadata shows human WT/eRF1 perturbation samples; UPF1 knockout samples belong to Neurospora crassa. This is not a matched human UPF1 response study.
Human UPF1/control RNA-seq exists, but the reviewed dataset measures abundance; reported REMBRANDTS analysis infers stability from RNA-seq rather than measuring a paired decay time course. No matched RPF response is established. It must not be substituted for direct decay or TE measurements.
M4: ExtraTrees is the strongest tested family in both transfer directions; retain it as the accuracy reference. M5: reconcile raw-count estimates with published contrasts and pairing metadata before further model tuning. Neither new candidate is promoted.
Targeted extension of the preceding qualification audit; not exhaustive. New candidate outcome files were not read or evaluated. No patentability or therapeutic-efficacy conclusion follows.
New M4 lead found; neither endpoint has a qualified fresh holdout yet.
Qualification audit complete for this search; no new accuracy benchmark run. See the latest human-holdout search above for source metadata and unresolved requirements.
Use directly estimated gene-level decay targets in each study; do not silently substitute the median of transcript predictions or steady-state RNA abundance. Audit control timing, mapping, units and decay-fit uncertainty before admitting a study.
M5 · endpoint contract
Human gene-level UPF1-associated change in translation efficiency.
Use a common replicate-aware RNA/RPF condition-interaction estimator. Keep perturbation duration and cell context explicit. Separate-normalization contrasts and basal TE are not interchangeable endpoints.
Source decisions · 2026-10-13
PRJDB2542 / DRP000622
M4 · Already-used Tani 2012 study family
The project title matches the Tani 2012 BRIC-seq study already used through its reprocessed results. The archive contains 25 runs, including siUPF1 and control time courses. Its later archive release date does not establish a new experiment.
The ENA project record explicitly maps this human UPF1 Ribo-Seq deposit to E-MTAB-13837_2. Its project description and publication lineage identify the Boehm NMDRHT experiments already used in model development. A different archive accession does not make a fresh study.
All 21 sample metadata records were checked. The 12 RPF entries cover control and ABCE1 depletion; three UPF1-depleted entries are total RNA only. Sequencing lanes are not additional biological replicates.
A separate Yunnan University deposit lists human HCT116 control/UPF1 time courses at 0, 4, 8 and 12 hours with three replicates. Six UPF1 samples have conflicting time labels: 4h titles versus I8 file descriptions, and 8h titles versus I4 descriptions. The series summary names alpha-amanitin, whereas its design names actinomycin D. Exact depletion duration and matched control assignment also need clarification.
M4 · Not located in the checked public repositories
The thesis repository contains one original attachment: the thesis PDF. Its metadata and file inventory expose no count matrix. Edinburgh DataShare returned no matches for the author/NMD query. The six GEO author-name matches include GSE152435/6/7 and three unrelated mouse studies; the relevant deposit lists total and subcellular RNA, not the missing nascent fraction.
RNA and ribosome libraries are present, but all six sample metadata records describe control or SR-alpha depletion and immunoprecipitation; no UPF1-depleted RNA/RPF arm. This cannot test the fixed UPF1 TE-response endpoint.
Deposited RNA abundance/fractionation does not supply the required nascent comparison. Longman2020 RNA libraries also appear in the upstream NMDRHT annotation metadata. Any later nascent release requires a sample-level overlap audit.
M4 · Promising endpoint; required data not located
The thesis describes nascent/total RNA stability estimates after UPF1 depletion. Its deposited-data section (printed p94 / PDF p127) specifies only total, membrane and cytoplasmic RNA under GSE152437. All 39 GEO sample titles/metadata were checked; no nascent fraction is listed. Obtain the nascent count matrix and sample metadata, then audit lineage before qualification.
Used for development. This revision aligns M4 to gene-level same-time decay contrasts and re-estimates M5 from counts. Assay, cell, perturbation and M5 control-timing differences remain.
These papers share the same BRIC-seq measurements and count as one study family. Previously tested; now explicitly admitted to retrospective development for aligned v3. Never a fresh final holdout again.
Previously tested and used for development. V4 traced the supplement/count source conflict and superseded the earlier TE interpretation. The corrected-count model still transfers negatively; this study cannot be a fresh holdout.
Paper methods describe basal HEK293 half-lives and separate UPF1-depletion RNA abundance. A genome-wide paired UPF1/control decay-rate endpoint has not been established from the reviewed material.
Published methods list HeLa ribosome profiling and separate UPF1-knockdown total RNA used to identify NMD-sensitive isoforms. Matched depleted/control ribosome profiling is not established. GEO sample verification was limited by access errors.
Mouse embryonic stem cells with paired RNA/RPF conditions. GEO lists only one control-depleted and one Upf1-depleted RPF sample. Species shift and no replicated RPF contrast prevent qualification for the primary human benchmark. Outcome tables remain unread in this audit.
Original qualification plan · development progress appears above
Completed: v4 alignment, source reconciliation and equal-budget development comparisons. Neither model establishes superiority over both strong baselines. Latest qualification audit found a new M4 time-course lead with unresolved sample labels.
Qualify a third, untouched human study per endpoint: matched control/UPF1 response, compatible measurements, replicate metadata, identifier coverage and accession/sample lineage. No final study is reserved yet.
Completed: two study-held-out development directions with equal tuning budgets. With only two study families, these are retrospective diagnostics rather than independent confirmation.
Freeze the model, preprocessing, inclusion rules and analysis protocol before opening final outcomes. Publish one final evaluation including all failures and missing rows.
Proposed pass criteria and runtime protocol
Primary accuracy: positive multiplicity-adjusted paired confidence bounds for Spearman improvement over both tuned ExtraTrees and ridge. Declare all model/baseline comparisons together before evaluation.
Absolute error: demonstrate lower MAE than the training-only median baseline using a paired interval. Report MAE, R², coverage and missingness for every method.
Power and relevance: specify a meaningful minimum gain and assess sample size using development evidence before accepting a final cohort. No significance guarantee or post-hoc winning subgroup.
Runtime: measure the same complete input-to-output path for all models on fixed hardware and thread settings. Separate cold startup, warm median/p95, preprocessing, memory and batch sizes; report accuracy/runtime trade-offs.
Claim scope: a single-study win supports that cohort and endpoint only. Gene-level intervals do not establish across-study certainty, therapeutic efficacy or patentability.
Targeted public-source qualification, not an exhaustive proof that no suitable dataset exists. Article narratives and assay metadata were reviewed; new candidate outcome tables were not downloaded or scored. No blind third-party custody has been established.
No new superiority result was produced by this qualification step.
The current v2 results and checkpoints remain unchanged.
An independent study may share genes with development; report study independence and gene independence separately.
Raw-read identity deduplication and complete accession lineage audits remain required for any admitted final study.
Historical response v2 · M5 TE interpretation superseded
Historical v2 results: M4 did not beat both baselines. The earlier M5 TE interpretation is superseded by the source reconciliation below; it no longer supports TE superiority.
M5 correction: Supplement S1 RNA-labelled contrasts closely match deposited RIBOSEQ-derived contrasts, not deposited RNASEQ contrasts. Prior S1 RPF-minus-RNA differences are therefore not a reliable TE reference. View corrected-count evidence.
New candidates use shallower trees, stronger regularization and an absolute-error objective option. Each model family received 12 development-only fits on identical rows. Checkpoints and external predictions were frozen before examining numerical test responses.
No improvement established. ExtraTrees has higher correlation; the prior M4 also has a higher point estimate. Keep v2 experimental.
Independent-study accuracy with gene-bootstrap intervals; higher correlation and lower absolute error are better.
Method
Spearman · 95% interval
MAE · lower is better
R²
New response v2
0.366 [0.3459, 0.3850]
0.722
-0.184
Tuned ExtraTrees
0.376 [0.3576, 0.3955]
0.716
-0.171
Tuned ridge
0.371 [0.3526, 0.3919]
0.712
-0.149
Previous response v1
0.372 [0.3520, 0.3908]
0.719
-0.178
Training median
Undefined · constant
0.680
-0.131
V2 minus Tuned ExtraTrees: -0.011, adjusted paired interval [-0.0174, -0.0038]. Baseline has higher correlation on this study.
V2 minus Tuned ridge: -0.005, adjusted paired interval [-0.0181, 0.0077]. Difference is inconclusive.
2,000 paired gene bootstrap resamples. Comparison intervals adjust across all four M4/M5 strong-baseline comparisons. These quantify uncertainty within the study, not across laboratories. Negative R² and MAE worse than the training median limit practical use.
Study mapping, unseen genes and development selection
Unique exact gene symbol; median prediction across native NMDRHT isoforms. External measurement combines isoforms.
8,426 published rows; 7,687 mapped; 0 mapped rows excluded for missing or invalid responses. No significance or effect-size filtering. M4 refuses ambiguous or non-text gene symbols, including spreadsheet date conversions. M5 requires the selected gene/transcript pair to match.
7,435 evaluated genes overlap development partitions. 252 do not. This primary comparison tests new study measurements, not wholly new genes.
Descriptive correlation on genes absent from development: v2 0.495, ExtraTrees 0.470, ridge 0.475. This stratum was not used to select a winning cohort.
Development correlation: v2 0.540, ExtraTrees 0.538, ridge 0.513; previous response v1 0.541. Development results are selection scores, not independent accuracy estimates.
Selected candidate: {"objective":"regression","num_leaves":15,"reg_lambda":10}. Every tuning fit is included in the evidence download.
Superseded as TE evidence: the supplement RNA-labelled columns closely match deposited ribosome counts. Historical numerical results are retained for audit, not as a current superiority claim.
Independent-study accuracy with gene-bootstrap intervals; higher correlation and lower absolute error are better.
Method
Spearman · 95% interval
MAE · lower is better
R²
New response v2
0.072 [0.0395, 0.1029]
0.216
-3.040
Tuned ExtraTrees
0.028 [-0.0067, 0.0614]
0.357
-11.480
Tuned ridge
0.056 [0.0243, 0.0890]
0.277
-5.423
Previous response v1
0.035 [0.0003, 0.0685]
0.359
-13.394
Training median
Undefined · constant
0.153
-0.979
V2 minus Tuned ExtraTrees: 0.044, adjusted paired interval [0.0070, 0.0827]. Historical statistic only; TE interpretation superseded.
V2 minus Tuned ridge: 0.016, adjusted paired interval [-0.0230, 0.0535]. Historical statistic only; TE interpretation superseded.
2,000 paired gene bootstrap resamples. Comparison intervals adjust across all four M4/M5 strong-baseline comparisons. These quantify uncertainty within the study, not across laboratories. Negative R² and MAE worse than the training median limit practical use.
Study mapping, unseen genes and development selection
Exact stable gene and stable transcript pair; transcript versions stripped, no fuzzy joins.
13,137 published rows; 3,572 mapped; 1 mapped rows excluded for missing or invalid responses. No significance or effect-size filtering. M4 refuses ambiguous or non-text gene symbols, including spreadsheet date conversions. M5 requires the selected gene/transcript pair to match.
3,290 evaluated genes overlap development partitions. 281 do not. This primary comparison tests new study measurements, not wholly new genes.
Descriptive correlation on genes absent from development: v2 0.092, ExtraTrees -0.038, ridge 0.026. This stratum was not used to select a winning cohort.
Development correlation: v2 0.303, ExtraTrees 0.267, ridge 0.209; previous response v1 0.231. Development results are selection scores, not independent accuracy estimates.
Selected candidate: {"objective":"regression_l1","num_leaves":7,"reg_lambda":100}. Every tuning fit is included in the evidence download.
Response definition: Published UPF1 Ribo-seq log2FC minus published UPF1 RNA-seq log2FC.
72-hour shRNA depletion versus acute 12-hour depletion in training, both HCT116.
TE change derived from separately normalized DESeq2 contrasts; not a joint TE model or the NMDRHT interaction estimate.
Selected published table and strict transcript matching limit coverage. Transcript version changes remain unresolved.
Correction dated 2026-10-13: the M5 supplement-derived TE endpoint is unreliable; its earlier positive baseline comparison is superseded. See reconciled v4 for corrected-count results.
Independent study measurements were excluded from fitting, tuning and checkpoint selection. Predictions were frozen before numerical response evaluation. These studies are now used test sets and cannot be reused as fresh holdouts.
Most evaluated genes occur in NMDRHT development partitions. The primary test measures transfer between studies, not generalization to unseen genes. Unseen-development-gene results are reported separately and descriptively.
M4 uses gene-level HeLa BRIC-seq outcomes and median native transcript predictions; this is a transfer stress test with isoform and assay mismatch. M5 uses exact stable gene/transcript pairs, but versions can differ.
M5 TE changes are derived by subtracting two published DESeq2 contrasts. This is not the same joint TE estimator as NMDRHT; 72-hour knockdown and acute 12-hour depletion also differ.
All v2 R² values are negative and both models have worse MAE than the training-median baseline. A statistically detectable ranking difference is not strong absolute prediction.
NMDRHT decay training compares UPF1 depletion at 12 hours with control at 0 hours. Time effects and source-annotation ascertainment remain potential confounders.
The M5 candidate improves development correlation. M4 v2 does not beat its parent on development or external correlation. Existing defaults are preserved.
Intervals resample genes within each published study; they do not quantify uncertainty across laboratories, perturbations or independently repeated studies. No therapeutic-efficacy or patentability conclusion follows.
Source audit: M4 DRA000591 (Tani2012), reprocessed in Imamachi2017; one external assay, not two independent studies. M5 GSE126385 (Aliouat2019). Publication, accession and experiment-table audit; raw read identity deduplication was not performed.
New M4 and M5 regressors predict measured response changes in the published UPF1-depletion context. These endpoints differ from annotation classification and basal translation efficiency. Chr8 genes were reserved before fitting; every isoform of a test gene stays in test.
Target: Source-specific log2 fold change in degradation rate · L2FC_kdeg_tx. These scores are changes, not NMD probabilities or absolute translation rates.
7,597 training genes · 1,886 validation genes · 356 held-out chr8 genes / 576 measured transcripts. No train/test gene overlap.
Measured-response Spearman 0.530. Higher than the fitted 50-nt rule; no established superiority over tuned ExtraTrees or ridge.
Method
Spearman · gene-bootstrap 95% interval
MAE · lower is better
R²
response model
0.530 [0.4486, 0.5987]
0.387
0.309
tuned ExtraTrees
0.532 [0.4514, 0.5961]
0.382
0.335
tuned ridge
0.490 [0.4044, 0.5609]
0.399
0.270
training median
Not defined · constant
0.452
-0.003
fitted 50-nt rule
0.232 [0.1465, 0.3126]
0.452
0.069
Response model minus tuned ExtraTrees: -0.002, adjusted paired interval [-0.0227, 0.0229]. No established superiority.
Response model minus tuned ridge: 0.040, adjusted paired interval [-0.0037, 0.0900]. No established superiority.
Response model minus fitted 50-nt rule: 0.298, adjusted paired interval [0.1808, 0.4012]. Higher correlation on this cohort.
1,000 whole-gene paired bootstrap resamples, with comparison intervals adjusted within this model’s declared baseline family. Each trainable family received six validation-selected fits on identical rows; M4 training weights balance genes with multiple isoforms. 578 of 1154 reserved test inputs lacked a finite response and were excluded under the declared rule, without significance or response-magnitude filtering.
Mapping scope and all tuning fits
Exact native transcript/gene/coordinates and exon/CDS/UTR lengths, with independently reconstructed source stop-end distance · 34,455 source transcripts pass native geometry validation. Non-coding annotations and inconsistent endpoint geometry are refused.
Family and configuration
Validation Spearman
response model · leaves=15, L2=1
0.541
response model · leaves=15, L2=10
0.541
response model · leaves=31, L2=1
0.531
response model · leaves=31, L2=10
0.533
response model · leaves=63, L2=1
0.519
response model · leaves=63, L2=10
0.521
tuned ExtraTrees · leaf=2, fraction=0.5
0.528
tuned ExtraTrees · leaf=2, fraction=1
0.525
tuned ExtraTrees · leaf=5, fraction=0.5
0.538
tuned ExtraTrees · leaf=5, fraction=1
0.512
tuned ExtraTrees · leaf=10, fraction=0.5
0.538
tuned ExtraTrees · leaf=10, fraction=1
0.505
tuned ridge · alpha=0.1
0.513
tuned ridge · alpha=1
0.513
tuned ridge · alpha=10
0.513
tuned ridge · alpha=100
0.512
tuned ridge · alpha=1000
0.508
tuned ridge · alpha=10000
0.500
M5 · measured translation-efficiency response
Target: Source-specific log2 fold change in translation efficiency · L2FC_TE. These scores are changes, not NMD probabilities or absolute translation rates.
4,397 training genes · 1,102 validation genes · 186 held-out chr8 genes / 186 measured genes. No train/test gene overlap.
Measured TE-response Spearman 0.265. No established superiority over tuned ExtraTrees or ridge. Absolute MAE is worse than the training-median predictor and R² is negative; predictions remain exploratory.
Method
Spearman · gene-bootstrap 95% interval
MAE · lower is better
R²
response model
0.265 [0.1151, 0.3942]
0.561
-0.010
tuned ExtraTrees
0.332 [0.1949, 0.4642]
0.556
0.020
tuned ridge
0.211 [0.0841, 0.3429]
0.574
-0.018
training median
Not defined · constant
0.545
-0.012
Response model minus tuned ExtraTrees: -0.067, adjusted paired interval [-0.1712, 0.0353]. No established superiority.
Response model minus tuned ridge: 0.054, adjusted paired interval [-0.1014, 0.2121]. No established superiority.
1,000 whole-gene paired bootstrap resamples, with comparison intervals adjusted within this model’s declared baseline family. Each trainable family received six validation-selected fits on identical rows; M4 training weights balance genes with multiple isoforms. 474 of 660 reserved test inputs lacked a finite response and were excluded under the declared rule, without significance or response-magnitude filtering.
Mapping scope and all tuning fits
Exact GENCODE.v42 gene/transcript FASTA and header UTR/CDS positions; representative longest CDS then transcript ID, before outcome access · 19,359 genes map to eligible GENCODE.v42 sequence inputs. One representative is selected by longest CDS, then transcript ID, independently of outcomes.
Family and configuration
Validation Spearman
response model · leaves=15, L2=1
0.223
response model · leaves=15, L2=10
0.226
response model · leaves=31, L2=1
0.193
response model · leaves=31, L2=10
0.231
response model · leaves=63, L2=1
0.204
response model · leaves=63, L2=10
0.215
tuned ExtraTrees · leaf=2, fraction=0.5
0.245
tuned ExtraTrees · leaf=2, fraction=1
0.157
tuned ExtraTrees · leaf=5, fraction=0.5
0.258
tuned ExtraTrees · leaf=5, fraction=1
0.125
tuned ExtraTrees · leaf=10, fraction=0.5
0.248
tuned ExtraTrees · leaf=10, fraction=1
0.125
tuned ridge · alpha=0.1
0.208
tuned ridge · alpha=1
0.207
tuned ridge · alpha=10
0.205
tuned ridge · alpha=100
0.201
tuned ridge · alpha=1000
0.197
tuned ridge · alpha=10000
0.180
Qualification and limits
Gene/chromosome holdout within one published experiment; no independent-lab or independent-study confirmation. The published workbook was previously inspected for schema, counts and five example rows; this is a new withheld-response experiment, not a never-downloaded dataset.
Native NMDRHT assembly uses control/depleted RNA samples; chromosome holdout does not remove assay or transcript-discovery ascertainment effects. Experimental turnover changes are estimated from assay data and do not establish an intrinsic NMD probability or therapeutic rescue.
The author EZbakR code compares the N-AID-UPF1 12-hour state with control at 0 hours. This published contrast can include time/background effects; it is not an isolated, time-matched causal NMD effect.
M5 input partitions were frozen before response processing. A global finite-response availability mask was computed during development feature preparation, but an explicit test-inclusion branch retained every reserved test gene, including all 474 missing-target rows. Test numerical responses were never supplied to fitting or selection. This parsing is disclosed; not represented as never-parsed outcome bytes.
M5 RNA/RPF response predictors are excluded because they define the TE response algebraically. M4 annotation-derived NMD labels, expression responses and assay outcomes are excluded from predictors. Missing source predictors use training-only medians.
Six training-only fits each for M5, ExtraTrees and ridge, selected on the same HeLa validation genes. The 525 finite K562 outcomes were first read after all models and predictions were frozen. All test genes exclude earlier M5 development and evaluation.
No accuracy superiority: M5 v3 0.363, ExtraTrees 0.394, ridge 0.400. V3 is effectively unchanged from v2. It remains a separate experimental option.
2,000 paired gene-bootstrap resamples; accuracy comparison intervals adjust across all three baselines. 75 of 600 frozen test genes lack K562 outcomes and were excluded under the declared finite-outcome rule. Listed metadata study indices and sample aliases are disjoint from HeLa/BJ; some GEO study names are missing. The compendium pipeline is shared, and uncertainty is conditional on the aggregate.
Inspect all 18 tuning fits and feature ablations
Same training/validation genes and six configuration evaluations per family. These are equal fit counts, not equal compute time. M5 compares full context, UTR-only and core context; test outcomes never select a configuration.
Measured latency advantage over ExtraTrees on these local workloads, with a lower accuracy point estimate. Accuracy non-inferiority is not established. No consistent full-runtime gain over v2 is demonstrated; ridge remains competitive. Folding and feature extraction are recomputed on every call; the earlier inference-only timings do not represent this full cost.
Method
Batch
Full pipeline median / p95 ms
HTTP median / p95 ms
Fresh process wall seconds
M5 v3
1
20.301 / 23.123
16.854 / 18.245
1.589
tuned ExtraTrees
1
39.344 / 43.822
30.682 / 33.942
2.213
tuned ridge
1
20.925 / 32.320
17.466 / 20.186
1.566
M5 v2
1
20.423 / 21.770
16.834 / 18.495
1.482
M5 v3
16
334.234 / 412.816
331.901 / 339.471
1.551
tuned ExtraTrees
16
351.540 / 409.743
349.248 / 354.296
1.728
tuned ridge
16
334.588 / 377.292
331.220 / 339.906
1.512
M5 v2
16
336.040 / 430.519
333.194 / 341.363
1.775
Batch 1, full-pipeline speedup versus ExtraTrees: 1.92×, paired 95% interval [1.904, 1.981]. These timing intervals are per comparison and are not adjusted as an overall superiority test.
Batch 16, full-pipeline speedup versus ExtraTrees: 1.05×, paired 95% interval [1.035, 1.062]. These timing intervals are per comparison and are not adjusted as an overall superiority test.
20 rotating-order warm rounds per workload, same real source transcripts and identical single-thread BLAS controls. Thread settings were selected on development inputs only: one thread for every single-item tree call; for batch 16, M5 uses four threads and ExtraTrees uses one. Every method has the same maximum four-thread budget. Full pipeline includes request JSON parsing, sequence validation, RNA folding, features, inference and result serialization. HTTP adds client encoding/decoding, request validation and local transport. Fresh-process wall time also includes interpreter/import startup and model load; it is one first-use observation per method/workload, not a confidence estimate. Pipeline and HTTP are measured in separate runs; compare methods within each run, not their absolute medians across runs. Timing uncertainty is conditional on this session, machine and workload.
M4 · earlier experimental dataset discovery
These cards preserve the earlier source-discovery assessment. Separate measured-response models and chromosome-held-out benchmarks now appear above; their endpoints differ from the historical annotation classifier.
Rapid UPF1 depletion / NMDRHT
Processed experimental response tables downloaded and inspected
SLAM-seq transcript degradation-rate change with adjusted p-value; transcript expression responses also available
45,757 transcript rows; 16,515 finite turnover changes with adjusted p-values; 13,678 of those map to a named GENCODE transcript.
Match assay condition, assembly, transcript coordinates, complete CDS/stop geometry and replicate structure. Current M4 is an exon annotation classifier, not a turnover-response predictor. Annotation-derived NMD flags in the same table cannot be experimental gold labels.
Next: Prefer measured turnover outcomes, reserve genes and study conditions before development, and define a separate experimental-response model.
Public numerical DMS files downloaded; assay/geometry qualification pending
Reported variant-reporter fitness values and sigma uncertainty, with stop-position/context fields
PE 453, LE 981, SP 10,666 CSV rows. Counts are assay-table rows, not independent genes or biological replicates.
Resolve normalization, controls, transcript/genome mappings, gene overlap and shared construct families. These are variant-reporter outcomes, not current M4 exon annotation labels. Author README says raw TCGA/GTEx variant sets are not shared in this distribution.
Next: Audit the reporter measurements and uncertainty; group splits by gene and construct family before any fitting.
Public metadata verified; numerical download returned HTTP 401
Transcript abundance responses to UPF1 knockdown, with combined transcript GTF and all-isoform response table listed
Repository lists a 16.3 MB all-isoform response table and a 223.6 MB combined GTF. Numeric response rows were not retrieved here.
Access to files and hg19-to-current transcript mapping. The preselected NMD-target list requires a 50-nt geometry rule, so treating that list as independent biological truth would be circular. UPF1 response can include indirect effects.
Next: Obtain the unfiltered response measurements and replicates through normal repository access; define assay-based outcomes independently of geometry.
Separately retrained M4 and M5 · fresh transfer tests
New checkpoints, frozen holdout definitions and matched baseline comparisons. Selection used development validation only. Historical checkpoints and results are preserved.
Per-candidate containing transcripts, correct strand-local exon indices, UTR-inclusive spliced stop coordinates and complete split stop codons. Excludes 28,935 human training/validation candidates and 15,739 mouse test candidates without complete containing annotation; accepted human and mouse rows have zero invalid position fractions.
9,866 training genes · 2,467 validation genes · 11,718 fresh test genes · 275,329 test exons. No training/test gene-ID overlap.
AP 0.9993 on fresh mouse annotations. Higher than the declared rule and logistic baselines; no established improvement over legacy M4 or ExtraTrees. This is annotation transfer, not experimental NMD accuracy.
Method
Average precision / PR-AUC
Revised model minus method
Adjusted paired interval
M4 geometry v2
0.99926
Reference
—
legacy M4 on corrected inputs
0.99904
0.00021
[-0.00023, 0.00060]
training prevalence
0.56820
0.43105
[0.42428, 0.43769]
50-nt annotation rule
0.99653
0.00273
[0.00203, 0.00337]
55-nt rule
0.99617
0.00308
[0.00245, 0.00381]
scaled logistic
0.99874
0.00051
[0.00004, 0.00095]
ExtraTrees
0.99941
-0.00016
[-0.00057, 0.00021]
1,000 paired gene-bootstrap resamples. Intervals adjust for every declared comparison within this model’s baseline family. Positive intervals support a higher score on this cohort; an interval crossing zero does not establish superiority.
Inference on matched batch of 128
Warm median ms
Warm p95 ms
m4-transcript-geometry-v2
0.180
0.217
legacy M4
0.295
0.335
Paired inference speedup versus legacy: 1.65×, 95% interval [1.61042, 1.69778].
30 warm local timings, four inference threads. These measure model inference on precomputed features; sequence folding, feature extraction, network and startup are excluded.
Mouse distribution shift
Shared annotation methodology and possible orthology; not independent annotation provenance
Labels and stop geometry share their definition
No experimental NMD or therapeutic efficacy claim
Revised M4 scores are uncalibrated annotation-class scores. Human annotation development was already examined historically; the fresh test is mouse.
Trained on HeLa using uORF features plus main-ORF Kozak context, transcript lengths, normalized folding energy and sequence motifs. BJ study families are disjoint from HeLa; test genes also exclude earlier M5 development and the examined HeLa holdout.
2,935 training genes · 787 validation genes · 759 fresh test genes. No training/test gene-ID overlap.
Spearman 0.603 on 759 fresh BJ genes. Higher than legacy M5 (0.217) and Kozak-only (0.418); no established superiority over all-feature ridge or ExtraTrees.
Method
Spearman correlation
Revised model minus method
Adjusted paired interval
M5 context v2
0.60306
Reference
—
legacy M5
0.21680
0.38626
[0.29152, 0.48915]
Kozak-only ridge
0.41792
0.18514
[0.09905, 0.27031]
UTR length/GC ridge
0.24862
0.35444
[0.27475, 0.44306]
all-feature ridge
0.57170
0.03135
[-0.00592, 0.06625]
ExtraTrees
0.60273
0.00032
[-0.02150, 0.01912]
1,000 paired gene-bootstrap resamples. Intervals adjust for every declared comparison within this model’s baseline family. Positive intervals support a higher score on this cohort; an interval crossing zero does not establish superiority.
Inference on matched batch of 128
Warm median ms
Warm p95 ms
m5-sequence-context-v2
0.221
0.257
legacy M5
0.327
0.375
Paired inference speedup versus legacy: 1.50×, 95% interval [1.45789, 1.66749].
30 warm local timings, four inference threads. These measure model inference on precomputed features; sequence folding, feature extraction, network and startup are excluded. Full feature extraction took 205.5 seconds for 4,600 transcripts. 41 of 800 frozen test genes lacked BJ outcomes and were excluded under the declared finite-outcome rule.
Published aggregate, not independent lab execution
Same compendium processing pipeline
Conditional gene-bootstrap uncertainty; no per-study response rows
No therapeutic construct or patient outcome claim
Training source and representation both changed; gains cannot be attributed solely to added features. BJ aggregate processing shares the published pipeline.
973 evaluable genes, excluding all 2,500 development genes. The declared source/sample audit covers 19 HeLa GEO studies with no overlap with M5 development or its earlier transfer cohort. Predictions and baseline definitions were frozen before reading HeLa measurements; source-audit completion followed evaluation.
M5 loses to Kozak-only and beats length/GC. Overall accuracy superiority is not established.
Method
Spearman
Gene-bootstrap 95% interval
M5
0.322
[0.262, 0.374]
Kozak-only linear
0.433
[0.377, 0.486]
UTR length/GC ridge
0.171
[0.108, 0.231]
M5 minus Kozak-only linear: -0.110, adjusted paired interval [-0.186, -0.042]. No superiority over this baseline.
M5 minus UTR length/GC ridge: 0.151, adjusted paired interval [0.079, 0.224]. Higher on this cohort.
Intervals are adjusted for two baseline comparisons. Uncertainty is conditional on the published HeLa aggregate; study-specific response rows are unavailable. Cell conditions and transcript selection differ from training. This measures endogenous-gene TE ranks, not therapeutic construct efficacy.
1,851 held-out genes and 31,608 exons from the existing GENCODE test partition. Training, calibration and test genes are disjoint. This partition was previously examined; the comparison is retrospective, with new baselines fitted only on training genes.
21,508 of 53,116 historical test rows were excluded by the deployed input contract. The historical builder can use mismatched transcripts and invalid position fractions. These legacy weights were not retrained; separate corrected checkpoints and fresh transfer tests appear above. These scores are a legacy diagnostic.
Method
Average precision / PR-AUC
Brier loss · lower is better
M4
0.937
0.087
Logistic same geometry
0.906
0.114
55-nt stop rule
0.548
0.364
Training prevalence
0.441
0.247
M4 minus Logistic same geometry: 0.031, adjusted paired interval [0.026, 0.037].
Scores are higher than these baselines on the restricted legacy annotation subset. The geometry quality gate fails, so this is not promoted as scientific superiority. Annotation geometry shares information with annotation labels. It does not establish fresh external superiority, experimentally measured NMD, ASO potency or protein rescue.
Prediction runtime · M4 batch improvement
One model call now handles all valid batch rows. Identical weights, input membership and returned results; 30 paired warm timings per workload on this machine. Times include local service validation/inference, exclude network and process startup. Single-item timing does not improve.
Batch
Before median ms
After median ms
After p95 ms
Speedup and paired 95% interval
1
0.057
0.058
0.070
0.971× [0.924, 1.021]
16
0.920
0.276
0.308
3.331× [3.198, 3.620]
128
4.302
1.223
1.292
3.517× [3.459, 3.616]
This compares the previous and current serving implementations. It is not a model-accuracy, competing-platform latency or patentability result. Thread-limit experiments were slower for these workloads and were not adopted.
Available prediction stages · real public references
Six calls per stage: first-call time and five warm samples. Workloads and stages differ; these timings cannot rank competing model speeds. Calibration-only and stored-index checks exclude sequence inference.
Model and measured stage
First call seconds
Warm median seconds
Warm p95 seconds
M1
Fresh sequence features and shipped-model inference, two public Huesken rows
1.040
0.127
0.129
M2
Supported-chemistry scoring, two public APP records
0.027
0.002
0.002
M4
Feature-vector scoring, two public rows
0.009
0.001
0.001
M5
Feature-vector scoring, two public rows
0.003
0.001
0.002
M3
Calibration only on two recorded ClinVar raw-score rows; excludes SpliceAI sequence inference
Five-fold pinned feature reference replay, includes repeated fold loading
0.351
0.037
0.039
Recovered exp12
Five-fold pinned feature reference replay, includes repeated fold loading
0.124
0.063
0.063
Recovered pooled-v4
Five checkpoints on three public training-reference rows; includes fold loading
1.302
0.773
0.841
Reconstructed KRAIG + OligoAI
Three supported public APP records; includes live constituent scoring
13.779
4.799
9.247
M6: Coverage calibration needs a qualified independent potency cohort; no standalone inference route.
Original KRAIG / SAR-FiLM: Required deployment artifacts unavailable.
M7 fresh sequence: Fresh encoder readiness and sequence inference are separate from the tested stored-index path.
Earlier dataset qualification and remaining evidence gaps
Unavailable or unqualified comparisons remain visible. Models with different endpoints cannot share a single accuracy leaderboard.
M1 current
No new qualified independent cohort
Sciabola is already development source simone; correct-strand core audit finds 160 matches. Davis uses incompatible 20/50 inputs and modified constructs.
Next evidence: A source-family-disjoint panel with measured efficacy, documented guide orientation and genuine 19/57 inputs.
22,536/22,845 MOE and 36,121/36,428 cEt descriptor rows overlap development sequences. WFDC1 eight-compound figure lacks exact sequence/assay rows.
Next evidence: Source-backed WFDC1 constructs, positional chemistry, context, complete assay membership and numerical measurements; then freeze predictions before opening outcomes.
Requires variant-level ClinVar calibration-overlap audit, matching reference context and verified raw SpliceAI predictions. Minigene disruption differs from clinical labeling used in calibration.
Next evidence: Qualify variant identities and labels; compare raw versus calibrated SpliceAI on a separately defined experimental endpoint.
21,508 of 53,116 test rows fail the deployed input contract. Historical builder uses union indices and mismatched representative transcripts. Higher scores on the valid subset do not resolve this source defect.
Next evidence: Corrected v2 and separate measured-response models are tested above. Independent-study transfer tests are shown above; superiority over both strong baselines and therapeutic outcomes remain unestablished.
Qualified external gene-disjoint transfer evaluated
973 evaluable unseen-development genes. M5 beats length/GC, loses to Kozak-only. Study-specific uncertainty cannot be recovered from the aggregate table.
Next evidence: Separate v2 is tested on BJ above. Per-study response rows and additional independent cell contexts are still needed.
New coverage benchmark blocked by qualified potency inputs
Coverage requires genuinely unseen model predictions, distinct calibration/test groups and complete assay membership. Historical outcome-derived coverage is not a deployment guarantee.
Next evidence: Qualify enough independent RNase-H groups, freeze calibration and test partitions, and evaluate both coverage and selected-set size.
Sequence-file availability or stored human-index replay does not establish OOD detection accuracy. Novel-gene labels alone are not applicability ground truth.
Next evidence: Define intended in/out-of-support labels before scoring and exclude stored index membership; verify fresh encoder execution.
LNA candidate panels are incompatible; ASO-RASAR development collections overlap. Global OligoAI source membership must also be checked for each candidate.
Next evidence: Training-disjoint supported complete assay sets with documented chemistry/context, then matched prediction locks and paired study-family evaluation.