SPIRAL2 LLRF · EPICS precursor + root-cause atlas

Where does each fault type actually show a precursor?

Full in-era population (845 fault + 571 Normal events, no subsampling), split three ways: pooled across all cryomodules, CMA vs. CMB side, and per individual cryomodule — wherever a slice has at least 15 fault events. Four independent methods per slice: a confound-matched unsupervised distance (paired against a same-module/same-side Normal control via optimal assignment), a temporally-validated classifier (block CV, held out exactly once), a nonlinear VAE latent-space classifier evaluated separately at four discrete 1-hour windows, and an LSTM with attention over the full 16-step sequence. Click any row for the full breakdown.

Headline findings

Curated virtual observer: fewer channels, same or better signal

Take the LSTM's top-8 channels by permutation importance (per fault type) and retrain a smaller model on just those — does curation cost accuracy, or was most of the full 23-channel ensemble noise?

fault type	full AUC (23 ch.)	curated AUC (8 ch.)	Δ	curated channels

Fault types with no usable EPICS data

Not a statistical-power judgment — these fault types occurred almost entirely before the archiver had coverage, at any window choice.

Explore by fault type

scope	n fault	n normal	classifier AUC	matched-pair median	wilcoxon p	frac. positive

Method and honesty notes

Minimum 15 fault events required for any slice to be attempted — smaller slices are simply not shown, not estimated anyway. 31 of a possible ~133 (fault type × scope) combinations clear this bar.
The aggregate matched-pair distance is sensitive to which Normal pool defines the reference model — re-fitting on the full 571-event Normal pool shifted the aggregate significance for one fault type even though the underlying per-channel signal was essentially unchanged. Per-channel breakdowns are the more stable evidence; treat the single aggregate p-value as a screening signal, not the final word.
Classifier AUCs from very small slices (roughly n<30) can swing to extreme values (nearly 0 or 1) from block-CV instability alone — flagged with ⚠ where n is small and the AUC is extreme. Extreme + small-n is a coin flip that happened to land hard, not necessarily a stronger effect than a well-powered 0.75.
Matched-control model is fit once on all Normal events and also serves as the matching pool (no separate held-out split for this part) — a deliberate simplification to keep enough data for the per-module slices; the classifier (approach A) is unaffected and remains fully temporally held out.

Glossary

AUC (area under the ROC curve): How well the classifier separates fault from Normal, from 0 to 1. 0.5 = coin flip, 1.0 = perfect separation, below 0.5 means it's systematically backwards. 0.7–0.85 is a real, useful signal for this kind of noisy operational data — treat anything near 1.0 from a small slice with suspicion (see the small-n caution above), not extra confidence.
Matched-pair median: For each fault event, we found a Normal event from the same (or side-matched) cryomodule close in time, and took the fault event's anomaly-distance score minus its matched partner's. The median of those differences across all pairs. Positive = the fault event looked more unusual than its matched Normal counterpart at the same point in time; negative = the opposite (which is exactly what the RF safety-threshold finding shows).
Wilcoxon p (Wilcoxon signed-rank test): Tests whether the matched-pair differences are consistently on one side of zero, using only their rank order rather than raw magnitude — which makes it robust to the occasional huge outlier. Small p (conventionally <0.05) means the direction is unlikely to be chance. We report it per-slice; with 31 slices tested, treat any single p just under 0.05 gently.
Bonferroni correction: When testing many channels (or many slices) at once, some will look "significant" by pure luck. Bonferroni divides the significance threshold by the number of tests, so a channel marked ** in the root-cause breakdown cleared that stricter bar — a single * means it passed the ordinary p<0.05 test but not the corrected one, worth a second look rather than full trust.
Fraction positive: Of all the matched pairs, what fraction had the fault event scoring higher than its Normal partner. 0.5 is a coin flip; consistently far from 0.5 in the same direction (like the threshold finding's 0.27) is what makes a signal believable even when the raw magnitude test is noisy.
Temporal-block cross-validation: The data is split into contiguous chunks by date; the classifier trains on all but one chunk and is scored on the chunk it never saw, rotating through every chunk once. This prevents the classifier from learning "which era/campaign this event is from" instead of the actual fault physics — a real failure mode caught earlier in this investigation when naive random splitting gave spectacular but fake results.
Hungarian assignment (optimal matching): The algorithm used to pick each fault event's matched Normal control, guaranteeing the best possible one-to-one pairing globally (no two fault events secretly sharing one control, and no event stuck with a bad match just because of processing order).
VAE (variational autoencoder) latent space: A second, independent method shown in each row's detail panel. Instead of the classical linear projection (PCA), a small neural network learns to compress each 1-hour window's full channel ensemble down to 8 numbers and reconstruct it back — a nonlinear analogue of PCA. A simple classifier is then trained on those 8 numbers to separate fault from Normal, per window, so the four AUC bars show whether that window's compressed state is separable — independently of, and without seeing, the classical per-channel evidence above it.
LSTM with attention: A third, genuinely sequential method (currently run at the pooled/ALL scope only): instead of 4 independent window snapshots, this model sees the full ordered 16-step sequence across the whole 4h and learns which timesteps to weight most heavily when deciding fault vs. Normal. The chart shows fault attention minus Normal attention at each timestep — where it's furthest from zero is where the model found the most differentiating signal, which is not always closest to the trigger.
Permutation importance: To find which channels the trained LSTM actually relies on: shuffle one channel's values across all events (destroying its real information while keeping everything else intact) and measure how much the AUC drops. A bigger drop means that channel mattered more. Computed on the training data as a diagnostic, not a held-out evaluation.