LLRF Fault Diagnosis for SPIRAL2

From raw acquisitions to labeled events

SPIRAL2's superconducting cavities log a postmortem buffer — a short window of RF and control-loop signals around any interlock trip or manual acquisition. 14,232 such files were scanned across the 2019–2025 operational period. A validated ground-truth classifier (Classify_PostMortemFile) applies six ordered quality checks (acquisition validity, feed-forward state, loop status, signal-quality metrics) and assigns each passing file to one of seven fault categories or Normal, yielding the 2,865-event dataset used throughout this pipeline (referred to internally as dataset version V7).

Class imbalance is severe (72× between rarest and most common fault category) and shapes results throughout the pipeline: rarer categories are consistently harder to classify, cluster, and explain, regardless of method.

810 features, mostly generic, a physics-informed core

Each postmortem trace is high-pass filtered, trigger-aligned, and Z-scored, then reduced to 810 features after correlation-based deduplication. The large majority (501 Temporal, 260 Statistical, 10 Metadata/Status) are generic time-series descriptors — derivatives, moments, segment statistics — applicable to any channel. A much smaller, physics-informed core (39 features across seven named families) targets specific known LLRF fault mechanisms:

Early fault-onset detection

A Random Forest trained on a fixed pre-trigger window (excluding whole-signal and steady-state features that would leak post-trigger information) distinguishes eventually-faulting captures from normal ones well before the interlock trips, reaching ≈0.999 ROC AUC — against a 0.58–0.79 AUC range for unsupervised anomaly-detection baselines (Isolation Forest, LOF, Mahalanobis distance, PCA reconstruction) on the same restricted features. Sampled true-positive events remain confidently detected (probability >0.5) well outside the window the classifier was trained to reason about, evidence that at least part of the observed separability reflects a stable per-capture characteristic rather than a purely fault-onset-proximate signal; this is disclosed as an open question rather than resolved.

Binary classification (fault vs. normal)

Restricted to the pre-trigger-only feature subset — the operationally meaningful version of this task, since it does not see the interlock-trip transient itself — four classical methods stay near-ceiling:

Excluding each category's own deterministic-classifier defining signal (e.g. vacuum pressure for vacuum-threshold faults) leaves accuracy essentially unchanged for most categories — evidence that pre-trigger separability is broadly redundant across channels, not a disguised readout of the one signal that will eventually trip the alarm.

Multi-label classification

Four supervised approaches predict all seven fault categories at once (Binary Relevance, Classifier Chains, Label Powerset, MultiOutput Random Forest). Label Powerset is the strongest:

The rarest category (Coupure externe rapide, 11 events) scores 0.000 F1 across every approach and every split — a sample-size floor rather than a methodological weakness.

Root-cause identification

Given that a fault occurred, a class-weighted Random Forest predicts which of the seven categories is responsible, reaching 92.4% ± 1.0% accuracy and 0.782 ± 0.011 macro-F1 (repeated-split mean ± std) on the 1,387 fault events. The training target is the first active fault flag in the ground-truth record — a deterministic rule, not an independently labeled physical root cause — disclosed as a limitation of the task's current formulation. Class weighting meaningfully improves only the rarest well-represented category (Claquage ou quench cavité, +0.099 F1); the dominant confusion runs between the two largest categories, consistent with a real category-size effect rather than a specific physical ambiguity.

Fault subtype clustering

Within each fault category with enough events, unsupervised clustering (K-Means, Agglomerative, Gaussian Mixture) searches for finer subtypes, requiring every candidate cluster to hold at least 15% of that category's events to reject outlier-driven splits.

By the standard silhouette scale, only Dép seuil de sécurité RF (0.335) clears the threshold conventionally read as real structure; the rest are technically balanced but weak. The largest category finds no split meeting the balance constraint at all — a follow-up investigation found three independent methods agreeing on a genuine 13% minority subgroup that simply falls under the floor, and a supervised classifier trained on this category's separate, rule-based 9-way physics subtype label reaches macro-F1 = 0.812, confirming that real subtype structure exists here even though unsupervised search alone could not find it.

Explainability

SHAP explains the per-category classical models, gradient-based saliency explains four deep architectures (CNN, CNN-LSTM, MLP, Transformer), and LIME provides local, instance-level explanations. Because the seven physics-informed feature families are vastly outnumbered by generic Statistical/Temporal features (39 vs. 771), importance is reported per-feature-mean (dividing out group size) rather than as a group sum, which would trivially favor the larger generic groups regardless of informativeness.

A physics-informed family dominates four of seven categories despite the severe count imbalance. Both quench-related categories (an abrupt collapse and a regulation deviation) converge on Effective Decay, consistent with each being, at the feature level, a rapid change in cavity voltage. Gradient-based saliency on the four deep architectures shows the same structural pattern (Temporal-family dominance by group sum, for the same count-imbalance reason) as an independent cross-check using a different model family entirely.

Inference latency

Single-event, single-threaded inference (scaler transform + predict, no batching) was benchmarked directly rather than assumed:

Every model is comfortably fast enough for the "one new postmortem file arrives" deployment scenario this benchmark targets. XGBoost is roughly 35× faster per event than the Random-Forest-based models.

Glossary & acronyms

LLRF

Low-Level Radio Frequency — the control system regulating a cavity's RF field amplitude and phase.

RF

Radio Frequency — the oscillating electromagnetic field used to accelerate the particle beam.

Cavity

A superconducting resonant structure that transfers RF energy to the particle beam.

Quench

A sudden, localized loss of superconductivity in a cavity, causing a rapid collapse of the stored RF field.

Postmortem buffer

A short recording of signals captured automatically around a fault trigger, used for offline diagnosis.

ALM

Alarme (French: alarm) — the control-system bit flagging an active interlock/fault condition.

Feed-forward

An LLRF control-loop mode that pre-compensates known disturbances rather than reacting to them; operationally meaningful only from 2021 onward in this dataset.

ROC AUC

Area under the Receiver Operating Characteristic curve — a threshold-independent measure of binary classification quality (1.0 = perfect, 0.5 = chance).

F1 / macro-F1

The harmonic mean of precision and recall for one class; macro-F1 averages this unweighted across all classes, so rare classes count equally with common ones.

SHAP

SHapley Additive exPlanations — a game-theoretic method attributing a model's prediction to its input features.

LIME

Local Interpretable Model-agnostic Explanations — explains one individual prediction by fitting a simple local surrogate model around it.

Saliency

A gradient-based explanation method for neural networks: how much a small change in each input feature would change the output.

Silhouette score

A clustering-quality measure from −1 to 1, comparing each point's distance to its own cluster vs. the nearest other cluster.

SPIRAL2

The superconducting linear accelerator facility at GANIL, Caen, France.

GANIL

Grand Accélérateur National d'Ions Lourds — the French national heavy-ion accelerator laboratory.

Bibliography

Methods, prior work, and SPIRAL2-specific references used throughout this pipeline. Every DOI below was checked directly against Crossref or the publisher's own record; a handful of legitimate older or preprint-venue papers have no DOI at all, and are marked as such rather than given a guessed one.

1. SPIRAL2 / GANIL — A. Orduz et al., “SPIRAL2 commissioning and operations”, Journal of Instrumentation 19, T05006 (2024). 10.1088/1748-0221/19/05/t05006
2. SPIRAL2 LLRF control — F. Bouly et al., “Superconducting Cavity and RF Control loop model for the SPIRAL2 LINAC”, LINAC2022 (2022). 10.18429/JACoW-LINAC2022-THPOPA13
3. AI for accelerators, overview — A. Ghribi et al., “Artificial intelligence for advancing particle accelerators”, Europhysics News 56(1), 15–19 (2025). 10.1051/epn/2025106
4. ML for accelerator fault diagnosis — C. Tennant et al., “Superconducting Radio-Frequency Cavity Fault Classification Using Machine Learning at Jefferson Laboratory”, Phys. Rev. Accel. Beams 23, 114601 (2020). 10.1103/PhysRevAccelBeams.23.114601
5. ML for accelerator anomaly detection — C. Peters et al., “Machine Learning for Time Series Prediction of an Accelerator Beam to Recognize Equipment Malfunction”, IPAC’21 (2021). 10.18429/JACoW-IPAC2021-TUPAB328
6. ML for beam-loss classification — G. Valentino et al., “Machine Learning Applied at the LHC for Beam Loss Pattern Classification”, J. Phys.: Conf. Ser. 1067, 072036 (2018). 10.1088/1742-6596/1067/7/072036
7. Quench prediction (LSTM) — M. Mertik & A. Skoczeń et al., “A Conceptual Development of Quench Prediction App based on LSTM and ELQA Framework”, arXiv:1610.09201 (2016). 10.48550/arXiv.1610.09201
8. Model-based fault estimation — B. Sayyar-Rodsari & C. Schweiger, “Optimal Model-Based Fault Estimation and Correction for Particle Accelerators and Industrial Plants”, SLAC tech. report (2010). 10.2172/992983
9. Rule-based accelerator fault diagnosis — E. Malandain & P. Skarek, “An Expert System for Accelerator Fault Diagnosis”, PAC’87, 559–561 (1987). JACoW archive — no DOI exists (pre-DOI-era proceedings).
10. XFEL anomaly detection — A. S. Nawaz, “Probabilistic Model-based Anomaly Detection for the European X-ray Free Electron Laser”, PhD thesis, Universität zu Lübeck (2021). No DOI found (institutional-repository thesis).
11. Isolation Forest — F. T. Liu, K. M. Ting & Z.-H. Zhou, ICDM, 413–422 (2008). 10.1109/icdm.2008.17
12. Local Outlier Factor — M. M. Breunig et al., ACM SIGMOD, 93–104 (2000). 10.1145/342009.335388
13. DBSCAN — M. Ester et al., “A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise”, KDD, 226–231 (1996). No DOI exists (pre-DOI-era proceedings).
14. Deep autoencoding GMM (anomaly detection) — B. Zong et al., ICLR (2018). No DOI found (ICLR/OpenReview venue).
15. GANomaly — S. Akcay, A. Atapour-Abarghouei & T. P. Breckon, ACCV, 622–637 (2018/2019). 10.1007/978-3-030-20893-6_39
16. Silhouette score — P. J. Rousseeuw, “Silhouettes: A Graphical Aid to the Interpretation and Validation of Cluster Analysis”, J. Comput. Appl. Math. 20, 53–65 (1987). 10.1016/0377-0427(87)90125-7
17. SHAP — S. M. Lundberg & S.-I. Lee, “A Unified Approach to Interpreting Model Predictions”, NeurIPS, 4765–4774 (2017). No DOI found (NeurIPS venue).
18. LIME — M. T. Ribeiro, S. Singh & C. Guestrin, “‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier”, ACM SIGKDD, 1135–1144 (2016). 10.1145/2939672.2939778
19. Explainable AI, survey — A. Barredo Arrieta et al., “Explainable Artificial Intelligence (XAI)”, Information Fusion 58, 82–115 (2020). 10.1016/j.inffus.2019.12.012
20. Classifier Chains — J. Read et al., ECML, 254–269 (2009). 10.1007/978-3-642-04174-7_17
21. ML-kNN — M.-L. Zhang & Z.-H. Zhou, Pattern Recognition 40, 2038–2048 (2007). 10.1016/j.patcog.2006.12.019
22. Multi-label decision trees — C. Vens et al., “Decision Trees for Hierarchical Multi-Label Classification”, Machine Learning 73, 185–214 (2008). 10.1007/s10994-008-5077-3
23. Multi-label learning, survey — M.-L. Zhang & Z.-H. Zhou, IEEE TKDE 26, 1819–1837 (2014). 10.1109/tkde.2013.39
24. Causal inference — J. Pearl, Causality: Models, Reasoning, and Inference, 2nd ed., Cambridge University Press (2009). 10.1017/cbo9780511803161
25. Granger causality — C. W. J. Granger, Econometrica 37, 424–438 (1969). 10.2307/1912791

Project team

• Adnan Ghribi
• Charly Lassalle
• Alexandre Dalibard Brund
• Marco Di Giacomo
• Frédéric Bouly