Architecting a novel, modality-aware Transformer framework to diagnose neurodegenerative diseases from multi-modal neuroimaging data. This work pioneers a 'modality dropout' scheme to build a system robust to the real-world challenge of incomplete clinical data.
Integrates a dual-level explainability framework using SHAP for modality-wise contribution and attention maps for region-wise anatomical interpretation.
This research addresses two of the most significant barriers to the clinical translation of AI in neurodiagnostics: the fragility of models when faced with incomplete data and the lack of integrated, clinically relevant explainability. We propose a novel, modality-aware Transformer framework designed to diagnose neurodegenerative diseases (including Alzheimer's, Parkinson's, and Multiple Sclerosis) from a multi-modal combination of structural MRI (sMRI), functional MRI (fMRI), and electroencephalography (EEG). The cornerstone of our architecture is a 'modality dropout' aware training scheme. This technique directly confronts the clinical reality of missing data by systematically training the model to make robust predictions from any available subset of modalities, thereby building a system that is intrinsically resilient to partial inputs. The architecture employs a dual-encoder system with a sophisticated cross-modal attention mechanism to learn the deep, synergistic interactions between disparate data streams. To ensure clinical trust and facilitate biomarker discovery, our framework generates a dual-level explanation for each prediction, using SHAP to provide a clear, quantitative ranking of each modality's contribution, paired with attention heatmaps that offer a granular, visual guide to the specific brain regions that were most influential.