C-MAT: Cross-Modal Aligned Transformer
A dual-stream transformer for neurodegenerative disease assessment that aligns MRI and resting-state EEG representations while remaining usable when a clinical modality is missing.
Engineering evidence
- My contribution
- Thesis researcher · Architecture, ETL, training, and evaluation
- Outcome
- Defense completed in May 2026 · Thesis grade A
- Decision record 01
- Used modality dropout and contrastive alignment because real clinical datasets rarely provide every imaging modality for every patient.
Technical highlights
- 01
Built an ETL pipeline for 1,400+ multimodal records spanning BrainLat, OASIS, PPMI, and AHEPA sources.
- 02
Combined cross-modal contrastive alignment, modality dropout, SHAP analysis, and Attention Rollout.
Abstract
C-MAT investigates modality-invariant representation learning for Alzheimer's disease, Parkinson's disease, and frontotemporal dementia using structural MRI and resting-state EEG. Its dual-stream vision-transformer design uses cross-modal contrastive alignment and modality dropout to learn shared representations and handle missing inputs. The research pipeline covers MRI registration, EEG artifact rejection, spectrogram generation, evaluation, SHAP analysis, and Attention Rollout across more than 1,400 multimodal records.
Stack
- Python
- PyTorch
- Transformers
- SHAP
- Neuroimaging