brain-mapping swin-transformer neural-decoding vision-transformer PDF

Neural Scroll Project: Predicting Brain Scrolls with Swin Transformer

Posted by Martin sertin on April 22, 2025

Neural Scroll Project: Predicting Brain Scrolls with Swin Transformer

Welcome to the Neural Scroll Project, where cutting-edge neuroscience meets state-of-the-art vision transformers to uncover hidden structures within the brain — what we call “brain scrolls”. These latent spatiotemporal patterns encode deep layers of cognition, memory, and sensory processing.

🧠 What is a Brain Scroll?

A brain scroll is a metaphorical term we use to describe highly structured, multi-dimensional neural activity patterns that emerge over time — like a scroll being unrolled. These patterns often represent:

  • Episodic memory sequences
  • Predictive cognition loops
  • Neural priors shaped by experience and environment

Our goal is to predict and decode these patterns in real-time using advanced machine learning architectures — particularly, the Swin Transformer.

🔍 Why Swin Transformer?

The Swin Transformer (Shifted Window Transformer) is a hierarchical vision transformer architecture that excels in learning from spatially localized features. Originally designed for computer vision, its properties make it ideal for decoding dynamic brain activity represented as high-dimensional time-series maps (e.g., fMRI slices or MEG signals over time).

Key advantages:

  • Local-global context fusion through window shifting
  • Hierarchical feature learning like CNNs, but fully attention-driven
  • Excellent at temporal prediction when adapted with positional encodings

🧪 Our Experimental Pipeline

  1. Data Input: Preprocessed fMRI/EEG tensors shaped as 3D arrays (Time × Channels × Regions)
  2. Embedding: Patch-based embedding of time-series slices
  3. Swin Transformer: Multi-stage encoder layers with shifted attention windows
  4. Prediction Head: Forecasting latent scroll patterns in future timesteps

Sample Architecture

🔥 python

1
2
3
4
5
6
7
8
9

class BrainScrollPredictor(nn.Module):
    def __init__(self):
        super().__init__()
        self.swin = SwinTransformer3D(...)
        self.head = nn.Linear(...)

    def forward(self, x):
        features = self.swin(x)
        return self.head(features)
CATALOG