Mind the Gap! Alignment of Transformer J-Space with Cortical Representations.
Lucas Nunn, Anke Borchers, Anna Zhu, Frank Peterlein
Do J-Space embeddings capture semantic information in a manner analogous to the human brain? Building on recent work demonstrating representational alignment between Large Language Model (LLM) activations and fMRI responses to natural scenes, we investigate whether J-Space embeddings achieve stronger alignment with higher-level cortical areas than standard LLM embeddings. Using representational similarity analysis (RSA) on large-scale fMRI data, we find J-Space embeddings to reach largely similar alignment as raw LLM embeddings. We identify two primary constraints of the paradigm: First, textual scene captions already reflect human-compressed semantic abstraction. Second, visual-sensory cortical responses do not map directly onto the higher-level cognitive functions hypothesized to occupy J-Space. Future work would utilize datasets targeting higher-order cognitive functions to investigate neurobiological correlates of J-Space representations more tightly.
No reviews are available yet
Cite this work
@misc {
title={
(HckPrj) Mind the Gap! Alignment of Transformer J-Space with Cortical Representations.
},
author={
Lucas Nunn, Anke Borchers, Anna Zhu, Frank Peterlein
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


