Decoding Silent Reading from Non-Invasive EEG: Extracting Open-Vocabulary Lexical Information
This paper addresses the central data dilemma in non-invasive decoding of inner speech: the inability to obtain paired corpora of brain activity and spontaneous inner monologue. It proposes silent reading as an scalable proxy task and investigates how much lexical and semantic information a contrastive decoder can extract. From a single high-density subject across 393 sessions (about 49 hours) of 19-channel dry-electrode EEG, roughly 240,000 words were presented using rapid serial visual presentation with randomized layout to decouple word identity from low-level visual form. A convolutional EEG encoder feeds a causal transformer, aligning EEG windows with LLM hidden-state embeddings via a CLIP-style contrastive objective. Decoding is evaluated by word-group top-10 retrieval, stably above chance, extending to mid-to-low frequencies and rare words, and growing linearly with training data without saturation. Removing occipital and posterior temporal electrodes reduced the word-level gain by about a third while preserving context tracking. The results establish that open-vocabulary word-level information is recoverable in silent reading, and that decoding is limited by data quantity rather than saturation.
Background and Context
Non-invasive decoding of inner speech has long been bottlenecked by a data problem rather than a modeling one. To recover spontaneous inner monologue, researchers need paired recordings of brain activity and the actual spoken-or-thought words, and such corpora cannot be systematically collected. Existing alternatives are worse: instructive repetition and retrospective report-based generation of inner speech are slow to acquire, poorly time-aligned, and rely on subject compliance that cannot be verified. This paper targets that gap directly, reframing the problem as an empirical, data-available question rather than an intractable one.
The proposed solution is a pragmatic pivot: use silent reading as a scalable proxy task. Participants read continuous narrative text while EEG is recorded, so the presented words are known exactly and the paradigm is fully controlled. The central scientific question becomes measurable—how much lexical and semantic information can a contrastive decoder extract from EEG during passive silent reading? Answering it establishes a methodological foundation for open-vocabulary brain decoding that does not depend on elusive spontaneous speech.
Deep Analysis
Data came from a single high-density subject across 393 sessions totaling about 49 hours of 19-channel dry-electrode EEG, with roughly 240,000 words presented. Words were shown via rapid serial visual presentation, and layout was randomized within each trial. This design deliberately decouples word identity from low-level visual form, preventing the model from shortcutting to visual appearance instead of learning genuine lexical signals.
On the modeling side, a convolutional EEG encoder serves as the feature-extraction backbone, optionally followed by a causal transformer to model temporal dependencies. Training uses a CLIP-style contrastive objective that aligns short EEG windows with hidden-state embeddings of the presented words drawn from a large language model. This alignment maps brain signals into a semantic space, which is what grants open-vocabulary capability: decoding does not need to be closed over a fixed training vocabulary, the key distinction from traditional closed-vocabulary approaches.
Decoding is evaluated by word-group top-10 retrieval against a permutation baseline. Performance sits stably above chance and extends beyond high-frequency words to mid-to-low frequencies and rare words, indicating the model learned generalized lexical representations rather than memorizing training words. Crucially, accuracy grows logarithmically-linearly with training data size with no sign of saturation, supporting the claim that decoding is constrained by data quantity rather than model capacity.
Industry Impact
The work's significance extends well beyond a single experiment's accuracy gains. It demonstrates to the open-source community and brain-decoding researchers that open-vocabulary word-level information can be recovered from readily available silent-reading EEG, providing a feasibility basis for low-cost dry-electrode, long-duration, and at-home brain-computer interaction. Because the bottleneck is data rather than saturation, engineering teams should prioritize scaling data collection over chasing model complexity, a direct guidance for resource allocation.
Ablation experiments revealed a clear spatial contribution structure: removing occipital and posterior-temporal electrodes reduced the word-level gain by about a third while preserving context tracking. This shows that word-level decoding and context tracking are partially separable in their neural sources, with the occipitotemporal region making a substantial contribution to word-level information. Control analyses successfully separated word-level decoding, narrative context tracking, and the non-neuronal position prior introduced by transformer position embeddings, ruling out the concern that positional information contaminated the decoding results.
Outlook
Although the study is based on a single subject, its data volume and paradigm precision are rare in the field, laying a solid foundation for multi-subject expansion and clinical translation. The established proxy paradigm, contrastive-alignment framework, and the analysis method separating word-level from context-level information provide reusable technical paths for the broader goals of inner-speech decoding and semantic brain decoding. The finding that silent reading yields recoverable open-vocabulary information suggests it may become an important data source for brain semantic decoding research. Future work will need to validate these results across subjects and modalities, but the data-driven conclusion points the field toward prioritizing collection scale as the primary lever for progress.