Decoding Visual Stimulus in Semantic Space from Electrocorticography Signals

Recent studies using functional magnetic resonance imaging (fMRI) have enabled quantitative evaluation of the semantic space during processing of visual stimuli. In the semantic space of the natural language processing model, called a skip-gram, decoders were shown to generalize to natural scenes of a movie that was not included in the training data of the decoders. Combined with electrocorticography (ECoG), which has a higher sampling rate than fMRI, this approach is expected to aid the development of a practical brain-machine interface. Here, we decoded vector representations of scenes within the semantic space of a skip-gram model to assess whether a decoder trained using ECoG features still generalizes to scenes new to the decoder.