Classification of non-time-locked rapid serial visual presentation events for brain-computer interaction using deep learning
Zijing MaoVernon J. LawhernLenis Mauricio MeriñoKenneth R. BallLi DengBrent J. LanceKay A. RobbinsYufei Huang
9
Citation
19
Reference
10
Related Paper
Citation Trend
Abstract:
Deep learning solutions based on deep neural networks (DNN) and deep stack networks (DSN) were investigated for classifying target images in a non-time-locked rapid serial visual presentation (RSVP) image target identification task using EEG. Several feature extraction methods associated with this task were implemented and tested for deep learning, where a sliding window method using the trained classifier was used to predict the occurrence of target events in a non-time-locked fashion‥ The deep learning algorithms explored based on deep stacking networks were able to improve the error rate by about 5% over existing algorithms such as linear discriminant analysis (LDA) for this task. Initial test results also showed that this method based on deep stacking networks for non-time-locked classification can produce an error rate close to that achieved for time-locked classification, thus illustrating the power of deep learning for complex feature spaces.Keywords:
Word error rate
Deep Neural Networks
Through rapid serial visual presentation (RSVP), we asked Ss to identify a partially specified letter (target) and then to detect the presence or absence of a fully specified letter (probe). Whereas targets are accurately identified, probes are poorly detected when they are presented during a 270-ms interval beginning 180 ms after the target. Probes presented immediately after the target or later in the RSVP stream are accurately detected. This temporary reduction in probe detection was not found in conditions in which a brief blank interval followed the target or Ss were not required to identify the target. The data suggest that the presentation of stimuli after the target but before target-identification processes are complete produces interference at a letter-recognition stage. This interference may cause the temporary suppression of visual attention mechanisms observed in the present study.
Attentional blink
Blank
Cite
Citations (2,267)
We previously reported that low vision readers do not benefit from a rapid serial visual presentation (RSVP) display relative to a scroll display. Each reader in those studies was presented with only one letter size, and it was the same for both displays. In the current study, we systematically varied the size of the letters and compared reading rates from the 2 displays for letters that were 2, 4, 6, 8, and 10 times each reader's acuity threshold. Using this paradigm, we found that subjects with normal vision (n = 12) read faster with RSVP for all text sizes. Low vision subjects (N = 20) showed no benefit of RSVP until the text was at least 8x their acuity threshold. As in our prior studies, there was a great deal of variability within the low vision group, and for a small number of subjects (25%), reading was faster from the scroll than from the RSVP display.
Presentation (obstetrics)
Cite
Citations (24)
This thesis explores the subjective experience of targets in rapid serial visual presentation (RSVP), an experimental paradigm where visual stimuli are displayed in rapid succession. In RSVP, items appear on the screen so briefly that not every item in the stream can be encoded reliably. Thus, it allows observation of conscious experience at the fringe of perception. The Attentional Blink (AB) - an effect in which an RSVP target is likely to be missed if it follows a fully processed first target - has been used in order to manipulate the accuracy of item identification by varying the target separation and presentation speed.
The main focus of studies using RSVP presentation to make inferences about conscious perception has been the question of whether conscious perception is all-or-none or gradual. We initially present some thoughts on the suitability of the AB paradigm for answering this question. Not much is known about the effect of different variables in the paradigm on subjective experience, and it is possible that AB mechanisms affect experience quite differently from other paradigms, limiting the generalisability of findings derived from work within the AB paradigm.
Based on this, we follow two lines of evidence: First, we explore the possibility of finding gradations in subjective visibility of targets measured on ratings scales and in the response of the electroencephalogram using a simple single target RSVP. Second, we investigate the effect of target separation and perceived order on this subjective experience in the AB paradigm.
Our results indicate that items in single-target RSVP can be perceived in a graded manner, with possible indications of a non-linear jump in brain activity between not-seen and seen items. Regarding subjective experience when separation of two targets is varied, we find a disconnect between accuracy and visibility of the second target when in close proximity to the first, showing relatively low subjective experience considering the high report accuracy. Target separation also affects the number of order confusions, which we find to reduce target visibility under specific conditions. These results add to our understanding of how targets are perceived in RSVP and have implications for research into conscious perception.
Attentional blink
Presentation (obstetrics)
Cite
Citations (1)
Nested Cavity Classifier (NCC) is a classification rule that pursues partitioning the feature space, in parallel coordinates, into convex hulls to build decision regions. It is claimed in some literatures that this geometric-based classifier is superior to many others, particularly in higher dimensions. First, we give an example on how NCC can be inefficient, then motivate a remedy by combining the NCC with the Linear Discriminant Analysis (LDA) classifier. We coin the term Nested Cavity Discriminant Analysis (NCDA) for the resulting classifier. Second, a simulation study is conducted to compare both, NCC and NCDA to another two basic classifiers, Linear and Quadratic Discriminant Analysis. NCC alone proves to be inferior to others, while NCDA always outperforms NCC and competes with LDA and QDA.
Quadratic classifier
Cite
Citations (0)
Previous research has found that participants can identify a target letter or number in a stream of rapidly presented items. More recently, researchers have shown that participants can detect a categorically defined target object in one of these rapid serial visual presentation (RSVP) streams. The results suggest that there may be practical applications for RSVP-based search. To explore this possibility, we performed two experiments that compared accuracy between a typical visual search task and a search in which the display was segmented and presented as a RSVP stream. Across conditions we equated the total search time. By presenting every segment of the overall display rapidly at fixation, the RSVP condition could benefit from the lack of attention shifts necessary in full display searches, a method that requires relatively slow eye movements and fixations. Experiment 1 presented 24 Landolt Cs (serially or in a whole display) and participants were required to identify the color of the C with a break on the left. Performance was significantly better for the RSVP condition than the full display condition. Experiment 2 replicated the RSVP advantage using much more complex scenes ("Wheres Waldo?" images). These results suggest that real world searches may benefit from segmenting the display and presenting images in a RSVP stream. Meeting abstract presented at VSS 2014
Visual Search
Presentation (obstetrics)
Cite
Citations (0)
Attentional blink
Presentation (obstetrics)
Repetition (rhetorical device)
Cite
Citations (3)
Presentation (obstetrics)
Cite
Citations (21)
Alphanumeric
Attentional blink
Cite
Citations (25)
Colored letters were presented by means of rapid serial visual presentation (RSVP). Two letters (T1 and T2) were targets that required a speeded (T1) or delayed (T2) response. T1 was shown on half of the trials. Two tasks were performed in the experimental conditions. Task 1 required a discrimination between 2 letters (when T1 was shown) or an indication that T1 was absent. In Experiment 1, Task 2 was to detect the presence of a unique color in the RSVP stream (e.g., green in a stream of alternating red and gray). In Experiment 2, Task 2 was to report the color of the first colored letter to appear in the RSVP stream. The lag between T1 and T2 in the RSVP stream was manipulated. Accuracy of color reports in Task 2 was poor at shorter lags and improved as the lag was lengthened in both experiments, demonstrating an attentional blink for chromatic information. Theoretical implications of the results are discussed.
Attentional blink
Colored
Cite
Citations (65)
Colored letters were presented by means of rapid serial visual presentation (RSVP). Two letters (T1 and T2) were targets that required a speeded (T1) or delayed (T2) response. T1 was shown on half of the trials. Two tasks were performed in the experimental conditions. Task 1 required a discrimination between 2 letters (when T1 was shown) or an indication that T1 was absent. In Experiment 1, Task 2 was to detect the presence of a unique color in the RSVP stream (e.g., green in a stream of alternating red and gray). In Experiment 2, Task 2 was to report the color of the first colored letter to appear in the RSVP stream. The lag between T1 and T2 in the RSVP stream was manipulated. Accuracy of color reports in Task 2 was poor at shorter lags and improved as the lag was lengthened in both experiments, demonstrating an attentional blink for chromatic information. Theoretical implications of the results are discussed.
Attentional blink
Colored
Cite
Citations (46)