MorphMLP: A Self-Attention Free, MLP-Like Backbone for Image and Video
0
Citation
0
Reference
10
Related Paper
Abstract:
Self-attention has become an integral component of the recent network architectures, e.g., Transformer, that dominate major image and video benchmarks. This is because self-attention can flexibly model long-range information. For the same reason, researchers make attempts recently to revive Multiple Layer Perceptron (MLP) and propose a few MLP-Like architectures, showing great potential. However, the current MLP-Like architectures are not good at capturing local details and lack progressive understanding of core details in the images and/or videos. To overcome this issue, we propose a novel MorphMLP architecture that focuses on capturing local details at the low-level layers, while gradually changing to focus on long-term modeling at the high-level layers. Specifically, we design a Fully-Connected-Like layer, dubbed as MorphFC, of two morphable filters that gradually grow its receptive field along the height and width dimension. More interestingly, we propose to flexibly adapt our MorphFC layer in the video domain. To our best knowledge, we are the first to create a MLP-Like backbone for learning video representation. Finally, we conduct extensive experiments on image classification, semantic segmentation and video classification. Our MorphMLP, such a self-attention free backbone, can be as powerful as and even outperform self-attention based models.Keywords:
Representation
Perceptron
Cite
Citations (31)
Cite
Citations (16)
Cite
Citations (0)
Torso
Cite
Citations (7)
Accordion
Representation
Adjacency list
Action Recognition
Cite
Citations (15)
Benchmark (surveying)
Representation
Cite
Citations (11)
The authors describe an alarm system which will detect and track a human moving around a scene from stationary cameras. The authors provide a summary of the system from the lower pixel based representation to the higher level edge grouping and stereo matching. Data flows from the three cameras into the initial segmentation modules before the stereo matching algorithm is applied. After stereo matching, between the three cameras, they utilise edge statistics and apply the disparity gradient limit. This allows a decision about a particular edges 'goodness' to be made. The disparity results are then extracted, considered in terms of recent frames, and analysed over time. The statistics from this module can then be used to alter the thresholds at lower levels of processing. The authors discuss calibration and accuracy, explain the reasons for using three cameras as opposed to two and give examples of the algorithms applied to trial sequences.
False alarm
Representation
Cite
Citations (1)
Image registration
Gaussian blur
Cite
Citations (0)
Representation
Similarity (geometry)
Cite
Citations (1)