Yunxiao Shi

Shenyang University of Technology

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Xing Zi

University of Technology Sydney

Fatih Porikli

Qualcomm (United Kingdom)

Min Xu

University of Technology Sydney

Qiang Wu

Northumbria University

Hong Cai

Hong Kong Polytechnic University

Manish Kumar Singh

Qualcomm (United Kingdom)

Haimin Zhang

Institute of Solid State Physics

Hong Cai

Agency for Science, Technology and Research

Zijing Shi

University of Technology Sydney

Risheek Garrepalli

Qualcomm (United Kingdom)

Cooperative Institutions

University of Technology Sydney

Qualcomm (United Kingdom)

University of Science and Technology of China

Chinese Academy of Sciences

Qualcomm (United States)

Southwest Jiaotong University

Fudan University

Guangdong University of Technology

Ministry of Transport

Rutgers Sexual and Reproductive Health and Rights

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

FutureDepth: Learning to Predict the Future Improves Video Depth Estimation

arXiv (Cornell University) (2024)

Rajeev Yasarla Manish Kumar Singh Hong Cai Yunxiao Shi Jisoo Jeong

In this paper, we propose a novel video depth estimation approach, FutureDepth, which enables the model to implicitly leverage multi-frame and motion cues to improve depth estimation by making it learn to predict the future at training. More specifically, we propose a future prediction network, F-Net, which takes the features of multiple consecutive frames and is trained to predict multi-frame features one time step ahead iteratively. In this way, F-Net learns the underlying motion and correspondence information, and we incorporate its features into the depth decoding process. Additionally, to enrich the learning of multiframe correspondence cues, we further leverage a reconstruction network, R-Net, which is trained via adaptively masked auto-encoding of multiframe feature volumes. At inference time, both F-Net and R-Net are used to produce queries to work with the depth decoder, as well as a final refinement network. Through extensive experiments on several benchmarks, i.e., NYUDv2, KITTI, DDAD, and Sintel, which cover indoor, driving, and open-domain scenarios, we show that FutureDepth significantly improves upon baseline models, outperforms existing video depth estimation methods, and sets new state-of-the-art (SOTA) accuracy. Furthermore, FutureDepth is more efficient than existing SOTA video depth estimation models and has similar latencies when comparing to monocular models

10.48550/arxiv.2403.12953

Cite

Citations (0)

EGA-Depth: Efficient Guided Attention for Self-Supervised Multi-Camera Depth Estimation

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2023)

Yunxiao Shi Hong Cai Amin Ansari Fatih Porikli

The ubiquitous multi-camera setup on modern autonomous vehicles provides an opportunity to construct surround-view depth. Existing methods, however, either perform independent monocular depth estimations on each camera or rely on computationally heavy self attention mechanisms. In this paper, we propose a novel guided attention architecture, EGA-Depth, which can improve both the efficiency and accuracy of self-supervised multi-camera depth estimation. More specifically, for each camera, we use its perspective view as the query to cross-reference its neighboring views to derive informative features for this camera view. This allows the model to perform attention only across views with considerable overlaps and avoid the costly computations of standard self-attention. Given its efficiency, EGA-Depth enables us to exploit higher-resolution visual features, leading to improved accuracy. Furthermore, EGA-Depth can incorporate more frames from previous time steps as it scales linearly w.r.t. the number of views and frames. Extensive experiments on two challenging autonomous driving benchmarks nuScenes and DDAD demonstrate the efficacy of our proposed EGA-Depth and show that it achieves the new state-of-the-art in self-supervised multi-camera depth estimation.

Monocular

10.1109/cvprw59228.2023.00017

Cite

Citations (10)

Based on the Conformer End-to-End Chinese Speech Recognition Method

Haiwen Feng Yunxiao Shi

End-to-end principle

Conformational isomerism

10.1109/eiecs63941.2024.10800424

Cite

Citations (0)

MAMo: Leveraging Memory and Attention for Monocular Video Depth Estimation

arXiv (Cornell University) (2023)

Rajeev Yasarla Hong Cai Jisoo Jeong Yunxiao Shi Risheek Garrepalli

We propose MAMo, a novel memory and attention frame-work for monocular video depth estimation. MAMo can augment and improve any single-image depth estimation networks into video depth estimation models, enabling them to take advantage of the temporal information to predict more accurate depth. In MAMo, we augment model with memory which aids the depth prediction as the model streams through the video. Specifically, the memory stores learned visual and displacement tokens of the previous time instances. This allows the depth network to cross-reference relevant features from the past when predicting depth on the current frame. We introduce a novel scheme to continuously update the memory, optimizing it to keep tokens that correspond with both the past and the present visual information. We adopt attention-based approach to process memory features where we first learn the spatio-temporal relation among the resultant visual and displacement memory tokens using self-attention module. Further, the output features of self-attention are aggregated with the current visual features through cross-attention. The cross-attended features are finally given to a decoder to predict depth on the current frame. Through extensive experiments on several benchmarks, including KITTI, NYU-Depth V2, and DDAD, we show that MAMo consistently improves monocular depth estimation networks and sets new state-of-the-art (SOTA) accuracy. Notably, our MAMo video depth estimation provides higher accuracy with lower latency, when omparing to SOTA cost-volume-based video depth models.

Monocular

Frame rate

10.48550/arxiv.2307.14336

Cite

Citations (0)

Clip4cs: Clip-Driven Sample Quality Discrimination and Coreset Selection

Xing Zi Yunxiao Shi T. J. Zhu Xian Tao Jun Li

Sample (material)

10.2139/ssrn.4963252

Cite

Citations (0)

Enhancing Retrieval and Managing Retrieval: A Four-Module Synergy for Improved Quality and Efficiency in RAG Systems

arXiv (Cornell University) (2024)

Yunxiao Shi Xing Zi Zijing Shi Haimin Zhang Qiang Wu

Retrieval-augmented generation (RAG) techniques leverage the in-context learning capabilities of large language models (LLMs) to produce more accurate and relevant responses. Originating from the simple 'retrieve-then-read' approach, the RAG framework has evolved into a highly flexible and modular paradigm. A critical component, the Query Rewriter module, enhances knowledge retrieval by generating a search-friendly query. This method aligns input questions more closely with the knowledge base. Our research identifies opportunities to enhance the Query Rewriter module to Query Rewriter+ by generating multiple queries to overcome the Information Plateaus associated with a single query and by rewriting questions to eliminate Ambiguity, thereby clarifying the underlying intent. We also find that current RAG systems exhibit issues with Irrelevant Knowledge; to overcome this, we propose the Knowledge Filter. These two modules are both based on the instruction-tuned Gemma-2B model, which together enhance response quality. The final identified issue is Redundant Retrieval; we introduce the Memory Knowledge Reservoir and the Retriever Trigger to solve this. The former supports the dynamic expansion of the RAG system's knowledge base in a parameter-free manner, while the latter optimizes the cost for accessing external knowledge, thereby improving resource utilization and response efficiency. These four RAG modules synergistically improve the response quality and efficiency of the RAG system. The effectiveness of these modules has been validated through experiments and ablation studies across six common QA datasets. The source code can be accessed at https://github.com/Ancientshi/ERM4.

10.48550/arxiv.2407.10670

Cite

Citations (0)

PADRe: A Unifying Polynomial Attention Drop-in Replacement for Efficient Vision Transformer

arXiv (Cornell University) (2024)

Pierre-David Létourneau Manish Kumar Singh Hsin-Pai Cheng Shizhong Han Yunxiao Shi

We present Polynomial Attention Drop-in Replacement (PADRe), a novel and unifying framework designed to replace the conventional self-attention mechanism in transformer models. Notably, several recent alternative attention mechanisms, including Hyena, Mamba, SimA, Conv2Former, and Castling-ViT, can be viewed as specific instances of our PADRe framework. PADRe leverages polynomial functions and draws upon established results from approximation theory, enhancing computational efficiency without compromising accuracy. PADRe's key components include multiplicative nonlinearities, which we implement using straightforward, hardware-friendly operations such as Hadamard products, incurring only linear computational and memory costs. PADRe further avoids the need for using complex functions such as Softmax, yet it maintains comparable or superior accuracy compared to traditional self-attention. We assess the effectiveness of PADRe as a drop-in replacement for self-attention across diverse computer vision tasks. These tasks include image classification, image-based 2D object detection, and 3D point cloud object detection. Empirical results demonstrate that PADRe runs significantly faster than the conventional self-attention (11x ~ 43x faster on server GPU and mobile NPU) while maintaining similar accuracy when substituting self-attention in the transformer models.

10.48550/arxiv.2407.11306

Cite

Citations (0)

DeCoTR: Enhancing Depth Completion with 2D and 3D Attentions

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

Yunxiao Shi Manish Kumar Singh Hong Cai Fatih Porikli

10.1109/cvpr52733.2024.01021

Cite

Citations (0)

A Parted-based Method for Unsupervised Visible-Infrared Re-identification

2021 16th International Conference on Intelligent Systems and Knowledge Engineering (ISKE) (2023)

Yunxiao Shi Yu Zeng Tianrui Li Shengdong Du Yeyiran Jing

Unsupervised Visible-Infrared Person Re-identification (USL- VI-ReID) problem is the common embranchment of supervised Visible-Infrared Person Re-identification (VI-ReID) and single modality Unsupervised Learning Person Re-identification (USL-ReID). To better target USL-VI-ReID problem, some specific methods are proposed. However, existing methods only mine image features from a global perspective and lack exploration of local details. To fill this gap, we designed a two-staged framework, which consists of a Global and Local Joint Learning (GLJL) framework and Part-based Cross-Modality Alignment (PCMA) method. GLJL aims to build basic local idea application. It segments the global image features into parts to extract local features. These local features are combined with global features for training, which enables the model to focus both global information and local details. With GLJL, the model has better performance in intra-modality learning stage. Based on GLJL, we design PCMA method to further overcome cross-modality differences. PCMA matches cross-modality pseudo labels at both global and local scale, which not only reduces modality gap, but also avoids the model being affected by environmental noise due to excessive attention to local details. With PCMA, the model shows effective improvement compared to GLJL. Extensive experiments prove that our method is effective in improving the performance compared other USL-VI-ReID methods.

Identification

10.1109/iske60036.2023.10481115

Cite

Citations (0)

Parameter Hierarchical Optimization for Visible-Infrared Person Re-Identification

arXiv (Cornell University) (2024)

Yu Zeng Yunxiao Shi

Visible-infrared person re-identification (VI-reID) aims at matching cross-modality pedestrian images captured by disjoint visible or infrared cameras. Existing methods alleviate the cross-modality discrepancies via designing different kinds of network architectures. Different from available methods, in this paper, we propose a novel parameter optimizing paradigm, parameter hierarchical optimization (PHO) method, for the task of VI-ReID. It allows part of parameters to be directly optimized without any training, which narrows the search space of parameters and makes the whole network more easier to be trained. Specifically, we first divide the parameters into different types, and then introduce a self-adaptive alignment strategy (SAS) to automatically align the visible and infrared images through transformation. Considering that features in different dimension have varying importance, we develop an auto-weighted alignment learning (AAL) module that can automatically weight features according to their importance. Importantly, in the alignment process of SAS and AAL, all the parameters are immediately optimized with optimization principles rather than training the whole network, which yields a better parameter training manner. Furthermore, we establish the cross-modality consistent learning (CCL) loss to extract discriminative person representations with translation consistency. We provide both theoretical justification and empirical evidence that our proposed PHO method outperform existing VI-reID approaches.

Discriminative model

Identification

Disjoint sets

Modality (human–computer interaction)

10.48550/arxiv.2404.07930

Cite

Citations (0)