René Schuster

German Research Centre for Artificial Intelligence

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Didier Stricker

German Research Centre for Artificial Intelligence

Oliver Wasenmüller

Mannheim University of Applied Sciences

Ramy Battrawy

German Research Centre for Artificial Intelligence

Christian Unger

BMW Group (Germany)

Christian Bailer

German Research Centre for Artificial Intelligence

Sandra Pfiel

IMC University of Applied Sciences Krems

Michael Fürst

University of Kaiserslautern

Florian Tiefenbacher

IMC University of Applied Sciences Krems

Michael Reiner

IMC University of Applied Sciences Krems

Katharina Bendig

University of Koblenz and Landau

Cooperative Institutions

German Research Centre for Artificial Intelligence

University of Kaiserslautern

IMC University of Applied Sciences Krems

Technical University of Munich

Graz University of Technology

Joanneum Research

University of Southampton

National Technical University of Athens

Birla Institute of Technology and Science, Pilani

BMW Group (Germany)

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

Multi-scale Iterative Residuals for Fast and Scalable Stereo Matching

Syed M. Raza René Schuster Didier Stricker

Despite the remarkable progress of deep learning in stereo matching, there exists a gap in accuracy between real-time models and slower state-of-the-art models which are suitable for practical applications. This paper presents an iterative multi-scale coarse-to-fine refinement (iCFR) framework to bridge this gap by allowing it to adopt any stereo matching network to make it fast, more efficient and scalable while keeping comparable accuracy. To reduce the computational cost of matching, we use multi-scale warped features to estimate disparity residuals and push the disparity search range in the cost volume to a minimum limit. Finally, we apply a refinement network to recover the loss of precision which is inherent in multi-scale approaches. We test our iCFR framework by adopting the matching networks from state-of-the art GANet and AANet. The result is 49 × faster inference time compared to GANet-deep and 4 × less memory consumption, with comparable error. Our best performing network, which we call FRSNet is scalable even up to an input resolution of 6K on a GTX 1080Ti, with inference time still below one second and comparable accuracy to AANet+. It out-performs all real-time stereo methods and achieves competitive accuracy on the KITTI benchmark.

Benchmark (surveying)

10.1145/3488904.3493376

Cite

Citations (1)

SSGP: Sparse Spatial Guided Propagation for Robust and Generic Interpolation

René Schuster Oliver Wasenmüller Christian Unger Didier Stricker

Interpolation of sparse pixel information towards a dense target resolution finds its application across multiple disciplines in computer vision. State-of-the-art interpolation of motion fields applies model-based interpolation that makes use of edge information extracted from the target image. For depth completion, data-driven learning approaches are widespread. Our work is inspired by latest trends in depth completion that tackle the problem of dense guidance for sparse information. We extend these ideas and create a generic cross-domain architecture that can be applied for a multitude of interpolation problems like optical flow, scene flow, or depth completion. In our experiments, we show that our proposed concept of Sparse Spatial Guided Propagation (SSGP) achieves improvements to robustness, accuracy, or speed compared to specialized algorithms.

Interpolation

Robustness

Optical Flow

Image scaling

10.1109/wacv48630.2021.00024

Cite

Citations (27)

Patent images - a glass-encased tool

Mihai Lupu René Schuster Roland Mörzinger Florina Piroi Tobias Schleser

The paper discusses the problem of patent image retrieval. It describes the issues faced when extracting semantic data of images in patents, as well as an integration framework between the data thus extracted and semantic information extracted from text. Combining the two sources of knowledge is on the wish list of many patent information users, as current systems either search only the textual data, or have extremely limited image processing functionality. In practice in the patent domain, depictions of the product or method are often vital to the understanding of the invention. Yet they are almost completely unsearchable. They are tools enclosed in a glass case, at which we can look, but of which we cannot really make use. The IMPEx Project (Image Mining for Patent Exploration) cracks open this case with a new focus on processing this particular type of images. This paper presents the motivations, status and aims of the project.

Patent visualisation

10.1145/2362456.2362477

Cite

Citations (8)

DeepLiDARFlow: A Deep Learning Architecture For Scene Flow Estimation Using Monocular Camera and Sparse LiDAR

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2020)

Rishav Rishav Ramy Battrawy René Schuster Oliver Wasenmüller Didier Stricker

Scene flow is the dense 3D reconstruction of motion and geometry of a scene. Most state-of-the-art methods use a pair of stereo images as input for full scene reconstruction. These methods depend a lot on the quality of the RGB images and perform poorly in regions with reflective objects, shadows, ill-conditioned light environment and so on. LiDAR measurements are much less sensitive to the aforementioned conditions but LiDAR features are in general unsuitable for matching tasks due to their sparse nature. Hence, using both LiDAR and RGB can potentially overcome the individual disadvantages of each sensor by mutual improvement and yield robust features which can improve the matching process. In this paper, we present DeepLiDARFlow, a novel deep learning architecture which fuses high level RGB and LiDAR features at multiple scales in a monocular setup to predict dense scene flow. Its performance is much better in the critical regions where image-only and LiDAR-only methods are inaccurate. We verify our DeepLiDARFlow using the established data sets KITTI and FlyingThings3D and we show strong robustness compared to several state-of-the-art methods which used other input modalities. The code of our paper is available at https://github.com/dfki-av/DeepLiDARFlow.

Robustness

RGB color model

Monocular

Optical Flow

10.1109/iros45743.2020.9341077

Cite

Citations (34)

SDC - Stacked Dilated Convolution: A Unified Descriptor Network for Dense Matching Tasks

arXiv (Cornell University) (2019)

René Schuster Oliver Wasenmüller Christian Unger Didier Stricker

Dense pixel matching is important for many computer vision tasks such as disparity and flow estimation. We present a robust, unified descriptor network that considers a large context region with high spatial variance. Our network has a very large receptive field and avoids striding layers to maintain spatial resolution. These properties are achieved by creating a novel neural network layer that consists of multiple, parallel, stacked dilated convolutions (SDC). Several of these layers are combined to form our SDC descriptor network. In our experiments, we show that our SDC features outperform state-of-the-art feature descriptors in terms of accuracy and robustness. In addition, we demonstrate the superior performance of SDC in state-of-the-art stereo matching, optical flow and scene flow algorithms on several famous public benchmarks.

Robustness

Convolution (computer science)

Optical Flow

Feature (linguistics)

10.48550/arxiv.1904.03076

Cite

Citations (0)

FlowFields++: Accurate Optical Flow Correspondences Meet Robust Interpolation

René Schuster Christian Bailer Oliver Wasenmüller Didier Stricker

Optical Flow algorithms are of high importance for many applications. Recently, the Flow Field algorithm and its modifications have shown remarkable results, as they have been evaluated with top accuracy on different data sets. In our analysis of the algorithm we have found that it produces accurate sparse matches, but there is room for improvement in the interpolation. Thus, we propose in this paper FlowFields++, where we combine the accurate matches of Flow Fields with a robust interpolation. In addition, we propose improved variational optimization as post-processing. Our new algorithm is evaluated on the challenging KITTI and MPI Sintel data sets with public top results on both benchmarks.

Interpolation

Optical Flow

10.1109/icip.2018.8451182

Cite

Citations (24)

CLEO: Continual Learning of Evolving Ontologies

Lecture notes in computer science (2024)

Shishir Muralidhara Syed Saqib Georg Schneider Didier Stricker René Schuster

10.1007/978-3-031-72949-2_19

Cite

Citations (0)

SSGP: Sparse Spatial Guided Propagation for Robust and Generic Interpolation

arXiv (Cornell University) (2020)

René Schuster Oliver Wasenmüller Christian Unger Didier Stricker

Interpolation

Robustness

Optical Flow

Image scaling

Source

Cite

Citations (2)

EvLiDAR-Flow: Attention-Guided Fusion Between Point Clouds and Events for Scene Flow Estimation

Ankit Sonthalia Ramy Battrawy René Schuster Didier Stricker

Optical Flow

10.5220/0011615900003411

Cite

Citations (0)

ShapeAug: Occlusion Augmentation for Event Camera Data

arXiv (Cornell University) (2024)

Katharina Bendig René Schuster Didier Stricker

Recently, Dynamic Vision Sensors (DVSs) sparked a lot of interest due to their inherent advantages over conventional RGB cameras. These advantages include a low latency, a high dynamic range and a low energy consumption. Nevertheless, the processing of DVS data using Deep Learning (DL) methods remains a challenge, particularly since the availability of event training data is still limited. This leads to a need for event data augmentation techniques in order to improve accuracy as well as to avoid over-fitting on the training data. Another challenge especially in real world automotive applications is occlusion, meaning one object is hindering the view onto the object behind it. In this paper, we present a novel event data augmentation approach, which addresses this problem by introducing synthetic events for randomly moving objects in a scene. We test our method on multiple DVS classification datasets, resulting in an relative improvement of up to 6.5 % in top1-accuracy. Moreover, we apply our augmentation technique on the real world Gen1 Automotive Event Dataset for object detection, where we especially improve the detection of pedestrians by up to 5 %.

Event data

RGB color model

10.48550/arxiv.2401.02274

Cite

Citations (2)