Sunghoon Im

Daegu Gyeongbuk Institute of Science and Technology

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

In So Kweon

Kootenay Association for Science & Technology

Hae‐Gon Jeon

International Graduate School of English

Hyowon Ha

META Health

Stephen Lin

Chung Shan Medical University Hospital

Seokju Lee

Kentech Instruments (United Kingdom)

Jae-Yeul Kim

Daegu Gyeongbuk Institute of Science and Technology

Jungwan Woo

Daegu Gyeongbuk Institute of Science and Technology

Seunghun Lee

Daegu Gyeongbuk Institute of Science and Technology

Minjun Kang

Kootenay Association for Science & Technology

François Rameau

SUNY Korea

Cooperative Institutions

Korea Advanced Institute of Science and Technology

Daegu Gyeongbuk Institute of Science and Technology

Kootenay Association for Science & Technology

Korea University

Seoul National University

Yonsei University

Hanyang University

Carnegie Mellon University

Daejeon Institute of Science and Technology

Pohang University of Science and Technology

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

Self-supervised Monocular Depth Estimation Robust to Reflective Surface Leveraged by Triplet Mining

arXiv (Cornell University) (2025)

Wonhyeok Choi Kyu-Min Hwang Peng Wei Minwoo Choi Sunghoon Im

Self-supervised monocular depth estimation (SSMDE) aims to predict the dense depth map of a monocular image, by learning depth from RGB image sequences, eliminating the need for ground-truth depth labels. Although this approach simplifies data acquisition compared to supervised methods, it struggles with reflective surfaces, as they violate the assumptions of Lambertian reflectance, leading to inaccurate training on such surfaces. To tackle this problem, we propose a novel training strategy for an SSMDE by leveraging triplet mining to pinpoint reflective regions at the pixel level, guided by the camera geometry between different viewpoints. The proposed reflection-aware triplet mining loss specifically penalizes the inappropriate photometric error minimization on the localized reflective regions while preserving depth accuracy in non-reflective areas. We also incorporate a reflection-aware knowledge distillation method that enables a student model to selectively learn the pixel-level knowledge from reflective and non-reflective regions. This results in robust depth estimation across areas. Evaluation results on multiple datasets demonstrate that our method effectively enhances depth quality on reflective surfaces and outperforms state-of-the-art SSMDE baselines.

Monocular

10.48550/arxiv.2502.14573

Cite

Citations (0)

Flow4D: Leveraging 4D Voxel Network for LiDAR Scene Flow Estimation

IEEE Robotics and Automation Letters (2025)

Jae-Yeul Kim Jungwan Woo Ukcheol Shin Jean Oh Sunghoon Im

10.1109/lra.2025.3542327

Cite

Citations (0)

Instance-wise Depth and Motion Learning from Monocular Videos

arXiv (Cornell University) (2019)

Seokju Lee Sunghoon Im Stephen Lin In So Kweon

We present an end-to-end joint training framework that explicitly models 6-DoF motion of multiple dynamic objects, ego-motion and depth in a monocular camera setup without supervision. Our technical contributions are three-fold. First, we propose a differentiable forward rigid projection module that plays a key role in our instance-wise depth and motion learning. Second, we design an instance-wise photometric and geometric consistency loss that effectively decomposes background and moving object regions. Lastly, we introduce a new auto-annotation scheme to produce video instance segmentation maps that will be utilized as input to our training pipeline. These proposed elements are validated in a detailed ablation study. Through extensive experiments conducted on the KITTI dataset, our framework is shown to outperform the state-of-the-art depth and motion estimation methods. Our code and dataset will be available at https://github.com/SeokjuLee/Insta-DM.

Monocular

Benchmark (surveying)

Code (set theory)

10.48550/arxiv.1912.09351

Cite

Citations (3)

DRANet: Disentangling Representation and Adaptation Networks for Unsupervised Cross-Domain Adaptation

arXiv (Cornell University) (2021)

Seunghun Lee Sunghyun Cho Sunghoon Im

In this paper, we present DRANet, a network architecture that disentangles image representations and transfers the visual attributes in a latent space for unsupervised cross-domain adaptation. Unlike the existing domain adaptation methods that learn associated features sharing a domain, DRANet preserves the distinctiveness of each domain's characteristics. Our model encodes individual representations of content (scene structure) and style (artistic appearance) from both source and target images. Then, it adapts the domain by incorporating the transferred style factor into the content factor along with learnable weights specified for each domain. This learning framework allows bi-/multi-directional domain adaptation with a single encoder-decoder network and aligns their domain shift. Additionally, we propose a content-adaptive domain transfer module that helps retain scene structure while transferring style. Extensive experiments show our model successfully separates content-style factors and synthesizes visually pleasing domain-transferred images. The proposed method demonstrates state-of-the-art performance on standard digit classification tasks as well as semantic segmentation tasks.

Optimal distinctiveness theory

Representation

Factor (programming language)

Domain Adaptation

10.48550/arxiv.2103.13447

Cite

Citations (0)

Offline-to-Online Knowledge Distillation for Video Instance Segmentation

2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (2024)

Hojin Kim Seunghun Lee Hyeon Kang Sunghoon Im

In this paper, we present offline-to-online knowledge distillation (OOKD) for video instance segmentation (VIS), which transfers a wealth of video knowledge from an offline model to an online model for consistent prediction. Unlike previous methods that have adopted either an online or offline model, our single online model takes advantage of both models by distilling offline knowledge. To transfer knowledge correctly, we propose query filtering and association (QFA), which filters irrelevant queries to exact instances. Our KD with QFA increases the robustness of feature matching by encoding object-centric features from a single frame supplemented by long-range global information. We also propose a simple data augmentation scheme for knowledge distillation in the VIS task that fairly transfers the knowledge of all classes into the online model. Extensive experiments show that our method significantly improves the performance in video instance segmentation, especially for challenging datasets, including long, dynamic sequences. Our method also achieves state-of-the-art performance on YTVIS-21, YTVIS-22, and OVIS datasets, with mAP scores of 46.1%, 43.6%, and 31.1%, respectively.

10.1109/wacv57701.2024.00023

Cite

Citations (2)

DRANet: Disentangling Representation and Adaptation Networks for Unsupervised Cross-Domain Adaptation

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

Seunghun Lee Sunghyun Cho Sunghoon Im

In this paper, we present DRANet, a network architecture that disentangles image representations and transfers the visual attributes in a latent space for unsupervised cross-domain adaptation. Unlike the existing domain adaptation methods that learn associated features sharing a domain, DRANet preserves the distinctiveness of each domain’s characteristics. Our model encodes individual representations of content (scene structure) and style (artistic appearance) from both source and target images. Then, it adapts the domain by incorporating the transferred style factor into the content factor along with learnable weights specified for each domain. This learning framework allows bi/multi-directional domain adaptation with a single encoder-decoder network and aligns their domain shift. Additionally, we propose a content-adaptive domain transfer module that helps retain scene structure while transferring style. Extensive experiments show our model successfully separates content-style factors and synthesizes visually pleasing domain-transferred images. The proposed method demonstrates state-of-the-art performance on standard digit classification tasks as well as semantic segmentation tasks.

Representation

Optimal distinctiveness theory

Factor (programming language)

Domain Adaptation

10.1109/cvpr46437.2021.01500

Cite

Citations (44)

VolumeFusion: Deep Depth Fusion for 3D Scene Reconstruction

2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021)

Jaesung Choe Sunghoon Im François Rameau Minjun Kang In So Kweon

To reconstruct a 3D scene from a set of calibrated views, traditional multi-view stereo techniques rely on two distinct stages: local depth maps computation and global depth maps fusion. Recent studies concentrate on deep neural architectures for depth estimation by using conventional depth fusion method or direct 3D reconstruction network by regressing Truncated Signed Distance Function (TSDF). In this paper, we advocate that replicating the traditional two stages framework with deep neural networks improves both the interpretability and the accuracy of the results. As mentioned, our network operates in two steps: 1) the local computation of the local depth maps with a deep MVS technique, and, 2) the depth maps and images' features fusion to build a single TSDF volume. In order to improve the matching performance between images acquired from very different viewpoints (e.g., large-baseline and rotations), we introduce a rotation-invariant 3D convolution kernel called PosedConv. The effectiveness of the proposed architecture is underlined via a large series of experiments conducted on the ScanNet dataset where our approach compares favorably against both traditional and deep learning techniques.

Depth map

Interpretability

Kernel (algebra)

Convolution (computer science)

10.1109/iccv48922.2021.01578

Cite

Citations (44)

Learning Shape-based Representation for Visual Localization in Extremely Changing Conditions

Hae‐Gon Jeon Sunghoon Im Jean Oh Martial Hebert

Visual localization is an important task for applications such as navigation and augmented reality, but is a challenging problem when there are changes in scene appearances through day, seasons, or environments. In this paper, we present a convolutional neural network (CNN)-based approach for visual localization across normal to drastic appearance variations such as pre- and post-disaster cases. Our approach aims to address two key challenges: (1) to reduce the biases based on scene textures as in traditional CNNs, our model learns a shape-based representation by training on stylized images; (2) to make the model robust against layout changes, our approach uses the estimated dominant planes of query images as approximate scene coordinates. Our method is evaluated on various scenes including a simulated disaster dataset to demonstrate the effectiveness of our method in significant changes of scene layout. Experimental results show that our method provides reliable camera pose predictions in various changing conditions.

Representation

Stylized fact

10.1109/icra40945.2020.9196842

Cite

Citations (5)

Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimation

arXiv (Cornell University) (2022)

Jin Woo Bae Sungho Moon Sunghoon Im

Self-supervised monocular depth estimation has been widely studied recently. Most of the work has focused on improving performance on benchmark datasets, such as KITTI, but has offered a few experiments on generalization performance. In this paper, we investigate the backbone networks (e.g. CNNs, Transformers, and CNN-Transformer hybrid models) toward the generalization of monocular depth estimation. We first evaluate state-of-the-art models on diverse public datasets, which have never been seen during the network training. Next, we investigate the effects of texture-biased and shape-biased representations using the various texture-shifted datasets that we generated. We observe that Transformers exhibit a strong shape bias and CNNs do a strong texture-bias. We also find that shape-biased models show better generalization performance for monocular depth estimation compared to texture-biased models. Based on these observations, we newly design a CNN-Transformer hybrid network with a multi-level adaptive feature fusion module, called MonoFormer. The design intuition behind MonoFormer is to increase shape bias by employing Transformers while compensating for the weak locality bias of Transformers by adaptively fusing multi-level representations. Extensive experiments show that the proposed method achieves state-of-the-art performance with various public datasets. Our method also shows the best generalization ability among the competitive methods.

Monocular

10.48550/arxiv.2205.11083

Cite

Citations (2)

Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimation

Proceedings of the AAAI Conference on Artificial Intelligence (2023)

Jin Woo Bae Sungho Moon Sunghoon Im

Self-supervised monocular depth estimation has been widely studied recently. Most of the work has focused on improving performance on benchmark datasets, such as KITTI, but has offered a few experiments on generalization performance. In this paper, we investigate the backbone networks (e.g., CNNs, Transformers, and CNN-Transformer hybrid models) toward the generalization of monocular depth estimation. We first evaluate state-of-the-art models on diverse public datasets, which have never been seen during the network training. Next, we investigate the effects of texture-biased and shape-biased representations using the various texture-shifted datasets that we generated. We observe that Transformers exhibit a strong shape bias and CNNs do a strong texture-bias. We also find that shape-biased models show better generalization performance for monocular depth estimation compared to texture-biased models. Based on these observations, we newly design a CNN-Transformer hybrid network with a multi-level adaptive feature fusion module, called MonoFormer. The design intuition behind MonoFormer is to increase shape bias by employing Transformers while compensating for the weak locality bias of Transformers by adaptively fusing multi-level representations. Extensive experiments show that the proposed method achieves state-of-the-art performance with various public datasets. Our method also shows the best generalization ability among the competitive methods.

Monocular

10.1609/aaai.v37i1.25090

Cite

Citations (59)