Kwanyong Park

Electronics and Telecommunications Research Institute

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

In So Kweon

Kootenay Association for Science & Technology

Sanghyun Woo

New York University

Inkyu Shin

Kootenay Association for Science & Technology

Joon‐Young Lee

Adobe Systems (United States)

Seoung Wug Oh

Adobe Systems (United States)

Dahun Kim

Google (United States)

Myungchul Kim

Kootenay Association for Science & Technology

Dong‐Geol Choi

Hanbat National University

Donghyun Kim

Yonsei University

Kyunghyun Lee

Korea Advanced Institute of Science and Technology

Cooperative Institutions

Korea Advanced Institute of Science and Technology

Kootenay Association for Science & Technology

Yonsei University

Korea University

Samsung (South Korea)

Hanyang University

Korea Association of Information Systems

Daejeon Institute of Science and Technology

Hanbat National University

Dankook University

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

Joint self-supervised learning and adversarial adaptation for monocular depth estimation from thermal image

Machine Vision and Applications (2023)

Ukcheol Shin Kwanyong Park Kyunghyun Lee Byeong-Uk Lee In So Kweon

Leverage (statistics)

Monocular

Robustness

Ground truth

Feature (linguistics)

10.1007/s00138-023-01404-3

Cite

Citations (2)

LabOR: Labeling Only if Required for Domain Adaptive Semantic Segmentation

arXiv (Cornell University) (2021)

Inkyu Shin Dong-Jin Kim Jae Won Cho Sanghyun Woo Kwanyong Park

Unsupervised Domain Adaptation (UDA) for semantic segmentation has been actively studied to mitigate the domain gap between label-rich source data and unlabeled target data. Despite these efforts, UDA still has a long way to go to reach the fully supervised performance. To this end, we propose a Labeling Only if Required strategy, LabOR, where we introduce a human-in-the-loop approach to adaptively give scarce labels to points that a UDA model is uncertain about. In order to find the uncertain points, we generate an inconsistency mask using the proposed adaptive pixel selector and we label these segment-based regions to achieve near supervised performance with only a small fraction (about 2.2%) ground truth points, which we call "Segment based Pixel-Labeling (SPL)". To further reduce the efforts of the human annotator, we also propose "Point-based Pixel-Labeling (PPL)", which finds the most representative points for labeling within the generated inconsistency mask. This reduces efforts from 2.2% segment label to 40 points label while minimizing performance degradation. Through extensive experimentation, we show the advantages of this new framework for domain adaptive semantic segmentation while minimizing human labor costs.

Domain Adaptation

Ground truth

Labeled data

10.48550/arxiv.2108.05570

Cite

Citations (0)

Align-and-Attend Network for Globally and Locally Coherent Video Inpainting.

British Machine Vision Conference (2019)

Sanghyun Woo Dahun Kim Kwanyong Park Joon‐Young Lee In So Kweon

We propose a novel feed-forward network for video inpainting. We use a set of sampled video frames as the reference to take visible contents to fill the hole of a target frame. Our video inpainting network consists of two stages. The first stage is an alignment module that uses computed homographies between the reference frames and the target frame. The visible patches are then aggregated based on the frame similarity to fill in the target holes roughly. The second stage is a non-local attention module that matches the generated patches with known reference patches (in space and time) to refine the previous global alignment stage. Both stages consist of large spatial-temporal window size for the reference and thus enable modeling long-range correlations between distant information and the hole regions. Therefore, even challenging scenes with large or slowly moving holes can be handled, which have been hardly modeled by existing flow-based approach. Our network is also designed with a recurrent propagation stream to encourage temporal consistency in video results. Experiments on video object removal demonstrate that our method inpaints the holes with globally and locally coherent contents.

Inpainting

Source

Cite

Citations (0)

Preserving Semantic and Temporal Consistency for Unpaired Video-to-Video Translation

Proceedings of the 30th ACM International Conference on Multimedia (2019)

Kwanyong Park Sanghyun Woo Dahun Kim Donghyeon Cho In So Kweon

In this paper, we investigate the problem of unpaired video-to-video translation. Given a video in the source domain, we aim to learn the conditional distribution of the corresponding video in the target domain, without seeing any pairs of corresponding videos. While significant progress has been made in the unpaired translation of images, directly applying these methods to an input video leads to low visual quality due to the additional time dimension. In particular, previous methods suffer from semantic inconsistency (i.e., semantic label flipping) and temporal flickering artifacts. To alleviate these issues, we propose a new framework that is composed of carefully-designed generators and discriminators, coupled with two core objective functions: 1) content preserving loss and 2) temporal consistency loss. Extensive qualitative and quantitative evaluations demonstrate the superior performance of the proposed method against previous approaches. We further apply our framework to a domain adaptation task and achieve favorable results.

10.1145/3343031.3350864

Cite

Citations (34)

Tracking by Associating Clips

arXiv (Cornell University) (2022)

Sanghyun Woo Kwanyong Park Seoung Wug Oh In So Kweon Joon‐Young Lee

The tracking-by-detection paradigm today has become the dominant method for multi-object tracking and works by detecting objects in each frame and then performing data association across frames. However, its sequential frame-wise matching property fundamentally suffers from the intermediate interruptions in a video, such as object occlusions, fast camera movements, and abrupt light changes. Moreover, it typically overlooks temporal information beyond the two frames for matching. In this paper, we investigate an alternative by treating object association as clip-wise matching. Our new perspective views a single long video sequence as multiple short clips, and then the tracking is performed both within and between the clips. The benefits of this new approach are two folds. First, our method is robust to tracking error accumulation or propagation, as the video chunking allows bypassing the interrupted frames, and the short clip tracking avoids the conventional error-prone long-term track memory management. Second, the multiple frame information is aggregated during the clip-wise matching, resulting in a more accurate long-range track association than the current frame-wise matching. Given the state-of-the-art tracking-by-detection tracker, QDTrack, we showcase how the tracking performance improves with our new tracking formulation. We evaluate our proposals on two tracking benchmarks, TAO and MOT17 that have complementary characteristics and challenges each other.

Tracking (education)

Tracking system

Tracking error

Chunking (psychology)

CLIPS

10.48550/arxiv.2212.10149

Cite

Citations (0)

Align-and-Attend Network for Globally and Locally Coherent Video Inpainting

arXiv (Cornell University) (2019)

Sanghyun Woo Dahun Kim Kwanyong Park Joon‐Young Lee In So Kweon

Inpainting

Similarity (geometry)

10.48550/arxiv.1905.13066

Cite

Citations (0)

Discover, Hallucinate, and Adapt: Open Compound Domain Adaptation for Semantic Segmentation

Neural Information Processing Systems (2020)

Kwanyong Park Sanghyun Woo Inkyu Shin In So Kweon

Unsupervised domain adaptation (UDA) for semantic segmentation has been attracting attention recently, as it could be beneficial for various label-scarce real-world scenarios (e.g., robot control, autonomous driving, medical imaging, etc.). Despite the significant progress in this field, current works mainly focus on a single-source single-target setting, which cannot handle more practical settings of multiple targets or even unseen targets. In this paper, we investigate open compound domain adaptation (OCDA), which deals with mixed and novel situations at the same time, for semantic segmentation. We present a novel framework based on three main design principles: discover, hallucinate, and adapt. The scheme first clusters compound target data based on style, discovering multiple latent domains (discover). Then, it hallucinates multiple latent target domains in source by using image-translation (hallucinate). This step ensures the latent domains in the source and the target to be paired. Finally, target-to-source alignment is learned separately between domains (adapt). In high-level, our solution replaces a hard OCDA problem with much easier multiple UDA problems. We evaluate our solution on standard benchmark GTA to C-driving, and achieved new state-of-the-art results.

Hallucinating

Benchmark (surveying)

Source

Cite

Citations (2)

Weak-to-Strong Compositional Learning from Generative Models for Language-based Object Detection

arXiv (Cornell University) (2024)

Kwanyong Park Kuniaki Saito Donghyun Kim

Vision-language (VL) models often exhibit a limited understanding of complex expressions of visual objects (e.g., attributes, shapes, and their relations), given complex and diverse language queries. Traditional approaches attempt to improve VL models using hard negative synthetic text, but their effectiveness is limited. In this paper, we harness the exceptional compositional understanding capabilities of generative foundational models. We introduce a novel method for structured synthetic data generation aimed at enhancing the compositional understanding of VL models in language-based object detection. Our framework generates densely paired positive and negative triplets (image, text descriptions, and bounding boxes) in both image and text domains. By leveraging these synthetic triplets, we transform 'weaker' VL models into 'stronger' models in terms of compositional understanding, a process we call "Weak-to-Strong Compositional Learning" (WSCL). To achieve this, we propose a new compositional contrastive learning formulation that discovers semantics and structures in complex descriptions from synthetic triplets. As a result, VL models trained with our synthetic data generation exhibit a significant performance boost in the Omnilabel benchmark by up to +5AP and the D3 benchmark by +6.9AP upon existing baselines.

Generative model

10.48550/arxiv.2407.15296

Cite

Citations (0)

An energy efficient extraction method of wet microalgae with synergy effect of shear stress and functional polymeric membrane

Kwanyong Park Young Min Yoo Ji‐Won Yang Jong-Hee Kwon Sung Gap Im

Source

Cite

Citations (0)

Bridging Images and Videos: A Simple Learning Framework for Large Vocabulary Video Object Detection

Lecture notes in computer science (2022)

Sanghyun Woo Kwanyong Park Seoung Wug Oh In So Kweon Joon‐Young Lee