Yuheng Li

University of Wisconsin–Madison

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Yong Jae Lee

University of Wisconsin–Madison

Chunyuan Li

Microsoft Research (United Kingdom)

Haotian Liu

Worcester Polytechnic Institute

Utkarsh Ojha

University of Wisconsin–Madison

Haotian Liu

Southern Medical University

Krishna Kumar Singh

Adobe Systems (United States)

Eli Shechtman

Adobe Systems (United States)

Yong Jae Lee

Kookmin University

Mu Cai

University of Wisconsin–Madison

Yijun Li

Adobe Systems (United States)

Cooperative Institutions

University of Wisconsin–Madison

University of California, Davis

Adobe Systems (United States)

Peking University

Sun Yat-sen University

Chinese Academy of Sciences

Shanghai Jiao Tong University

Xidian University

Microsoft (United States)

Tsinghua University

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

Residual Attention Mechanism for Remote Sensing Target Hiding

Remote Sensing (2023)

Hao Yuan Yongjian Shen Ning Lv Yuheng Li Chen Chen

In this paper, we investigate deep-learning-based image inpainting techniques for emergency remote sensing mapping. Image inpainting can generate fabricated targets to conceal real-world private structures and ensure informational privacy. However, casual inpainting outputs may seem incongruous within original contexts. In addition, the residuals of original targets may persist in the hiding results. A Residual Attention Target-Hiding (RATH) model has been proposed to address these limitations for remote sensing target hiding. The RATH model introduces the residual attention mechanism to replace gated convolutions, thereby reducing parameters, mitigating gradient issues, and learning the distribution of targets present in the original images. Furthermore, this paper modifies the fusion module in the contextual attention layer to enlarge the fusion patch size. We extend the edge-guided function to preserve the original target information and confound viewers. Ablation studies on an open dataset proved the efficiency of RATH for image inpainting and target hiding. RATH had the highest similarity, with a 90.44% structural similarity index metric (SSIM), for edge-guided target hiding. The training parameters had 1M fewer values than gated convolution (Gated Conv). Finally, we present two automated target-hiding techniques that integrate semantic segmentation with direct target hiding or edge-guided synthesis for remote sensing mapping applications.

Inpainting

Similarity (geometry)

10.3390/rs15194731

Cite

Citations (0)

A New Intrusion Detection Algorithm AE-3WD for Industrial Control燦etwork

Journal of New Media (2022)

Yongzhong Li Cong Li Yuheng Li Shipeng Zhang

In this paper, we propose a intrusion detection algorithm based on auto-encoder and three-way decisions (AE-3WD) for industrial control networks, aiming at the security problem of industrial control network. The ideology of deep learning is similar to the idea of intrusion detection. Deep learning is a kind of intelligent algorithm and has the ability of automatically learning. It uses self-learning to enhance the experience and dynamic classification capabilities. We use deep learning to improve the intrusion detection rate and reduce the false alarm rate through learning, a denoising AutoEncoder and three-way decisions intrusion detection method AE-3WD is proposed to improve intrusion detection accuracy. In the processing, deep learning AutoEncoder is used to extract the features of high-dimensional data by combining the coefficient penalty and reconstruction loss function of the encode layer during the training mode. A multi-feature space can be constructed by multiple feature extractions from AutoEncoder, and then a decision for intrusion behavior or normal behavior is made by three-way decisions. NSL-KDD data sets are used to the experiments. The experiment results prove that our proposed method can extract meaningful features and effectively improve the performance of intrusion detection.

Autoencoder

Deep belief network

Feature (linguistics)

10.32604/jnm.2022.034778

Cite

Citations (0)

CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes

Proceedings of the AAAI Conference on Artificial Intelligence (2022)

Hao Huang Yongtao Wang Zhaoyu Chen Yuze Zhang Yuheng Li

Malicious applications of deepfakes (i.e., technologies generating target facial attributes or entire faces from facial images) have posed a huge threat to individuals' reputation and security. To mitigate these threats, recent studies have proposed adversarial watermarks to combat deepfake models, leading them to generate distorted outputs. Despite achieving impressive results, these adversarial watermarks have low image-level and model-level transferability, meaning that they can protect only one facial image from one specific deepfake model. To address these issues, we propose a novel solution that can generate a Cross-Model Universal Adversarial Watermark (CMUA-Watermark), protecting a large number of facial images from multiple deepfake models. Specifically, we begin by proposing a cross-model universal attack pipeline that attacks multiple deepfake models iteratively. Then, we design a two-level perturbation fusion strategy to alleviate the conflict between the adversarial watermarks generated by different facial images and models. Moreover, we address the key problem in cross-model optimization with a heuristic approach to automatically find the suitable attack step sizes for different models, further weakening the model-level conflict. Finally, we introduce a more reasonable and comprehensive evaluation method to fully test the proposed method and compare it with existing ones. Extensive experimental results demonstrate that the proposed CMUA-Watermark can effectively distort the fake facial images generated by multiple deepfake models while achieving a better performance than existing methods. Our code is available at https://github.com/VDIGPKU/CMUA-Watermark.

Code (set theory)

10.1609/aaai.v36i1.19982

Cite

Citations (57)

Edit One for All: Interactive Batch Image Editing

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

Thao D. Nguyen Utkarsh Ojha Yuheng Li Haotian Liu Yong Jae Lee

Image editing

10.1109/cvpr52733.2024.00790

Cite

Citations (2)

Improved Baselines with Visual Instruction Tuning

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee

10.1109/cvpr52733.2024.02484

Cite

Citations (221)

Information-Theoretic Domain Adaptative Training of Medicine Language Models with Rigorous Generalization

2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (2024)

Guiqiang Zhang Haowen Jiang Yang Li Daochun Li Yuheng Li

Domain-specific language

10.1109/bibm62325.2024.10822365

Cite

Citations (0)

MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generation

arXiv (Cornell University) (2019)

Yuheng Li Krishna Kumar Singh Utkarsh Ojha Yong‐Jae Lee

We present MixNMatch, a conditional generative model that learns to disentangle and encode background, object pose, shape, and texture from real images with minimal supervision, for mix-and-match image generation. We build upon FineGAN, an unconditional generative model, to learn the desired disentanglement and image generator, and leverage adversarial joint image-code distribution matching to learn the latent factor encoders. MixNMatch requires bounding boxes during training to model background, but requires no other supervision. Through extensive experiments, we demonstrate MixNMatch's ability to accurately disentangle, encode, and combine multiple factors for mix-and-match image generation, including sketch2color, cartoon2img, and img2gif applications. Our code/models/demo can be found at https://github.com/Yuheng-Li/MixNMatch

Leverage (statistics)

ENCODE

Bounding overwatch

Code (set theory)

Generative model

10.48550/arxiv.1911.11758

Cite

Citations (2)

Residual Attention Mechanism for Remote Sensing Target Hiding

Hao Yuan Yongjian Shen Ning Lv Yuheng Li Chen Chen

Remote sensing imagery is of great significance for policy decisions, especially for disaster assessment and disaster relief. To ensure the privacy and inviolability of personal buildings, the information containing these buildings must be anonymized during the remote sensing mapping process. Traditional processing methods for these targets in remote sensing mapping are mainly based on manual retrieval and image editing tools, which are inefficient. Deep learning provides a new direction for target hiding. Although the image inpainting method based on deep learning is faster than the manual method, the cost of training calculation is a disadvantage. And the element-wise product operation used in the model increases the risk of vanished or exploded gradients. We propose a Residual Attention Target Hiding (RATH) model for remote sensing target hiding based on deep learning. RATH uses residual attention modules to replace gated convolutions, reducing parameters and mitigating gradient issues. The residual attention module preserves gated convolution performance but provides an adjustable kernel size. RATH retains gated convolutions for dynamic feature selection and balances model depth and width. Furthermore, this paper modifies the contextual attention layer by adjusting the fusion process to enlarge the fusion patch size. Finally, we extend the edge-guided function to preserve the original target information and confound viewers. Ablation studies on an open dataset prove RATH’s efficiency for image inpainting and target hiding. RATH achieves state-of-the-art results with lower complexity. And it has the highest similarity for edge-guided target hiding. RATH enables robust, efficient target hiding for privacy protection in remote sensing imagery while balancing performance and complexity. Experiments show RATH's superiority over existing methods in hiding arbitrary-shaped targets.

Convolution (computer science)

Feature (linguistics)

Inpainting

10.20944/preprints202307.1811.v1

Cite

Citations (1)

Leveraging Large Language Models for Scalable Vector Graphics-Driven Image Understanding

arXiv (Cornell University) (2023)

Mu Cai Zeyi Huang Yuheng Li Haohan Wang Yong Jae Lee

Recently, large language models (LLMs) have made significant advancements in natural language understanding and generation. However, their potential in computer vision remains largely unexplored. In this paper, we introduce a new, exploratory approach that enables LLMs to process images using the Scalable Vector Graphics (SVG) format. By leveraging the XML-based textual descriptions of SVG representations instead of raster images, we aim to bridge the gap between the visual and textual modalities, allowing LLMs to directly understand and manipulate images without the need for parameterized visual components. Our method facilitates simple image classification, generation, and in-context learning using only LLM capabilities. We demonstrate the promise of our approach across discriminative and generative tasks, highlighting its (i) robustness against distribution shift, (ii) substantial improvements achieved by tapping into the in-context learning abilities of LLMs, and (iii) image understanding and generation capabilities with human guidance. Our code, data, and models can be found here https://github.com/mu-cai/svg-llm.

Scalable Vector Graphics

Software portability

Robustness

Modalities

Vector graphics

10.48550/arxiv.2306.06094

Cite

Citations (0)

Collaging Class-specific GANs for Semantic Image Synthesis

2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021)

Yuheng Li Yijun Li Jingwan Lu Eli Shechtman Yong Jae Lee

We propose a new approach for high resolution semantic image synthesis. It consists of one base image generator and multiple class-specific generators. The base generator generates high quality images based on a segmentation map. To further improve the quality of different objects, we create a bank of Generative Adversarial Networks (GANs) by separately training class-specific models. This has several benefits including – dedicated weights for each class; centrally aligned data for each model; additional training data from other sources, potential of higher resolution and quality; and easy manipulation of a specific object in the scene. Experiments show that our approach can generate high quality images in high resolution while having flexibility of object-level control by using class-specific generators.

Base (topology)

10.1109/iccv48922.2021.01415

Cite

Citations (27)