Yuhang Lai

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Xuebing Leng

Shanghai Institute of Organic Chemistry

Xuanjing Huang

Fudan University

Zhongyu Wei

Fudan University

Jie Xiao

Helmholtz-Zentrum Berlin für Materialien und Energie

Liang Deng

University of Chinese Academy of Sciences

Peng Wang

Southwest University

Shujun Liu

Cardiff University

Dongyang Wang

Zhengzhou University

Siyuan Wang

Fudan University

Liang Deng

Cooperative Institutions

Chinese Academy of Sciences

Fudan University

University of Chinese Academy of Sciences

Chongqing University

Shanghai Institute of Organic Chemistry

Jilin University

Shanghai Jiao Tong University

Dalian University of Technology

Microsoft Research (United Kingdom)

Southern University of Science and Technology

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

CCDC 2086750: Experimental Crystal Structure Determination

The Cambridge Structural Database (2021)

Dongyang Wang Yuhang Lai Peng Wang Xuebing Leng Jie Xiao

10.5517/ccdc.csd.cc281fj9

Cite

Citations (0)

CCDC 2086752: Experimental Crystal Structure Determination

The Cambridge Structural Database (2021)

Dongyang Wang Yuhang Lai Peng Wang Xuebing Leng Jie Xiao

10.5517/ccdc.csd.cc281flc

Cite

Citations (0)

CCDC 2086753: Experimental Crystal Structure Determination

The Cambridge Structural Database (2021)

Dongyang Wang Yuhang Lai Peng Wang Xuebing Leng Jie Xiao

10.5517/ccdc.csd.cc281fmd

Cite

Citations (0)

How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation

arXiv (Cornell University) (2025)

Zheng‐Wen Long Siyuan Wang Shujun Liu Yuhang Lai Xuanjing Huang

Jailbreak attacks, where harmful prompts bypass generative models' built-in safety, raise serious concerns about model vulnerability. While many defense methods have been proposed, the trade-offs between safety and helpfulness, and their application to Large Vision-Language Models (LVLMs), are not well understood. This paper systematically examines jailbreak defenses by reframing the standard generation task as a binary classification problem to assess model refusal tendencies for both harmful and benign queries. We identify two key defense mechanisms: safety shift, which increases refusal rates across all queries, and harmfulness discrimination, which improves the model's ability to distinguish between harmful and benign inputs. Using these mechanisms, we develop two ensemble defense strategies-inter-mechanism ensembles and intra-mechanism ensembles-to balance safety and helpfulness. Experiments on the MM-SafetyBench and MOSSBench datasets with LLaVA-1.5 models show that these strategies effectively improve model safety or optimize the trade-off between safety and helpfulness.

10.48550/arxiv.2502.14486

Cite

Citations (0)

ALaRM: Align Language Models via Hierarchical Rewards Modeling

arXiv (Cornell University) (2024)

Yuhang Lai Siyuan Wang Shujun Liu Xuanjing Huang Zhongyu Wei

We introduce ALaRM, the first framework modeling hierarchical rewards in reinforcement learning from human feedback (RLHF), which is designed to enhance the alignment of large language models (LLMs) with human preferences. The framework addresses the limitations of current alignment approaches, which often struggle with the inconsistency and sparsity of human supervision signals, by integrating holistic rewards with aspect-specific rewards. This integration enables more precise and consistent guidance of language models towards desired outcomes, particularly in complex and open text generation tasks. By employing a methodology that filters and combines multiple rewards based on their consistency, the framework provides a reliable mechanism for improving model alignment. We validate our approach through applications in long-form question answering and machine translation tasks, employing gpt-3.5-turbo for pairwise comparisons, and demonstrate improvements over existing baselines. Our work underscores the effectiveness of hierarchical rewards modeling in refining LLM training processes for better human preference alignment. We release our code at https://ALaRM-fdu.github.io.

Hierarchical database model

10.48550/arxiv.2403.06754

Cite

Citations (0)

CCDC 2086754: Experimental Crystal Structure Determination

The Cambridge Structural Database (2021)

Dongyang Wang Yuhang Lai Peng Wang Xuebing Leng Jie Xiao

10.5517/ccdc.csd.cc281fnf

Cite

Citations (0)

CCDC 2086751: Experimental Crystal Structure Determination

The Cambridge Structural Database (2021)

Dongyang Wang Yuhang Lai Peng Wang Xuebing Leng Jie Xiao

10.5517/ccdc.csd.cc281fkb

Cite

Citations (0)

ALaRM: Align Language Models via Hierarchical Rewards Modeling

Findings of the Association for Computational Linguistics: ACL 2022 (2024)

Yuhang Lai Siyuan Wang Shujun Liu Xuanjing Huang Zhongyu Wei

Modeling language

Hierarchical database model

10.18653/v1/2024.findings-acl.465

Cite

Citations (0)

HAF-RM: A Hybrid Alignment Framework for Reward Model Training

arXiv (Cornell University) (2024)

Shujun Liu Xiaoyu Shen Yuhang Lai Siyuan Wang Shengbin Yue

The reward model has become increasingly important in alignment, assessment, and data construction for large language models (LLMs). Most existing researchers focus on enhancing reward models through data improvements, following the conventional training framework for reward models that directly optimizes the predicted rewards. In this paper, we propose a hybrid alignment framework HaF-RM for reward model training by introducing an additional constraint on token-level policy probabilities in addition to the reward score. It can simultaneously supervise the internal preference model at the token level and optimize the mapping layer of the reward model at the sequence level. Theoretical justifications and experiment results on five datasets show the validity and effectiveness of our proposed hybrid framework for training a high-quality reward model. By decoupling the reward modeling procedure and incorporating hybrid supervision, our HaF-RM framework offers a principled and effective approach to enhancing the performance and alignment of reward models, a critical component in the responsible development of powerful language models. We release our code at https://haf-rm.github.io.

10.48550/arxiv.2407.04185

Cite

Citations (0)