defense 2025

VisuoAlign: Safety Alignment of LVLMs with Multimodal Tree Search

MingSheng Li 1, Guangze Zhao 1, Sichen Liu 2

1 Harbin Institute of Technology

2 Xi’an Jiaotong-Liverpool University

0 citations · 45 references · arXiv
α

Published on arXiv

2510.15948

Input Manipulation Attack

OWASP ML Top 10 — ML01

Prompt Injection

OWASP LLM Top 10 — LLM01

Key Finding

VisuoAlign achieves alignment safety rates above 0.92 while substantially reducing jailbreak success rates, outperforming all baseline methods on four safety benchmarks.

VisuoAlign

Novel technique introduced

Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal perception and generation, yet their safety alignment remains a critical challenge.Existing defenses and vulnerable to multimodal jailbreaks, as visual inputs introduce new attack surfaces, reasoning chains lack safety supervision, and alignment often degrades under modality fusion.To overcome these limitation, we propose VisuoAlign, a framework for multi-modal safety alignment via prompt-guided tree search.VisuoAlign embeds safety constrains into the reasoning process through visual-textual interactive prompts, employs Monte Carlo Tree Search(MCTS) to systematically construct diverse safety-critical prompt trajectories, and introduces prompt-based scaling to ensure real-time risk detection and compliant responses.Extensive experiments demonstrate that VisuoAlign proactively exposes risks, enables comprehensive dataset generation, and significantly improves the robustness of LVLMs against complex cross-modal threats.

Key Contributions

  • Embeds safety constraints into multimodal reasoning via visual-textual interactive prompts, achieving intrinsic safety awareness across modalities
  • Uses Monte Carlo Tree Search (MCTS) to systematically generate diverse safety-critical prompt trajectories for alignment dataset construction
  • Introduces prompt-based safety scaling at inference time for real-time risk detection and compliant response generation

🛡️ Threat Analysis

Input Manipulation Attack

The threat model explicitly includes adversarial visual perturbations and hidden prompt injections in images that bypass text-based filters in VLMs — qualifying for dual ML01+LLM01 tagging per the multimodal adversarial visual input rule. VisuoAlign defends against these visual attack surfaces.

Details

Domains
visionnlpmultimodal
Model Types
vlmllm
Threat Tags
inference_timetraining_time
Datasets
safety benchmarks (4, names not specified in excerpt)
Applications
large vision-language modelsmultimodal safety alignmentcross-modal jailbreak defense
Read PDF arXiv DOI

Similar Papers

defense

CrossGuard: Safeguarding MLLMs against Joint-Modal Implicit Malicious Attacks

2025 0 cit.
Input Manipulation Attack
89%
defense

Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoring

2025 0 cit.
Input Manipulation Attack
89%
defense

Directional Embedding Smoothing for Robust Vision Language Models

2026 0 cit.
Input Manipulation Attack
84%
defense

Two Birds, One Projection: Harmonizing Safety and Utility in LVLMs via Inference-time Feature Projection

2026 0 cit.
Input Manipulation Attack
84%
defense

Agentic Moderation: Multi-Agent Design for Safer Vision-Language Models

2025 1 cit.
Input Manipulation Attack
84%
defense

Learning to Detect Unseen Jailbreak Attacks in Large Vision-Language Models

2025 0 cit.
Input Manipulation Attack
84%
defense

VALD: Multi-Stage Vision Attack Detection for Efficient LVLM Defense

2026 0 cit.
Input Manipulation Attack
84%
defense

Risk-adaptive Activation Steering for Safe Multimodal Large Language Models

2025 1 cit.
Input Manipulation Attack
84%