Publications
in reversed chronological order
2026
- EMNLP
VIBE: Video Instruction-aligned Background Music GenerationIn EMNLP (Findings) 2026, 2026A text-and-video-to-music generation model that brings semantic control and instruction adherence to video-to-music. VIBE introduces a depth-wise cross-layer conditioning mechanism to overcome the representational bottleneck of static cross-modal conditioning in diffusion-autoregressive architectures, together with a reward-modeling taxonomy that optimizes both verifiable constraints (tempo, key) and subjective qualities (musicality, multimodal alignment) through a five-stage training curriculum.
- NeurIPS
DuplexWorld: Can Voice Agents Help You Get Through the Day?In NeurIPS (RTCA) ’26, 2026A benchmark for holistically evaluating speech-to-speech voice agents beyond database tool-calling. DuplexWorld spans six domains where voice agents are especially useful, with agents evaluated on eleven conversation types across 156 scenarios (350+ hours of dialogue), each probing conversational and analytical ability.
- EMNLP
TORUS: A Test of Rendering-Understanding Self-Coherence for Unified Audio ModelsIn EMNLP 2026 Workshop on Speech and Audio Language Models (SALMA), 2026The first self-coherence test for audio-native unified models, asking whether a model’s generation and understanding heads agree about the same audio rather than evaluating each capability in isolation. TORUS comprises 48 three-stage self-coherence tests with 432 six-option questions spanning speech, sound, and music across five task families.
- NeurIPS
Exploring Textual Guidance for Video-to-Music Generation ModelsIn NeurIPS 2026 Creative AI Track, 2026A text-conditioned video-to-music generation model for fast, high-quality, semantically and stylistically controllable background music creation. An autoregressive module aligns visual and textual inputs for high-level musical planning, and Diffusion Transformers refine the plan into high-quality audio.
2024
- Preprint
- Preprint