Reasoning in the Dark: Event-RGB Adaptation for Low-Light Video MLLMs

Abstract

Video multimodal large language models (video MLLMs) rely on RGB appearance cues and degrade sharply in low light, where texture, color, and object boundaries are lost. Event cameras offer a complementary signal, asynchronous brightness changes that preserve motion and temporal structure under extreme illumination, but integrating event streams into language-based reasoning is bottlenecked by scarce paired event-language supervision, especially under low light. We propose LITE-Event, a sample-efficient framework that fuses event and RGB streams for low-light video reasoning while keeping both visual encoders and the LLM frozen. A lightweight visual-language interface combines modality-specific projectors with gated fusion and is adapted at inference by a hypernetwork that generates LoRA-style residual updates from a small support set, allowing the model to specialize without retraining. To support training and evaluation, we release three resources: NExT-QA-Dark, a large-scale synthetic low-light RGB-event dataset for pretraining; EventLang-LowLight, a curated event-language dataset for adaptation; and RealDark-Bench, a real-world low-light RGB-event reasoning benchmark. Trained predominantly on synthetic data, LITE-Event reaches 87.1% on RealDark-Bench, outperforming the strongest enhancement-then-reason baseline by 8.0 points and surpassing both RGB-only and event-only video MLLMs.