Deep learning-enhanced LiDAR and vision fusion for robust object detection in autonomous driving under adverse weather

In increasingly complex traffic environments, autonomous driving systems demand higher levels of perception accuracy and robustness—particularly under adverse weather conditions such as rain, fog, and low illumination. Traditional single-modal perception frameworks tend to suffer from severe performance degradation in these scenarios. To tackle the challenges of semantic inconsistency and spatial misalignment in heterogeneous multimodal data, this paper proposes LiViFusionNet, a novel deep fusion architecture that integrates LiDAR point clouds with RGB image features to enhance 3D object detection performance and environmental adaptability. LiViFusionNet employs a dual-branch encoder, using ResNet-50 and PointNet++ to extract modality-specific features, followed by a Cross-Attention-based fusion module for deep semantic interaction. These features are then projected into a unified Bird’s Eye View (BEV) space to ensure spatial alignment, and fed into a Transformer-based detection head for end-to-end object prediction. Extensive experiments on DAWN, nuScenes, and Waymo datasets demonstrate that the proposed method achieves superior performance over state-of-the-art models across multiple metrics, including mAP, BEV IoU, and inference latency. Notably, LiViFusionNet improves average mAP by over 6.5% in rainy and foggy conditions, while maintaining a real-time inference speed of 22 FPS on embedded platforms. Overall, the framework proposed in this paper achieves a convincing balance between detection accuracy and computational efficiency, demonstrating significant potential for deployment in real-world autonomous driving systems.