Deep learning approach for realistic traffic video changes across lighting and weather conditions

Alam, M. S., Parmar, S. H., Martens, M. H., Bazilinskyy, P.

8th International Conference on Information and Computer Technologies (ICICT). Hilo, HI, USA (2025)

ABSTRACT Recent advances in GAN-based architectures have led to innovative methods for image transformation. The lack of diversity of environmental factors, such as lighting conditions and seasons in public data, prevents researchers from effectively studying the differences in the behaviour of road users under varying conditions. This study introduces a deep learning pipeline that combines CycleGAN-turbo and Real-ESRGAN to improve video transformations of traffic scenes. Evaluated using dashcam videos from Los Angeles, London, and Hong Kong, our pipeline demonstrates a notable improvement in T-SIMM for temporal consistency during night-to-day transformations, achieving a 7.97% increase for Hong Kong, 7.35% for Los Angeles, and 3.41% for London compared to CycleGAN-turbo. PSNR and VPQ scores are comparable, but the pipeline performs better in DINO structure similarity and KL divergence, with up to 153.49% better structural fidelity in Hong Kong compared to Pix2Pix and 107.32% better compared to ToDayGAN. This approach demonstrates better realism and temporal coherence in day-to-night, night-to-day, and clear-to-rainy transitions.