Weather-scene semantics
A ResNet image stream is aligned with CLIP-derived weather prototypes and refined with a Kolmogorov-Arnold Network adapter to form compact, weather-discriminative embeddings.
Robust visual perception
OmniRestore
Can one compact model restore vision across rain, fog, snow, low light, and compound weather?
Presented at the NTIRE Workshop on New Trends in Image Restoration and Enhancement at CVPR 2026
Why I built it
How can semantic knowledge about weather guide image restoration without running a large vision-language model for every image?
Autonomous systems need reliable visual perception in the conditions where cameras struggle most. Many restoration models are trained for one degradation at a time, while universal models often become too large for practical edge deployment.
OmniRestore is my answer to that efficiency gap. Coauthored with Prof. Diksha Shukla, it combines a weather-scene embedder with a lightweight restoration backbone so one model can identify the degradation context and adapt its reconstruction strategy.
Qualitative restoration results place the degraded input beside competing reconstructions, the OmniRestore output, and the clean ground truth. The comparisons cover CDD-11, Snow100K, WeatherBench, and LOL scenes.
How the system works
Lightweight restoration
A parameter-efficient U-Net-style backbone injects the condition embedding through hybrid attention, shifted-window self-attention, and channel recalibration.
Adaptive refinement
Condition-dependent modulation suppresses residual artifacts and improves perceptual reconstruction across single and composite degradations.
2.60Minference-time parameters
29.86 dBPSNR on CDD-11
0.9244SSIM on CDD-11
6.3 msinference on 256 x 256 inputs
What this work contributes
From a research question to an inspectable system.
- Universal restoration across rain, fog, snow, low light, and composite weather.
- A 56.5 percent parameter reduction relative to OneRestore while improving CDD-11 PSNR by 1.14 dB.
- No text encoder or large vision-language model is executed per image at inference time.
- Evaluation across CDD-11, Snow100K, LOL, and WeatherBench.
How I read the results
What the evidence means
The restored-image comparisons matter because aggregate metrics can hide failure modes. Looking across degraded input, reconstructed output, and ground truth shows whether the model removes weather artifacts without erasing road structure, vehicle boundaries, illumination cues, or fine scene detail. OmniRestore was designed to improve that visual fidelity while remaining small enough for resource-constrained perception systems.
Current scope
What has been validated so far
The current study focuses on image restoration as the first stage of a robust perception pipeline. The next evaluation will measure how restored images affect downstream detection, tracking, and scene understanding under real, mixed weather.
Inside the system
More evidence from the research.
Where I am taking it next