3D Depth & Terrestrial Dehazing
Interactive 3D elevation wireframe rendering and single-image terrestrial dehazing parallax
Framework Architecture at a Glance
Multi-Modal Fusion
Combines optical LISS-IV visual bands with cloud-penetrating Sentinel-1 SAR C-band microwave radar data to reconstruct physical ground geometry.
Generative AI Core
Leverages Latent Diffusion Models (LDM) and multi-modal transformers with cross-attention to synthesize fine textures and preserve spatial resolution.
Downstream LULC Enhancement
Restores the semantic consistency of land cover. Cloud shadows no longer misclassify as water bodies; cloud peaks no longer hide infrastructure.
Hackathon Solution Showcase
Use the Interactive Sandbox in the sidebar to run simulations of different generative algorithms on four distinct regions in the North East:
- Shillong (Valley & Forest): Learn how vegetation textures are restored.
- Cherrapunji (Heavy Cloud): See how deep convective cloud cover is removed using radar cues.
- Guwahati (Urban Riverine): Witness how city outlines and Brahmaputra riverbanks are mapped.
- Gangtok (High Snow Peaks): Differentiate between clouds and ice peaks.
Spectral and Structural Accuracy
Average Structural Similarity Index (SSIM) across test sets using Latent Diffusion fusion.
Scene Configuration
Our primary architecture projects cloudy LISS-IV, Sentinel-1 SAR, and temporal references into a compressed latent space via an Encoder. It then performs reverse diffusion denoising guided by SAR spatial cross-attention, which forces spectral and spatial coherence where optical sensors see only cloud tops.
Uses self-attention within optical frames and cross-attention across Sentinel-1 SAR and Sentinel-2 optical temporal series. Patches containing cloud cover are designated as tokens and query the corresponding SAR patch to fill missing details.
Downstream Land Use - Land Cover Accuracy
Comparing classification accuracy (%) on LISS-IV cloudy images versus CLEAR_SEE reconstructed images. Cloud removal prevents severe misclassifications.
Spectral Consistency Analysis
Spectral reflectance curve alignment showing LDM reconstruction vs. Cloud-free Ground Truth across Green, Red, and Near-Infrared (NIR) bands.
LISS-IV Sensor Specifications
The LISS-IV (Linear Imaging Self Scanning Sensor-IV) is a high-resolution multispectral camera onboard ISRO's Resourcesat-2/2A satellites. It operates in three spectral bands:
| Spectral Band | Spectral Range | Spatial Resolution |
|---|---|---|
| Band 2 (Green) | 0.52 - 0.59 µm | 5.8 meters (Nadir) |
| Band 3 (Red) | 0.62 - 0.68 µm | 5.8 meters (Nadir) |
| Band 4 (NIR) | 0.77 - 0.86 µm | 5.8 meters (Nadir) |
While 5.8m resolution provides exceptional clarity for agricultural parcels and urban grids, LISS-IV lacks a Short-Wave Infrared (SWIR) or thermal band, which are traditionally used for thin cloud penetration and cloud-shadow mask extraction. Hence, Generative AI models are crucial to infer surface features.
Auxiliary Sensor Guidance (Sentinel-1 SAR)
Sentinel-1 is a C-band Synthetic Aperture Radar (SAR) satellite mission. Unlike optical sensors, C-band microwaves (wavelength ~5.6 cm) penetrate cloud layers, dust, and rain without scattering.
Polarimetric SAR Features
Our model utilizes dual polarization: VV (Vertical-transmit, Vertical-receive) which is highly sensitive to surface roughness, water body shorelines, and soil moisture; and VH (Vertical-transmit, Horizontal-receive) which is sensitive to volume scattering from vegetation canopies.
By inputting the SAR co-polarized and cross-polarized bands into the cross-attention blocks of the generator, the model establishes solid boundary conditions for roads, forests, and rivers, avoiding hallucination and stabilizing generative textures.
Algorithm Comparison Matrix
| Model Architecture | Key Feature | Spectral Fidelity (SAM) | Spatial Resolution (SSIM) | Inference Time |
|---|---|---|---|---|
| Latent Diffusion (LDM) | Iterative reverse diffusion with SAR Cross-Attention | 0.052 rad (Excellent) | 0.948 (Superior) | Medium (1.2 sec / patch) |
| Multi-modal Transformer | Cross-attention token mapping over temporal stack | 0.068 rad (Good) | 0.902 (Very Good) | Fast (0.3 sec / patch) |
| Spectral CycleGAN | Unpaired image-to-image translation | 0.145 rad (Moderate) | 0.801 (Moderate) | Real-time (0.04 sec / patch) |
Interactive 3D Surface Topography Simulator Procedural 3D Mesh Projection
Satellite-based surface reconstruction models physical terrain elevation. In this active 3D viewport, drag to rotate the mountain model to see how cloud cover blocks optical visibility, and see how the multi-modal generative engine leverages Sentinel-1 SAR values to rebuild the elevation mesh underneath.
Terrestrial Atmospheric Dehazing Baseline Atmospheric Attenuation Correction
Compare ground-level dehazing on the brick building photo. Drag the comparison slider to slide between the **hazy input image (a)** and the **dehazed recovery (b)**. Unlike satellite clouds, haze scatters light but preserves some low-frequency spatial outlines, enabling transmission modeling without radar fusion.
Parallax Depth Segmentation Explanation
To reconstruct depth structures from a single 2D image, the model performs **Semantic Edge Slicing**. It separates the image into depth layers:
- Foreground (Foliage & Tree): Fast lateral displacement.
- Midground (Bushes & Fence): Moderate displacement.
- Background (Brick Building): Stationary baseline.