Scale-Aware Recognition in Satellite Images under Resource Constraints

Shreelekha Revankar1, Cheng Perng Phoo1, Utkarsh Mall2, Bharath Hariharan1, Kavita Bala1

1Cornell University, 2Columbia University

Paper Code arXiv
Research Overview

With these images, we can see how concept scale is linked to spatial resolution. If we are seeking out a spatially large concept like forest, lower resolutions are sufficient and favorable (b), as higher resolutions may lack the context to discern between a forest (a) and a park (c). At the same time while seeking out finer concepts such as sports track, certain details can only be discerned well at higher resolutions (d) and are obscured at lower resolutions (e).

Abstract

Recognition of features in satellite imagery (forests, swimming pools, etc.) depends strongly on the spatial scale of the concept and therefore the resolution of the images. This poses two challenges: Which resolution is best suited for recognizing a given concept, and where and when should the costlier higher-resolution (HR) imagery be acquired?

We present a novel scheme to address these challenges by introducing three components: (1) A technique to distill knowledge from models trained on HR imagery to recognition models that operate on imagery of lower resolution (LR), (2) a sampling strategy for HR imagery based on model disagreement, and (3) an LLM-based approach for inferring concept "scale". With these components we present a system to efficiently perform scale-aware recognition in satellite imagery, improving accuracy over single-scale inference while following budget constraints. Our novel approach offers up to a 26.3% improvement over entirely HR baselines, using 76.3% fewer HR images.


Research Overview

System Overview: First, we determine which resolution is best suited for the search concept based on its scale (Section 3.3). Then, we analyze the search area to find which regions would benefit the most from higher resolution inference (Section 3.5). We sample the best suited regions while staying within a user specified budget. Based on this guidance we perform inference using one of three models, a high resolution satellite model, a low resolution satellite model, and a low resolution satellite model with knowledge distilled from its high resolution counterpart (Section 3.4). This knowledge distilled model allows us to infer finer details using low resolution satellite imagery alone

Paper Resources

PDF Preview

Paper & Supplementary PDF

Full research paper including additional experiments, ablations, and detailed results

Download PDF
Arxiv

Arxiv

Arxiv

View on arXiv
Github

Github

Code and Data

View on Github

Citation

BibTeX 
@article{revankar2024scale,
    title={Scale-Aware Recognition in Satellite Images under Resource Constraint},
    author={Revankar, Shreelekha and Phoo, Cheng Perng and Mall, Utkarsh and Hariharan, Bharath and Bala, Kavita},
    journal={ICLR},
    year={2025}
}

Updates

Code Update

Github now Public! Demo and training code coming soon.

Code Update

Github to be made public soon!

Paper Update

Paper accepted to ICLR 2025

Paper Update

Updated version of the paper is now available on arXiv