Self-Supervision in time for Satellite Images(S3-TSS) A novel method of SSL technique in Satellite images

Team 18, High-level Computer Vision 2023 by Bernt Schiele
Presentation . Report . Arxiv paper

Table of Contents

1. About The Project
2. Folder structure
3. Results on other datasets
4. Team Members
5. Acknowledgments
6. Paper and Citation
7. Contact

About The Project

With the limited availability of labeled data with various atmospheric conditions in remote sensing images, it seems useful to work with self-supervised algorithms. Few pretext based algorithms including from rotation, spatial context and jigsaw puzzles are not appropriate for satellite images \cite{wang2022selfsupervised}. Often, satellite images have a higher temporal frequency. So, the temporal dimension of remote sensing data provides natural augmentation without requiring us to create artificial augmentation of images. Here, we propose S3-TSS, a novel method of self-supervised learning technique that leverages natural augmentation occurring in temporal dimension. We compare our results with current state-of-the-art methods and also perform various experiments. We observed that our method was able to perform better than baseline SeCo \cite{seco} in four downstream datasets. Code for our work can be found here

Folder structure

• All the code can be found under notebook/
• The final dataset can be found under data/processed/all_city_data_with_pop.csv

├── data
│   └── assets
├── Dockerfile
├── figures
├── README.md
├── report
│   └── HLCV_Project_team_18.pdf
└── src
    ├── modules
    │   ├── preprocess.py
    │   ├── seco_datamodule.py
    │   ├── seco_dataset.py
    │   ├── seco_dataset_temporal.py
    │   └── seco_utils.py
    ├── notebooks
    │   ├── docker
    │   │   └── exp_1
    │   │       ├── requirements.txt
    │   │       └── resnet18_dino_100k.py
    │   ├── EDA_EuroSAT.ipynb
    │   ├── SECO_EDA.ipynb
    │   ├── SSL_experiment
    │   │   ├── exp_1.ipynb
    │   │   └── exp_2_DINO_temporal.ipynb
    │   ├── SSL_seco.ipynb
    │   └── VIT-classification.ipynb
    └── pipeline
        ├── resnet18_pipeline_eurosat.ipynb
        ├── resnet18_pipeline_eurosat pretrained.ipynb
        ├── resnet18_pipeline_eurosat_SECO.ipynb
        ├── resnet50_pipeline_final.ipynb
        └── resnet50_pipeline.ipynb

Results on other datasets

Experiment 1:

• Architecture: ResNet18
• Dataset: SeCo-20k(out of 100k)
• Epochs: 30 and 100
• Downstream Datasets: Eurosat(In report), AID, UCMerced, WHU-RS19
• Metric: Fine-tuning and Linear-probe

Experiment 2:

• Architecture: ResNet18
• Dataset: SeCo-100k
• Epochs: 100
• Downstream Datasets: Eurosat(In report), AID, UCMerced, WHU-RS19
• Metric: Fine-tuning and Linear-probe

Experiment 3:

• Architecture: ResNet18
• Dataset: SeCo-100k
• Self-Supervision in Time (S3-TSS)
• Epochs: 100
• Downstream Datasets: Eurosat(In report), AID, UCMerced, WHU-RS19
• Metric: Fine-tuning and Linear-probe
• Comparison with SeCo baseline

Team Members

1. Akansh Maurya (7047939)
2. Hewan Shrestha (7047533)
3. Mohammad Munem Shahriar (7002640)

Acknowledgments

We would like to thank Prof. Dr. Bernt Schiele and tutors of HLCV course 2023 at Saarland University for giving us the opportunity to work in this project.

Contact

Email: akanshmaurya@gmail.com

Paper and Citation

To cite our work, please use the following:

@misc{maurya2024selfsupervision,
      title={Self-Supervision in Time for Satellite Images(S3-TSS): A novel method of SSL technique in Satellite images}, 
      author={Akansh Maurya and Hewan Shrestha and Mohammad Munem Shahriar},
      year={2024},
      eprint={2403.04859},
      archivePrefix={arXiv},
      primaryClass={cs.AI}
}