Despite international efforts to reduce deforestation, the world loses, so far, an area of forest that is equivalent to the size of 40 football fields every minute. Deforestation in the Amazon rainforest accounts for the largest share, contributing to reduced biodiversity, habitat loss, and climate change. Since much of the region is difficult to access, satellite remote sensing offers a powerful tool to track changes in the Amazon. However, obtaining a continuous time series of images is hindered by seasonal weather, clouds, smoke, and other inherent limitations of optical sensors. Synthetic aperture radar (SAR), which is insensitive to lighting and weather conditions, appears to be a well-suited tool for the task, but SAR images are more difficult for humans to interpret than optical images.

The 2022 Multimodal Learning for Earth and Environment Workshop (MultiEarth 2022) aims to gather a wide audience of researchers in academia, industry, and related fields to leverage remote sensing images collected by multiple sensors for positive environmental impact. The workshop comprises two tracks: the challenge track and the paper track. For the challenge track, we conduct the following three sub-challenges to support the interpretation and analysis of the rainforest at any time and under any weather conditions: 1) matrix completion, 2) deforestation estimation and 3) multimodal image-to-image translation sub-challenges. The paper track includes a broader call for papers that use machine learning to address problems in earth and environmental monitoring.

6/20 - Multi Earth 2022 Workshop @ CVPR

Paper Track:

5/23 11:59PM PST - Paper submission deadline

6/06 - Acceptance decisions

6/13 - Camera ready paper submission

Challenge Track:

4/02 - Challenge training data released.

5/30 6/06 - Evaluation server open for the test set, with leaderboard available.

6/03 6/10 - Evaluation server closes

6/08 6/12 11:59PM PST - Paper submission deadline

6/20 - Challenge winners announced

We are soliciting papers that use machine learning to address problems in earth and environmental science and monitoring, including but not limited to the following topics:

• Agriculture

• Climate modeling

• Disaster prediction, management, and relief

• Ecosystems and natural systems

• Carbon capture and sequestration

• Land use applications

• Land cover dynamics

• Forest and vegetation: application and modeling

• Forest and vegetation: biomass and carbon cycle

• Topography, geology, and geomorphology

• Soils and soil moisture

• Wetlands

• Inland waters

• Multimodal remote sensing analysis

• Change detection and multi-temporal analysis

• Geographic information science

• Multimodal generative modeling

• Multimodal representation learning

• We accept submissions of max 8 pages (excluding references) on the aforementioned and related topics. We encourage authors to submit 4-page works.

• Submitted manuscripts should follow the CVPR 2022 paper template.

• Accepted papers are not archival and will not be included in the proceedings of CVPR 2022

• Submissions will be rejected without review if they:

  • • Contain more than 8 pages (excluding references)

    • Violate the double-blind policy or violate the dual-submission policy

• Paper submission must contain substantial original contents not submitted to any other conference, workshop, or journal

• Papers will be peer-reviewed under a double-blind policy and need to be submitted online through the CMT submission website.

A key component of this event will be to monitor the Amazon rainforest in all weather and lighting conditions using our multimodal remote sensing dataset, which includes a time series of multispectral and synthetic aperture radar (SAR) images. We propose to conduct the following challenges to support the interpretation and analysis of the rainforest at any time and any weather conditions:

• Image-to-Image Translation: Given a SAR image, predict a set of possible cloud-free corresponding electro-optical (EO) images

• Matrix Completion: Given images taken at different locations and times, and in different modalities, predict appearance at a novel (time, location, modality) query

• Downstream Task: Environment change estimation (e.g. deforestation, fire, water coverage) from the cloud-free view of the Amazon

Additional Data:

• Challenge 1 - Matrix Completion

  • challenge1_test_region_data.zip

• Challenge 2 - Deforestation Estimation

  • challenge2_test_region_data.zip

Test Queries:

• Challenge 1 - Matrix Completion

  • challenge1_test_queries.csv

• Challenge 2 - Deforestation Estimation

  • challenge2_test_queries.csv

• Challenge 3 - Image-to-Image Translation

  • challenge3_sent1_vh_test_queries.txt

  • challenge3_sent1_vv_test_queries.txt

  • challenge3_sent1_vh.zip

  • challenge3_sent1_vv.zip

Submission sites:
*Please review the exact submission format in FAQ

• Challenge 1 - Matrix Completion

  • http://visiongpu44.csail.mit.edu:3000/matrix-completion/

• Challenge 2 - Deforestation Estimation

  • http://visiongpu44.csail.mit.edu:3000/estimation/

• Challenge 3 - Image-to-Image Translation

  • http://visiongpu44.csail.mit.edu:3000/translation/

• Miriam Cha, Technical staff, MIT Lincoln Laboratory

• Sam Goldberg, Postdoc researcher, EAPS, MIT

• Morgan Schmidt, Research affiliate, EAPS, MIT

• Armando Cabrera, Flight chief, DAF MIT AI Accelerator

• Taylor Perron, Professor, EAPS, MIT

• Phillip Isola, Associate professor, EECS, MIT

• Bill Freeman, Professor, EECS, MIT

• Gregory Angelides, Technical staff, MIT Lincoln Laboratory

• Kuan Wei Huang, Masters Student, EECS, MIT

• Mark Hamilton, PhD Student, EECS, MIT

• Yen-Chen Lin, PhD Student, EECS, MIT

• Brandon Swenson, Technical Sergeant, DAF MIT AI Accelerator

• Jean Piou, Technical Staff, MIT Lincoln Laboratory

@misc{https://doi.org/10.48550/arxiv.2204.07649,

doi = {10.48550/ARXIV.2204.07649},

url = {https://arxiv.org/abs/2204.07649},

author = {Cha, Miriam and Huang, Kuan Wei and Schmidt, Morgan and Angelides, Gregory and Hamilton, Mark and Goldberg, Sam and Cabrera, Armando and Isola, Phillip and Perron, Taylor and Freeman, Bill and Lin, Yen-Chen and Swenson, Brandon and Piou, Jean},

keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},

title = {MultiEarth 2022 -- Multimodal Learning for Earth and Environment Workshop and Challenge},

publisher = {arXiv},

year = {2022},

copyright = {Creative Commons Attribution 4.0 International}

}