Embedded Files
Multiple views of the same tree obtained using Geograypher
Russell, D., Weinstein, B., Wettergreen, D., & Young, D. (2024). Classifying geospatial objects from multiview aerial imagery using semantic meshes. https://doi.org/10.48550/arXiv.2405.09544
Most of the traditional forest monitoring methods rely on single top-down views of trees to study species-level information. These approaches do not take advantage of the numerous distinct views of each tree from multiple perspectives provided by raw drone imagery. In this project during my time at the Forest Change Analysis Lab (FOCAL) and Open Forest Observatory (OFO) in UC Davis, we developed an automated workflow that integrates ground observations, canopy height models (CHMs), and 3-channel drone imagery to train a computer vision model for multi-view tree species classification.
Each plot was assigned a pair of drone missions, one from a top-down nadir view and the another from an low angled oblique view. Using Geograypher, an open-source tool developed at OFO, we were able to easily translate between geospatial mapping of trees and its corresponding location in raw drone imagery, and vice-versa. Our dataset included trees form more than 100 diverse set of plots, most of them across the Sierra Nevada, Klamath, and Cascades regions.
With the CHM data as input, we used the variable window filter algorithm of Popescu and Wynne (2004) to identify tree tops, and used seed-based watershed segmentation to obtain the tree crown polygons. The implementation of this two-stage tree detection is available as the `GeometricDetector` class in tree-detection-framework, our open-source Python package. Most of my contributions were towards training the species classification model. I selected EfficientNetV2, a CNN-based computer vision model architecture, due to its strong performance on benchmark datasets and its relatively lightweight design. After experimenting with data augmentation strategies and hyperparameter optimizations, the model achieved 85.78% macro precision across eight different species classes. As each tree had an average of 50 different views, the image-level predictions were aggregated at the tree level through a voting process, resulting in a significant improvement in accuracy.
One of the main challenges in this project was class imbalance in the dataset, which appeared at multiple levels: the number of views per tree, the number of trees per species, and species abundance across plots. To address this, I used a stratified sampling approach where all trees from the same plot were assigned to the same train, validation, or test split, preventing spatial autocorrelation and maintaining similar species proportion across all three splits.
My favorite part about this project was getting to integrate data from multiple different sources and achieve our goal of predicting tree species in such a large scale. Link to the preprint.
Page updated
Google Sites
Report abuse