research

documentation of research

Our initial research involved attempting to replicate known inventories of fens in the immediate vicinity of Telluride, Colorado using the area of San Miguel County as a test bed. The methodology involved using standard feature extraction procedures available in ArcGIS Pro (v. 3.4.0)

Fig. 1. The image above shows the boundary of San Miguel County, a region some 100 miles long in SW Colorado.  The western border abuts the State of Utah.  Per the analysis of Geospatial analyst, Kirk McPhail, most of the fens are distributed in the east end of the county at an elevation between 2700 - 3600 meters (8860 - 11,800 feet).

Fig. 2.  The image shows a detail of Prospect Basin in San Miguel County.  The area circled in red is Buckbean Fen.

Kirk:   Let's insert screenshots of your work characterizing the aspect, elevation, flow, and slope .

Bradley:  I like your idea of generating a "heat map" showing slopes and ground water.

20250516

RE:  Recent progress with the fen detection project by Aditya Deshpande

Aditya successfully processed the San Juan Mountains DEM data and generated several terrain analysis visualizations (see Fig. 3 below). The images show different characteristics of the landscape:

1) Normalized DEM - Shows elevation patterns where brighter spots are higher areas
2) Slope - Highlights steeper terrain (areas that appear as small bright spots)
3) Aspect - Indicates the direction terrain faces (useful for understanding water flow)
4) Terrain Ruggedness Index - Shows areas with rough, variable terrain
5) Topographic Position Index - Helps identify ridges and valleys
6) Elevation Bands - Groups areas by similar elevation ranges

The blue spots scattered across the images likely indicate potential fen locations, as they appear in places with distinct terrain patterns typical of wetland areas.

Regarding challenges, the original approach using the Segment Anything Model (SAM) proved too computationally intensive for Aditya’s computer. The DEM file is extremely large (31,457 × 38,765 pixels), causing memory overflows and kernel crashes. He had to simplify by downsampling (reducing the sample size) of the data and using traditional terrain analysis instead.

Aditya will be meeting soon with Dr. Gil Speyer, one of our team members and head of the supercomputing research cluster at ASU. With greater computing resources, Aditya feels he could run the full SAM model (see link to SAM research here) on the high-resolution data, which would likely provide more precise fen identification. This work is being supported by Kirk McPhail in ASU’s Map and Geospatial Hub.

Ground truthing.  In our meeting of April 24, 2025 we discussed the importance of ground truthing the analysis derived from high level satellite data and GIS maps with the drone data generated by Evan and Bradley. 

Question for Kirk:  Would it be possible to overly your work in characterizing the "slope" and other parameters derived from satellite data with  the mosaic below to determine degrees of confidence and/or change over time?

Fig. 3.  Buckbean Fen classification mosaic (courtesy Iverson and Sowder).

Fig. 4: 3D comparison--"difference map"-- of the Morgan life mask (taken upon George Washington's death in 1799)  and the Jean-Antoine Houdon bust (1786) scaled by 9% to accommodate shrinking of the terracotta material during firing.  Laser scan of the original 1786 sculpture is shown in blue.  Deviation is shown in increasingly warmer colors.

Question:  What data visualization tools would be most effective in showing "change over time" relative to high altitude wetlands?

Here are a couple of examples that show very different approaches to depicting "change over time."   The first is a  map overlay created in 1944 by the US Army Corp of Engineers to show the change in the position of the Mississippi River between 1765 - 1944.  The second shows a sequence of images to show issues related to deforestation and land cover.  Both images rae taken from the following article:  https://www.geographyrealm.com/time-and-gis/  

20250602

Aditya now has access to supercomputer cluster at ASU.  Dan and Aditya had phone call in which Aditya updated Dan on workplan for week.  The goal is to successfully submit his script to the supercomputer to see if it can identify Fens in San Miguel County.  He willl report on progress at June 5 meeting.

20250814

Notes on Polarization

This sounds like it would have real potential for the fens mapping project…

Polarized geospatial data: unlocking deeper insights

Polarized geospatial data, captured using techniques like polarimetric synthetic aperture radar (PolSAR) and polarization lidar (P-lidar), goes beyond conventional intensity measurements by analyzing the polarization state of electromagnetic waves reflected or scattered from objects on or near the Earth's surface. This additional information, often hidden in traditional remote sensing, reveals crucial details about the physical properties and scattering mechanisms of targets, leading to enhanced understanding of various environments. 

Key aspects and technologies

• Polarimetric Synthetic Aperture Radar (PolSAR):

  • Actively transmits and receives radio waves in multiple polarization directions (e.g., horizontal (H) and vertical (V)).

  • Analyzes the polarization state of backscattered echoes, which can change upon interaction with different surface features.

  • Provides information about surface structure, geometry, material properties, and scattering mechanisms.

  • Common polarization modes include:

• • 1. Dual-polarization (Dual-pol): Transmits in one polarization and receives in both (e.g., HH/HV or VV/VH).

• • 2. Quad-polarization (Quad-pol): Alternates between transmitting H and V, receiving both, producing HH, HV, VH, and VV imagery.

• • Different scattering effects (surface, volume, double bounce, helix) are captured based on polarization, enabling discrimination of terrain types, vegetation, urban features, and more.

• Polarization Lidar (P-lidar):

  • Emits and receives polarized laser pulses, extending detection dimensions beyond intensity and spectrum.

  • Utilizes polarization degree, angle, and ellipticity to gain deeper insights into target objects.

  • Applications span atmospheric (aerosols, clouds, ice crystals), oceanic (phytoplankton layers, fish schools, seawater optical properties), and terrestrial (vegetation, urban structures) remote sensing.

  • Various P-lidar configurations exist, including dual-channel, single-channel, three/four-channel, and full P-lidar, each offering different levels of polarization information.

• Polarized Hyperspectral Imaging (PHSI):

  • Combines the capabilities of polarization measurement, spectral analysis, and spatial imaging.

  • Provides polarization, spectral, and morphological information simultaneously.

  • Has shown potential in biomedical applications, such as enhancing visualization of collagen fibers and differentiating tumor cells. 

Applications and benefits

• Land Cover Classification: PolSAR's ability to differentiate scattering behaviors enhances terrain classification, vegetation mapping, and urban area monitoring.

• Environmental Monitoring: P-lidar assists in tracking deforestation, mapping soil moisture, and monitoring atmospheric pollution through aerosol and cloud characterization.

• Precision Agriculture: Polarized lidar can detect water stress and disease in plants, enabling targeted interventions and improved resource management.

• Hazard Assessment: PolSAR contributes to flood mapping and oil spill detection.

• Infrastructure Planning: Geospatial data, including polarized information, informs decisions related to resource allocation and land management.

• Autonomous Driving: Integrating polarization into lidar systems can improve autonomous vehicle performance, particularly in low-visibility conditions. 

Challenges and advancements

While offering immense potential, working with polarized geospatial data presents challenges:

• Data Size and Processing: Large datasets demand high-performance computing resources and efficient processing methods.

• Standardization and Integration: Lack of consistent standards and siloed systems hinder data interoperability and analysis across different platforms and geographic locations.

• Data Quality and Accuracy: Ensuring the accuracy and reliability of polarized data requires careful acquisition, processing, and validation. 

However, advancements in deep learning and machine learning are helping overcome these hurdles:

• Models trained on polarized data can process large datasets, merge information from different sources, and extract relevant details rapidly.

• Clustering algorithms and convolutional neural networks can reduce noise and improve data quality, leading to more accurate analyses.

• Deep learning models are increasingly used for classifying PolSAR images and extracting semantic information from complex terrain types. 

In essence, polarized geospatial data provides a rich source of information that, when combined with advanced analytical techniques, unlocks a new level of understanding about our planet and its complex systems. 

20251031

The following three images show mapping of fens and related wetlands shared by  Michigan Technological University (Jeremy Graham et al).  The area depicted details the northern San Juan Mountains of SW Colorado.  

Fig. 1.  MTU data (shown in context of Southwestern Colorado).  The black outline is San Miguel County.

Fig. 2.  MTU data.  More detailed image of northern reaches of San Juan Mountains (grey area).

Fig. 3.  MTU Data.   Image detailing the fens in Prospect Basin, just south of Telluride, Colorado.

20251113

Aditya reported on the following:

This is a summary of the work Aditya completed today on the San Juan fen detection project, continuing from the workflows established in his initial report.

His main goal was to replicate the successful ArcGIS Pro workflows on the new, larger dataset.

1. Troubleshooting: He initially encountered a PermissionError when trying to run the Spatial Analyst tools. He identified this as a software permissions issue and resolved it by running ArcGIS Pro as an administrator.

2. Workflow A (Unsupervised Classification):

   • He ran the Iso Cluster Unsupervised Classification tool on the main 1m DEM (5 classes). The tool completed in 5 minutes, matching the remarkable efficiency noted in my report.

   • He then used the Reclassify tool to convert the 5-class output into a final binary (1/0) Fen_Binary_Unsupervised.tif map.

3. Workflow B (Supervised ML - Feature Engineering):

   • The rest of the day was dedicated to generating the complete set of predictor rasters required for the supervised machine learning model.

   • He successfully ran the Slope, Aspect, and Curvature (including Profile and Plan) tools.

   • He then completed the full hydrological analysis by running Fill, Flow Direction, and Flow Accumulation.

4. Current Status & Next Steps:

   • He is currently at the final stage of data preparation: running the Composite Bands tool to stack all 8 predictor rasters (DEM, Slope, Aspect, Curvatures, Flow Direction, and Flow Accumulation) into a single file.

   • He encountered a WARNING 160105 because some of the rasters had slightly different dimensions. He is now resolving this by re-running the tool with the Environment Settings (Processing Extent, Cell Size, and Snap Raster) set to match the original DEM.

He will continue working on this tomorrow. His first step will be to ensure the Composite Bands tool completes successfully. After that, He will move on to the main machine learning phase: training the supervised classifier (e.g., Random Forests) using this new predictor stack and the filtered_polygons.gpkg file as my training data.

Once he generates the new classification, he will compare his results against the previous model outputs he has (the SanJuan_050125_classification.tif and SanJuan_050125_probabilities.tif files) to evaluate the performance of the new model.

20251114

Aditya made significant progress with the supervised machine learning workflow today.  

His first step was to resolve the WARNING 160105 and successfully run the Composite Bands tool.

This led to a challenging but very productive day of deep troubleshooting. He has now identified and solved all technical issues, and in the process, uncovered a fundamental issue with our training data that we must address.

Here is a summary of the roadblocks and solutions:

1. Composite Bands Tool Failure: His first step was to align all 8 predictor rasters using the Extract by Mask tool. This was a long (multi-hour) but successful process. However, even with these perfectly aligned rasters, the Composite Bands tool failed again with a generic ERROR 999999. This confirms that the tool is not robust enough to stack this many massive rasters (over 100GB of data).

2. Bypassing Composite Bands: To bypass this, he moved to a more advanced workflow: feeding all 8 rasters directly into the Train Random Trees Classifier tool.

3. UI & Permissions Failure: This new approach revealed two new problems:

   • UI Bug: The tool's user interface was bugged and would not allow him to add all 8 rasters to the input list.

   • Permissions Error: To bypass the UI, he moved to the ArcGIS Pro Python window.

4. Python API Failures: In the Python window, I encountered a long series of TypeError and AttributeError messages. This was due to an extremely confusing and poorly documented conflict between two different tools with the same name (arcpy.ia vs. arcpy.sa), which use different, rigid Python syntax.

The Solution and the Real Diagnosis

To solve all of these issues at once, he executed a comprehensive Python script. This script correctly imports the arcpy.ia module, sets all environment paths, and uses the correct positional arguments to call the TrainRandomTreesClassifier tool.

This script worked perfectly—it ran all its pre-checks and successfully diagnosed the true underlying problem, which is not with the software, but with our data.

As you will see in the console log screenshot I'll attach, the script analyzed the filtered_polygons.gpkg file and found that it only contains one class of data. All 60,770 polygons are positive "fen" samples (Class 1). The script correctly aborted the training, as a supervised model cannot learn without "negative" samples (Class 0) for comparison. The final error was: ERROR: Aborting: cannot train on single-class labels.

Next Steps

This is an excellent outcome, as we now have a perfectly clear path forward:

1. His next task is to manually create these "negative" (Class 0) training samples.

2. He will edit the filtered_polygons.gpkg file and digitize 50-100 new polygons over areas that are clearly not fens (e.g., steep rocky slopes, dense forests, open water).

3. The script has already created a class_id field. He will assign these new polygons a class_id of 0.

Once he has a true two-class training set, he will re-run the same Python script. It will pass all checks and will finally, and successfully, train the Random Forests model.