Machine Learning Based Spectroscopy from RGB Images of Optical Phantoms

In the field of physics, spectroscopy refers to the study of how matter interacts with light, revealing detailed information about the composition and properties of materials.

For example, biomedical imaging uses diffuse reflectance spectroscopy to analyze how biological tissue absorbs and scatters light, which provides insight into its composition and structure.

While very useful, a major limitation arises in diffuse spectroscopy experimentation: spectroscopic systems are traditionally reliant on expensive, specialized equipment, making research relatively inaccessible. Recent advances in machine learning have enabled new methods for extracting complex information from simple image data. In particular, RGB data from a single image contains implicit spectral information that can potentially be used to infer material properties.

In this work, we investigate a cost-effective ML-based approach for reconstructing diffuse reflectance spectra from RGB images taken through a common mobile device. To accomplish this, we construct optical phantoms using milk and food coloring to simulate variations in scattering and absorption properties, enabling repeatable and consistent experimentation for data collection and model training.

Research Questions

Can diffuse reflectance spectra be accurately reconstructed from RGB images using machine learning methods?
How effective is a low-cost controlled optical phantom system in the training of a model for diffuse reflectance spectra prediction?
Does RGB image data contain sufficient spectral information to infer the optical properties of materials?

Methods

To investigate the relationship between RGB data and diffuse reflectance spectra, a controlled experimental setup was designed and constructed to produce consistent and accurate readings.

The optical phantoms used as the primary data source will be constructed from a controlled mix of food coloring and milk. This solution will provide a suitable medium for light scattering and absorption to simulate different properties for the machine learning model to learn from.

The images of the phantoms will be captured with a mobile phone, while a single fiber optic spectrometer will record the diffuse spectrum. Both devices will be fixed in an overhead position above the phantom samples. Samples will then be illuminated from underneath their container for the measurements to be taken. The information gathered from each device will be input into the ML model for calibration. The measurements from the single fiber optic serve as ground truth, enabling the model to learn the mapping between the RGB image data and its corresponding diffuse reflectance spectra. This paired dataset allows for supervised training of the model to predict spectral information directly from RGB image input.

Results/Findings and Discussion

Discussion

At the current stage of development, experimental data collection is still in progress, and no final quantitative results are available. The design of this system however allows for several important considerations regarding reconstructing spectral information from RGB image data and its feasibility.

One of the largest aspects to the approach that we are taking is the trade-off between accessibility and spectral resolution. RGB image data is limited to three-channel array information, which contains significantly less detail than measurements taken from fiber optic spectrometers. Therefore, all spectral data reconstructed from RGB input is inherently an approximation rather than a direct measurement.

Despite this limitation, the RGB values present in these images still contain information related to absorption and scattering within these samples. Exploiting this relationship to infer diffuse spectral patterns is the central goal of the machine learning model used in this study. Using the ground truth spectrometer measurements, the machine learning model's reconstructed spectra will be compared to evaluate its accuracy and ability to recover key spectral features such as shape and relative intensity.

Conclusions and Future Study

This study presents a framework for investigating the feasibility of reconstructing diffuse reflectance spectra from RGB image data using machine learning. Using optical phantoms and a controlled environment, this project establishes a reproducible method for creating a link between simple mobile device image inputs and higher-dimensional spectral information.

While data collection and machine learning development are still ongoing, the proposed methodology is designed to evaluate how effectively RGB image data can preserve and encode spectral information. Once a sufficient amount of data is collected, the machine learning model's performance will be assessed using ground truth data from the fiber optic spectrometer.

Future work will focus on expanding the range of phantom compositions as well as exploring more advanced machine learning architectures to improve spectral reconstruction accuracy. A key goal is to eventually operate outside of a heavily controlled environment, which would help determine real-world applicability of this approach.

Acknowledgements

I would like to thank my instructor, Professor Vishwanath, for providing guidance, resources, and support throughout this project.

NACE Career Readiness Competencies

Communication - Through this research, I have developed a stronger ability to communicate with other members of the department and collaborate with my instructor by discussing experimental design and project progress.
Critical Thinking - This project has helped me hone my critical thinking skills by analyzing and evaluating data, as well as understanding the limitations of reconstructing information from RGB inputs when working independently.
Technology - During my research I have worked closely with spectrometer-related software and gained experience with machine learning model architecture, which was initially unfamiliar to me.

Research Compliance Protocols

This research did not require Institutional Review Board (IRB) or Institutional Animal Care and Use Committee (IACUC) approval, as no animal or human subjects were involved. All materials used were non-hazardous and handled in a controlled environment, and standard laboratory safety protocols were followed.

Page updated

Report abuse