I am Co-Founder and Chief Science Advisor of Descartes Labs [ descarteslabs.com | @DescartesLabs ], a venture-backed start-up based in Los Alamos NM and San Francisco CA.
Our mission is to better understand planet Earth by analyzing massive amounts of overhead imagery. We’ll begin this journey by understanding satellite imagery to enable real-time global awareness — whether it’s food production, the growth of cities, or our impact on the environment. Not only will we identify what’s in a single image, but we’ll also be looking at how our Earth changes over time. This deep understanding of our planet will allow us to be better stewards of this world and give people worldwide the knowledge to make intelligent decisions about the future.
As of January 2017, I am also a Senior Fellow at World Resources Institute (wri.org). WRI is a global research organization that spans more than 50 countries, with offices in the United States, China, India, Brazil, Indonesia and more. WRI's work focuses on critical issues at the intersection of environment and development: climate, energy, food, forests, water, and cities and transport. I'm working with WRI to bring large-scale machine learning and remote sensing science to some of these intensely challenging topics.
Prior to Descartes Labs, I was an remote sensing scientist and information scientist at Los Alamos National Laboratory, applying machine learning to satellite imagery and other scientific image, video and signal datasets. Prior to that I was a Theoretical Physicist at the University of Melbourne, working on the mathematical foundations of quantum mechanics.
This is my personal information website.
Professional Biography:Dr. Steven P. Brumby is Co-Founder and Chief Science Advisor of Descartes Labs, a venture-backed start-up based in Los Alamos NM and San Francisco CA focused on understanding agriculture, natural resources and human geography using deep learning technology and satellite imagery. Previous to Descartes Labs, Steven was a Senior Research Scientist at Los Alamos National Laboratory (LANL) Information Sciences Group (CCS-3) working on development of sparse-coding and deep-learning algorithms for video, image and signals analysis. He received his Ph.D. in Theoretical Physics at the University of Melbourne (Australia) in 1997. He is a co-inventor of LANL’s award-winning GENIE image analysis algorithm (R&D100 Award 2002).
Past projects:Genetic Algorithms for Imagery Exploitation: Co-inventor of GENIE, a machine learning genetic programming
software package for rapid evolution of automatic feature extraction
algorithms exploiting multi-spectral imagery. GENIE has been
successfully applied to a number of real-world problems, including environmental science, land-cover mapping, change detection, and medical imagery analysis, and is
currently in commercial use (remote sensing and digital pathology applications have been licensed).
Deep Sparse Generative Models for robust, real-time computer vision: I was Principal Investigator of LANL's LDRD-DR internal R&D project on hierarchical sparse representations for robust analysis of video. Our project aimed to develop a large-scale, real-time computer vision system inspired by neuroscience models of visual cortex and based on recent ideas in adaptive sparse signal processing and deep learning. This system used LANL's GPU-accelerated multi-teraflop machines and commercial cloud computing systems. We call this system VAST (Video Analysis and Search Technology), and it was licensed to Descartes Labs for commercial applications. The VAST project received over $15M in US Government R&D funding.
2017–present World Resources Institute - Senior Fellow
2014–present Descartes Labs - Co-Founder & CTO/Chief Science Advisor2014–2014 Senior Research Scientist, Los Alamos National Laboratory (LANL),
CCS-3 Information Sciences Group, Computer Computational and Statistical Sciences (CCS) Division.
2013-2014 Research Scientist, Applied Machine Learning Team, ISR-3 Space Data Systems Group, LANL.
2000-2013 Research Scientist, Applied Machine Learning Team, ISR-2 Space and Remote Sensing Sciences Group, LANL.
1998–2000 Postdoctoral Research Associate, NIS-2 Space and Remote Sensing Sciences Group, LANL.
1997–1998 Postdoctoral Research Fellow, Lobster-eye X-ray Optics, Optics Group, School of Physics, University of Melbourne.
1993–1997 Tutor (Graduate Teaching Assistant), School of Physics, University of Melbourne.
Education:1997 Ph.D., Theoretical Particle Physics, University of Melbourne (Australia).
1993 B.Sc.(Honours), Physics, University of Melbourne (Australia).
2011 Federal Laboratory Consortium (FLC) Award for Excellence in Technology Transfer for "Genie Pro (GENetic Imagery Exploitation)" [link]2005 LANL Distinguished Copyright Award for Genie Pro software
2002 LANL Distinguished Performance Award (Large Team) for GENIE
2002 R&D Magazine “R&D 100 Award” for “GENIE: Evolving feature extraction algorithms for image analysis" [link]
Recent and Key Publications (over 100 publications):
- Daniela Moody, Steven P. Brumby, Michael S. Warren, et al., "Building a living atlas of the world in the cloud", in Fiftieth Asilomar Conference on Signals, Systems and Computers, Asilomar, CA, Nov 2016.
- Daniela Moody, Steven P. Brumby, Michael S. Warren, et al., "Building a living atlas in the cloud to analyze and monitor global patterns", in IEEE Applied Imagery Pattern Recognition Workshop, 2016, Washington DC, Oct 2016.
- Samuel Skillman, Michael S. Warren, Steven P. Brumby, et al, "Processing a Petabyte of Planetary Pixels with Python", in Scientific Computing with Python SciPy2016, Austin, TX, July 2016.
- Michael S. Warren, Steven P. Brumby, Samuel W. Skillman, Tim Kelton, Brendt Wohlberg, Mark Mathis, Rick Chartrand, Ryan Keisler, Mark Johnson, "Seeing the Earth in the Cloud: Processing One Petabyte of Satellite Imagery in One Day", in IEEE Applied Imagery Pattern Recognition Workshop, 2015, Washington DC, Oct 2015. [article in MIT Technology Review]
- Daniela I. Moody, Przemek R. Wozniak, Steven P. Brumby, "Automated Variability Selection in Time-domain Imaging Surveys using Sparse Representations with Learned Dictionaries", in IEEE Applied Imagery Pattern Recognition Workshop, 2015, Washington DC, Oct 2015.
- D. I. Moody, S. P. Brumby, J. C. Rowland, G. L. Altmann, "Land Cover Classification in Multispectral Imagery using Clustering of Sparse Approximations (CoSA) over Learned Feature Dictionaries", Journal of Applied Remote Sensing, Special Section on High-Performance Computing in Applied Remote Sensing: Part 3, 2014 (in review).
- D. I. Moody, D. A. Smith, S. P. Brumby, “Automatic Detection of Pulsed Radiofrequency (RF) Targets using Sparse Representations in Undercomplete Learned Dictionaries,” SPIE Automatic Target Recognition XXIV, (2014) – Lockheed Martin Best Paper Award.
- D. I. Moody, S. P. Brumby, J. C. Rowland, G. L. Altmann, "Land Cover Classification in Multispectral Satellite Imagery using Sparse Approximations on Learned Dictionaries," SPIE Satellite Data Compression, Communications, and Processing X, Baltimore, MD, May 2014.
- Zhengping Ji, James Theiler, Rick Chartrand, Steven P. Brumby, “SIFT-based sparse coding for large-scale visual recognition”, SPIE Defense, Security, and Sensing, Compressive Sampling Applications I, May 2013.
- Daniela I. Moody, Steven P. Brumby, Joel C. Rowland, Chandana Gangodagamage , “Undercomplete learned dictionaries for land cover classification in multispectral imagery of Arctic landscapes using CoSA: clustering of sparse approximations”, SPIE Defense, Security, and Sensing, Spectral Methodologies and Applications II, April 2013.
- Przemyslaw R. Wozniak, D. I. Moody, Z. Ji, S. P. Brumby, H. Brink, J. Richards, J. S. Bloom, "Automated Variability Selection in Time-domain Imaging Surveys Using Sparse Representations with Learned Dictionaries", American Astronomical Society, AAS Meeting #221, #431.05, Long Beach, CA, 06-10 Jan 2013.
- Garrity, Steven R.; Allen, Craig D.; Brumby, Steven P.; Gangodagamage, Chandana ; McDowell, Nate G.; Cai, D. Michael, "Quantifying tree mortality in a mixed species woodland using multitemporal high spatial resolution satellite imagery", Remote Sensing of Environment, 129: 54 - 65, 2013.
- Min Chen, Joel C. Rowland, Cathy J. Wilson, Garrett L. Altmann, Steven P. Brumby, "Temporal and spatial pattern of thermokarst lake area changes at Yukon Flats, Alaska", Hydrologic Processes, 21 Nov 2012, DOI: 10.1002/hyp.9642.
- D. I. Moody, S. P. Brumby, Joel C. Rowland, Chandana Gangodagamage, “Learning Sparse Discriminative Representations for Land Cover Classification in the Arctic,” Proc. SPIE 8514, Satellite Data Compression, Communications, and Processing VIII, 85140Q (October 19, 2012); doi:10.1117/12.930182
- Amy E. Galbraith, Steven P. Brumby and Rick Chartrand, "Simulating vision through time: Hierarchical, sparse models of visual cortex for motion imagery", in IEEE Applied Imagery Pattern Recognition Workshop, 2012, Washington DC, Oct 2012.
- Wentao Huang, Zhengping Ji, Steven P. Brumby, Garrett Kenyon and Luis M. A. Benttencourt, Development of Invariant Feature Maps via a Computational Model of Simple and Complex Cells, IEEE International Joint Conference on Neural Networks (IJCNN) 2012, June 10-15, Brisbane, Australia.
- Zhengping Ji, Rick Chartrand and Steven P. Brumby, Learning Sparse Representation via a Nonlinear Shrinkage Encoder and a Linear Sparse Decoder, IEEE International Joint Conference on Neural Networks (IJCNN) 2012, June 10-15, Brisbane, Australia.
- Daniela I. Moody, Steven P. Brumby, Joel C. Rowland, Chandana Gangodagamage, Arctic Land Cover Classification using Multispectral Imagery with Adaptive Sparse Representations, Conference on Data Analysis (CoDA) 2012, Feb 29-March 2, Santa Fe, NM.
- Steven P Brumby, Michael I Ham, Garrett T Kenyon Semi-supervised learning of high-level representations of natural video sequences. Computational and Systems Neuroscience (COSYNE) 2012, 23-26 Feb 2012, Salt Lake City, Utah.
- D. I. Moody, S. P. Brumby, K. L. Myers, N. H. Pawley, “Sparse Classification of RF (radio frequency) Transients using Chirplets and Learned Dictionaries,” Proceedings of IEEE Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, Nov 2011.
- Daniela I. Moody, Steven P. Brumby, Kary L. Myers, Norma H. Pawley, "Radio Frequency (RF) transient classification using sparse representations over adaptive dictionaries", Proc. SPIE 8138, 81381S (2011)
- Gintautas V, Ham MI, Kunsberg B, Barr S, Brumby SP, Rasmussen C, George JS, Nemenman I, Bettencourt LM, Kenyon GT, (2011) Model Cortical Association Fields Account for the Time Course and Dependence on Target Complexity of Human Contour Perception. PLoS Comput Biol 7(10): e1002162. doi:10.1371/journal.pcbi.1002162 [link]
- M. Ham, S. Brumby, Z. Ji, K. Sanbonmatsu, G. Kenyon, J. George, and L. Bettencourt., "Task-specific saliency from sparse, hierarchical models of visual cortex compared to eye-tracking data for object detection in natural video sequences", Vision Sciences Society (VSS) 2011 Annual Meeting , May 6-11, Naples, FL.
- Steven P. Brumby, "Image fusion for remote sensing using fast, large-scale neuroscience models", Proc. SPIE 8064, 806402 (2011)
- Daniela I. Moody, Steven P. Brumby, Kary L. Myers, Norma H. Pawley, "Classification of transient signals using sparse representations over adaptive dictionaries", Proc. SPIE 8058, 805804 (2011)
- Steven Brumby, Michael Ham, Will A. Landecker, Garrett Kenyon, Luis Bettencourt, "Visualizing classification of natural video sequences using sparse, hierarchical models of cortex", Computational and Systems Neuroscience (COSYNE) 2011, 24-27 Feb 2011, Salt Lake City, Utah. [Nature Precedings, npre.2011.5971.1]
- J.S.George, M.Ham, S.Barr, V.Gintautas, C.Rinaudo, A.Guthormsen, M.Anghel, P.Loxley, S.Brumby, L.Bettencourt, G.Kenyon, "Visual object recognition and masking in speed-of-sight tasks", Society for Neuroscience Annual Meeting 2010, 15 Nov 2010, San Diego, CA.
- L.M.Bettencourt, S.P.Brumby, J.George, M.I.Ham, G.Kenyon, "Receptive field properties in primate visual cortical hierarchy from large scale statistics of natural images", Society for Neuroscience Annual Meeting 2010, 15 Nov 2010, San Diego, CA.
- Steven P. Brumby, Amy E. Galbraith, Michael Ham, Garrett Kenyon, and John S. George, "Visual Cortex on a Chip: Large-scale, real-time functional models of mammalian visual cortex on a GPGPU", GPU Technology Conference (GTC) 2010, 20-23 Sep 2010, San Jose, CA [PDF]
- Gintautas, V., Kunsberg, B., Ham, M. I., Barr, S., Zucker, S., Brumby, S., Bettencourt, L. M. A., and Kenyon, G. T., Perceptual organization: Contours and 2D form An improved model for contour completion in V1 using learned feature correlation statistics, Journal of Vision, 2010, vol. 10(7), p. 1162; doi:10.1167/10.7.1162 [link]
- Vadas Gintautas, Benjamin Kunsberg, Michael Ham, Shawn Barr, Steven Zucker, Steven Brumby, Luis M A Bettencourt, Garrett T Kenyon, "An improved model for contour completion in V1 using learned feature correlation statistics", Visual Sciences Society (VSS) 2010 Annual Meeting, 7-12 May 2010, Naples, Florida
- Garrett Kenyon, Shawn Barr, Michael Ham, Vadas Gintautas, Cristina Rinaudo, Ilya Nemenman, Marian Anghel, Steven Brumby, John George, Luis Bettencourt, "Top-down models explain key aspects of a Speed-of-Sight character recognition task", Visual Sciences Society (VSS) 2010 Annual Meeting, 7-12 May 2010, Naples, Florida
- Kevin Sanbonmatsu, Ryan Bennett, Shawn Barr, Cristina Renaudo, Michael Ham, Vadas Gintautas, Steven Brumby, John George, Garrett Kenyon, Luis Bettencourt, "Comparing Speed-of-Sight studies using rendered vs.natural images", Visual Sciences Society (VSS) 2010 Annual Meeting, 7-12 May 2010, Naples, Florida
- Steven P. Brumby, Kary L. Myers, and Norma H. Pawley, "Capturing Dynamics on Multiple Time Scales: A Multilevel Fusion Approach for Cluttered Electromagnetic Data", SPIE Defense, Security and Sensing, April 5-9 2010, Orlando, FL.
- Steven Brumby, Luis Bettencourt, Michael Ham, Ryan Bennett, and Garrett Kenyon, "Quantifying the difficulty of object recognition tasks via scaling of accuracy versus training set size", Computational and Systems Neuroscience (COSYNE) 2010, 25-28 Feb 2010, Salt Lake City, Utah [PDF]
- Will Landecker, Steven Brumby, Mick Thomure, Garrett Kenyon, Luis Bettencourt, and Melanie Mitchell, "Visualizing Classification Decisions of Hierarchical Models of Cortex", Computational and Systems Neuroscience (COSYNE) 2010, 25-28 Feb 2010, Salt Lake City, Utah [PDF]
- Norma Pawley, Kary Myers, John Galbraith, and Steven Brumby,
"Capturing Dynamics on Multiple Time Scales: A Hybrid Approach for
Cluttered Electromagnetic Data", 43rd Asilomar Conference on Signals,
Systems, and Computers, Pacific Grove, CA, Nov 2009.
- L. M. Bettencourt, S. Brumby, V. Gintautas, M. I. Ham, S. Barr,
P. Loxley, K. Sanbonmatsu, S. Swaminarayan, J. George, G. Kenyon, I.
Nemenman. "Image categorization through large-scale hierarchical models
of the primate visual cortex.", Program No. 652.2/Y13. Society for
Neuroscience 2009, Oct 17-21, 2009, Chicago IL.
- Steven P. Brumby, Garrett Kenyon, Will Landecker, Craig Rasmussen, Sriram Swaminarayan, and Luis M. A. Bettencourt, "Large-scale functional models of visual cortex for remote sensing", 2009 38th IEEE Applied Imagery Pattern Recognition, Vision: Humans, Animals, and Machines, Cosmos Club, Washington DC October 14-16, 2009 [PDF]
- Rowland, J. C., Wilson, C. J., Brumby, S. P. & Pope, P. River
Mobility in a Permafrost Dominated Floodplain, European Geosciences
Union Annual Meeting, 2009.
- Harvey, N.R., Brumby, S.P., Pawley, N., Ruggiero, C., Hixson, R.,
Balick, L.K., Oyer, A., MacDonald, B., “Detection of Facilities in
Satellite Imagery using Semi-supervised Image Classification and
Auxiliary Contextual Observables”. SPIE Visual Information Processing
XVIII, Proc. SPIE 7341, in press, 2009.
- S. P. Brumby, S. W. Koch, and L. A. Hansen. "Evolutionary computation and post-wildfire land-cover mapping with multispectral imagery." Remote Sensing for Environmental Monitoring, Proc. SPIE 4545 (2002) 174-183.
- N. R. Harvey, J. Theiler, S. P. Brumby, S. Perkins, J. J. Szymanski, J. J. Bloch, R. B. Porter, M. Galassi, and A. C. Young. "Comparison of GENIE and Conventional Supervised Classifiers for Multispectral Image Feature Extraction."IEEE Trans. Geoscience and Remote Sensing 40 (2002) 393-404.
- S. P. Brumby, J. Theiler, S. J. Perkins, N. R. Harvey, J. J. Szymanski, J. J. Bloch, and M. Mitchell. "Investigation of image feature extraction by a genetic algorithm." Proc. SPIE 3812 (1999) 24-31. [PDF]