Steve Brumby

Professional biography

Steve Brumby is the Co-Founder and CEO/CTO of Impact Observatory, a mission-driven technology company bringing AI-powered geospatial monitoring to environment, climate, and sustainability risk analysis. In June 2021, Impact Observatory produced the world’s first fully automated, high resolution (10m) land use and land cover map using deep learning at global scale in commercial cloud, released as a digital public good. In March 2022 we produced the world’s first annual time series of 10m land use and land cover maps (2017-2021), available via our partners Esri Living Atlas and Microsoft Azure Planetary Computer. Impact Observatory's maps are available via the UN Biodiversity Lab, and have been used by the New York Times to analyze the impact of catastrophic flooding in Pakistan due to the climate crisis. Steve serves on the Department of the Interior's Landsat Advisory Group.

Prior to Impact Observatory, Steve founded and directed the Geographic Visualization Lab at National Geographic Society, was a Senior Fellow at World Resources Institute supporting the Global Forest Watch program, and was the Co-Founder and CTO of Descartes Labs, a venture-backed start-up using AI and space data to forecast global agriculture. Steve started his AI+space career at Los Alamos National Laboratory developing machine learning algorithms for image and signals datasets. His work at Los Alamos included co-authoring the GENIE machine learning system that won an R&D Magazine R&D100 award in 2002. Steve received his Ph.D. in Theoretical Particle Physics at the University of Melbourne (Australia) in 1997, has authored over 150 scientific publications, and works and lives in Washington, DC, with his family and N+1 cats.

Past projects

Genetic Algorithms for Imagery Exploitation: Co-inventor of GENIE, a machine learning genetic programming software package for rapid evolution of automatic feature extraction algorithms exploiting multi-spectral imagery. GENIE has been successfully applied to a number of real-world problems, including environmental science, land-cover mapping, change detection, and medical imagery analysis, and is currently in commercial use (remote sensing and digital pathology applications have been licensed).

Deep Sparse Generative Models for robust, real-time computer vision: I was Principal Investigator of LANL's LDRD-DR internal R&D project on hierarchical sparse representations for robust analysis of video. Our project aimed to develop a large-scale, real-time computer vision system inspired by neuroscience models of visual cortex and based on recent ideas in adaptive sparse signal processing and deep learning. This system used LANL's GPU-accelerated multi-teraflop machines and commercial cloud computing systems. We call this system VAST (Video Analysis and Search Technology), and it was licensed to Descartes Labs for commercial applications. The VAST project received over $15M in US Government R&D funding.

Contact details

email: steve@impactobservatory.com

LinkedIn: steven brumby

Twitter: @stevenpbrumby

Google Scholar: Steven P Brumby

Employment

2020–present Impact Observatory - CEO/CTO & Co-Founder

2018–2020 National Geographic Society - Senior Director for Geographic Visualization

2017–2020 World Resources Institute - Senior Fellow

2014–2018 Descartes Labs - Co-Founder & Founding CTO

2014–2014 Senior Research Scientist, Los Alamos National Laboratory (LANL), CCS-3 Information Sciences Group, Computer Computational and Statistical Sciences (CCS) Division.

2013-2014 Research Scientist, Applied Machine Learning Team, ISR-3 Space Data Systems Group, LANL.

2000-2013 Research Scientist, Applied Machine Learning Team, ISR-2 Space and Remote Sensing Sciences Group, LANL.

1998–2000 Postdoctoral Research Associate, NIS-2 Space and Remote Sensing Sciences Group, LANL.

1997–1998 Postdoctoral Research Fellow, Lobster-eye X-ray Optics, Optics Group, School of Physics, University of Melbourne.

1993–1997 Tutor (Graduate Teaching Assistant), School of Physics, University of Melbourne.

Education

1997 Ph.D., Theoretical Particle Physics, University of Melbourne (Australia).

1993 B.Sc.(Honours), Physics, University of Melbourne (Australia).

Service

2018–present DOI NGAC/Landsat Advisory Group

Awards

2011 Federal Laboratory Consortium (FLC) Award for Excellence in Technology Transfer for "Genie Pro (GENetic Imagery Exploitation)" [link]

2005 LANL Distinguished Copyright Award for Genie Pro software

2002 LANL Distinguished Performance Award (Large Team) for GENIE

2002 R&D Magazine “R&D 100 Award” for “GENIE: Evolving feature extraction algorithms for image analysis" [link]

Recent and Key Publications (over 150 publications)

  1. Brown, C.F., Brumby, S.P., Guzder-Williams, B. et al. Dynamic World, Near real-time global 10 m land use land cover mapping. Sci Data 9, 251 (2022). https://doi.org/10.1038/s41597-022-01307-4

  2. K. Karra, C. Kontgis, Z. Statman-Weil, J. C. Mazzariello, M. Mathis and S. P. Brumby, "Global land use / land cover with Sentinel 2 and deep learning," 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2021, pp. 4704-4707, doi: 10.1109/IGARSS47720.2021.9553499.

  3. S. Saatchi, M. Longo, L. Xu, et al., Detecting vulnerability of humid tropical forests to multiple stressors, One Earth 4 (7), 988-1003

  4. Riggio, J, Baillie, JEM, Brumby, S, et al. Global human influence maps reveal clear opportunities in conserving Earth’s remaining intact terrestrial ecosystems. Glob Change Biol. 2020; 26: 4344– 4356. https://doi.org/10.1111/gcb.15109

  5. Immerzeel, W.W., Lutz, A.F., Andrade, M. et al. Importance and vulnerability of the world’s water towers. Nature 577, 364–369 (2020). https://doi.org/10.1038/s41586-019-1822-y

  6. Daniela I. Moody, Steven P. Brumby, Rick Chartrand, et al., "Satellite imagery analysis for automated global food security forecasting," Proc. SPIE 10644, Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XXIV, 1064429 (8 June 2018); https://doi.org/10.1117/12.2315960

  7. Daniela I. Moody, Steven P. Brumby, Rick Chartrand, et al., "Crop classification using temporal stacks of multispectral satellite imagery," Proc. SPIE 10198, Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XXIII, 101980G (5 May 2017); https://doi.org/10.1117/12.2262804

  8. Daniela Moody, Steven P. Brumby, Michael S. Warren, et al., "Building a living atlas of the world in the cloud", in Fiftieth Asilomar Conference on Signals, Systems and Computers, Asilomar, CA, Nov 2016.

  9. Daniela Moody, Steven P. Brumby, Michael S. Warren, et al., "Building a living atlas in the cloud to analyze and monitor global patterns", in IEEE Applied Imagery Pattern Recognition Workshop, 2016, Washington DC, Oct 2016.

  10. Samuel Skillman, Michael S. Warren, Steven P. Brumby, et al, "Processing a Petabyte of Planetary Pixels with Python", in Scientific Computing with Python SciPy2016, Austin, TX, July 2016.

  11. Michael S. Warren, Steven P. Brumby, Samuel W. Skillman, Tim Kelton, Brendt Wohlberg, Mark Mathis, Rick Chartrand, Ryan Keisler, Mark Johnson, "Seeing the Earth in the Cloud: Processing One Petabyte of Satellite Imagery in One Day", in IEEE Applied Imagery Pattern Recognition Workshop, 2015, Washington DC, Oct 2015. [article in MIT Technology Review]

  12. Daniela I. Moody, Przemek R. Wozniak, Steven P. Brumby, "Automated Variability Selection in Time-domain Imaging Surveys using Sparse Representations with Learned Dictionaries", in IEEE Applied Imagery Pattern Recognition Workshop, 2015, Washington DC, Oct 2015.

  13. D. I. Moody, S. P. Brumby, J. C. Rowland, G. L. Altmann, "Land Cover Classification in Multispectral Imagery using Clustering of Sparse Approximations (CoSA) over Learned Feature Dictionaries", Journal of Applied Remote Sensing, Special Section on High-Performance Computing in Applied Remote Sensing: Part 3, 2014 (in review).

  14. D. I. Moody, D. A. Smith, S. P. Brumby, “Automatic Detection of Pulsed Radiofrequency (RF) Targets using Sparse Representations in Undercomplete Learned Dictionaries,” SPIE Automatic Target Recognition XXIV, (2014) – Lockheed Martin Best Paper Award.

  15. D. I. Moody, S. P. Brumby, J. C. Rowland, G. L. Altmann, "Land Cover Classification in Multispectral Satellite Imagery using Sparse Approximations on Learned Dictionaries," SPIE Satellite Data Compression, Communications, and Processing X, Baltimore, MD, May 2014.

  16. Zhengping Ji, James Theiler, Rick Chartrand, Steven P. Brumby, “SIFT-based sparse coding for large-scale visual recognition”, SPIE Defense, Security, and Sensing, Compressive Sampling Applications I, May 2013.

  17. Daniela I. Moody, Steven P. Brumby, Joel C. Rowland, Chandana Gangodagamage , “Undercomplete learned dictionaries for land cover classification in multispectral imagery of Arctic landscapes using CoSA: clustering of sparse approximations”, SPIE Defense, Security, and Sensing, Spectral Methodologies and Applications II, April 2013.

  18. Przemyslaw R. Wozniak, D. I. Moody, Z. Ji, S. P. Brumby, H. Brink, J. Richards, J. S. Bloom, "Automated Variability Selection in Time-domain Imaging Surveys Using Sparse Representations with Learned Dictionaries", American Astronomical Society, AAS Meeting #221, #431.05, Long Beach, CA, 06-10 Jan 2013.

  19. Garrity, Steven R.; Allen, Craig D.; Brumby, Steven P.; Gangodagamage, Chandana ; McDowell, Nate G.; Cai, D. Michael, "Quantifying tree mortality in a mixed species woodland using multitemporal high spatial resolution satellite imagery", Remote Sensing of Environment, 129: 54 - 65, 2013.

  20. Min Chen, Joel C. Rowland, Cathy J. Wilson, Garrett L. Altmann, Steven P. Brumby, "Temporal and spatial pattern of thermokarst lake area changes at Yukon Flats, Alaska", Hydrologic Processes, 21 Nov 2012, DOI: 10.1002/hyp.9642.

  21. D. I. Moody, S. P. Brumby, Joel C. Rowland, Chandana Gangodagamage, “Learning Sparse Discriminative Representations for Land Cover Classification in the Arctic,” Proc. SPIE 8514, Satellite Data Compression, Communications, and Processing VIII, 85140Q (October 19, 2012); doi:10.1117/12.930182

  22. Amy E. Galbraith, Steven P. Brumby and Rick Chartrand, "Simulating vision through time: Hierarchical, sparse models of visual cortex for motion imagery", in IEEE Applied Imagery Pattern Recognition Workshop, 2012, Washington DC, Oct 2012.

  23. Wentao Huang, Zhengping Ji, Steven P. Brumby, Garrett Kenyon and Luis M. A. Benttencourt, Development of Invariant Feature Maps via a Computational Model of Simple and Complex Cells, IEEE International Joint Conference on Neural Networks (IJCNN) 2012, June 10-15, Brisbane, Australia.

  24. Zhengping Ji, Rick Chartrand and Steven P. Brumby, Learning Sparse Representation via a Nonlinear Shrinkage Encoder and a Linear Sparse Decoder, IEEE International Joint Conference on Neural Networks (IJCNN) 2012, June 10-15, Brisbane, Australia.

  25. Daniela I. Moody, Steven P. Brumby, Joel C. Rowland, Chandana Gangodagamage, Arctic Land Cover Classification using Multispectral Imagery with Adaptive Sparse Representations, Conference on Data Analysis (CoDA) 2012, Feb 29-March 2, Santa Fe, NM.

  26. Steven P Brumby, Michael I Ham, Garrett T Kenyon Semi-supervised learning of high-level representations of natural video sequences. Computational and Systems Neuroscience (COSYNE) 2012, 23-26 Feb 2012, Salt Lake City, Utah.

  27. D. I. Moody, S. P. Brumby, K. L. Myers, N. H. Pawley, “Sparse Classification of RF (radio frequency) Transients using Chirplets and Learned Dictionaries,” Proceedings of IEEE Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, Nov 2011.

  28. Daniela I. Moody, Steven P. Brumby, Kary L. Myers, Norma H. Pawley, "Radio Frequency (RF) transient classification using sparse representations over adaptive dictionaries", Proc. SPIE 8138, 81381S (2011)

  29. Gintautas V, Ham MI, Kunsberg B, Barr S, Brumby SP, Rasmussen C, George JS, Nemenman I, Bettencourt LM, Kenyon GT, (2011) Model Cortical Association Fields Account for the Time Course and Dependence on Target Complexity of Human Contour Perception. PLoS Comput Biol 7(10): e1002162. doi:10.1371/journal.pcbi.1002162 [link]

  30. M. Ham, S. Brumby, Z. Ji, K. Sanbonmatsu, G. Kenyon, J. George, and L. Bettencourt., "Task-specific saliency from sparse, hierarchical models of visual cortex compared to eye-tracking data for object detection in natural video sequences", Vision Sciences Society (VSS) 2011 Annual Meeting , May 6-11, Naples, FL.

  31. Steven P. Brumby, "Image fusion for remote sensing using fast, large-scale neuroscience models", Proc. SPIE 8064, 806402 (2011)

  32. Daniela I. Moody, Steven P. Brumby, Kary L. Myers, Norma H. Pawley, "Classification of transient signals using sparse representations over adaptive dictionaries", Proc. SPIE 8058, 805804 (2011)

  33. Steven Brumby, Michael Ham, Will A. Landecker, Garrett Kenyon, Luis Bettencourt, "Visualizing classification of natural video sequences using sparse, hierarchical models of cortex", Computational and Systems Neuroscience (COSYNE) 2011, 24-27 Feb 2011, Salt Lake City, Utah. [Nature Precedings, npre.2011.5971.1]

  34. J.S.George, M.Ham, S.Barr, V.Gintautas, C.Rinaudo, A.Guthormsen, M.Anghel, P.Loxley, S.Brumby, L.Bettencourt, G.Kenyon, "Visual object recognition and masking in speed-of-sight tasks", Society for Neuroscience Annual Meeting 2010, 15 Nov 2010, San Diego, CA.

  35. L.M.Bettencourt, S.P.Brumby, J.George, M.I.Ham, G.Kenyon, "Receptive field properties in primate visual cortical hierarchy from large scale statistics of natural images", Society for Neuroscience Annual Meeting 2010, 15 Nov 2010, San Diego, CA.

  36. Steven P. Brumby, Amy E. Galbraith, Michael Ham, Garrett Kenyon, and John S. George, "Visual Cortex on a Chip: Large-scale, real-time functional models of mammalian visual cortex on a GPGPU", GPU Technology Conference (GTC) 2010, 20-23 Sep 2010, San Jose, CA [PDF]

  37. Steven Brumby, Luis Bettencourt, Michael Ham, Ryan Bennett, and Garrett Kenyon, "Quantifying the difficulty of object recognition tasks via scaling of accuracy versus training set size", Computational and Systems Neuroscience (COSYNE) 2010, 25-28 Feb 2010, Salt Lake City, Utah [PDF]

  38. Will Landecker, Steven Brumby, Mick Thomure, Garrett Kenyon, Luis Bettencourt, and Melanie Mitchell, "Visualizing Classification Decisions of Hierarchical Models of Cortex", Computational and Systems Neuroscience (COSYNE) 2010, 25-28 Feb 2010, Salt Lake City, Utah [PDF]

  39. Norma Pawley, Kary Myers, John Galbraith, and Steven Brumby, "Capturing Dynamics on Multiple Time Scales: A Hybrid Approach for Cluttered Electromagnetic Data", 43rd Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, Nov 2009.

  40. L. M. Bettencourt, S. Brumby, V. Gintautas, M. I. Ham, S. Barr, P. Loxley, K. Sanbonmatsu, S. Swaminarayan, J. George, G. Kenyon, I. Nemenman. "Image categorization through large-scale hierarchical models of the primate visual cortex.", Program No. 652.2/Y13. Society for Neuroscience 2009, Oct 17-21, 2009, Chicago IL.

  41. Steven P. Brumby, Garrett Kenyon, Will Landecker, Craig Rasmussen, Sriram Swaminarayan, and Luis M. A. Bettencourt, "Large-scale functional models of visual cortex for remote sensing", 2009 38th IEEE Applied Imagery Pattern Recognition, Vision: Humans, Animals, and Machines, Cosmos Club, Washington DC October 14-16, 2009 [PDF]

  42. Harvey, N.R., Brumby, S.P., Pawley, N., Ruggiero, C., Hixson, R., Balick, L.K., Oyer, A., MacDonald, B., “Detection of Facilities in Satellite Imagery using Semi-supervised Image Classification and Auxiliary Contextual Observables”. SPIE Visual Information Processing XVIII, Proc. SPIE 7341, in press, 2009.

  43. S. P. Brumby, S. W. Koch, and L. A. Hansen. "Evolutionary computation and post-wildfire land-cover mapping with multispectral imagery." Remote Sensing for Environmental Monitoring, Proc. SPIE 4545 (2002) 174-183.

  44. N. R. Harvey, J. Theiler, S. P. Brumby, S. Perkins, J. J. Szymanski, J. J. Bloch, R. B. Porter, M. Galassi, and A. C. Young. "Comparison of GENIE and Conventional Supervised Classifiers for Multispectral Image Feature Extraction."IEEE Trans. Geoscience and Remote Sensing 40 (2002) 393-404.

  45. S. P. Brumby, J. Theiler, S. J. Perkins, N. R. Harvey, J. J. Szymanski, J. J. Bloch, and M. Mitchell. "Investigation of image feature extraction by a genetic algorithm." Proc. SPIE 3812 (1999) 24-31. [PDF]