Resources

General Purpose Software & Tools

• http://python.org

• scikit-learn Python machine learning software http://scikit-learn.org/

• IDL machine learning resources https://www.harrisgeospatial.com/docs/Machine_Learning.html

• Dan Foreman Mackey has an extensive array of github resources at https://github.com/dfm including Markov Chain Monte Carlo toolkit, Fast Gaussian Processes for regression, Scalable 1D Gaussian Processes, etc.

• The "Guide to Available Mathematical Software" (GAMS): http://www.netlib.org/bib/gams.html

• Trinket: online python simulator, can be useful for devices and platforms that don't easily support python: https://trinket.io/

• Gaussian Process Regression package written in C++ and Python: Developed and maintained by Sivaram Ambikasaran, Dan-Foreman Mackey et al.; Fast Direct Methods for Gaussian Processes, Ambikasaran et al. 2014, http://adsabs.harvard.edu/abs/2014arXiv1403.6015A

• Kur: High-level deep learning tool that supports packages like Theano and TensorFlow https://kur.deepgram.com/index.html

• Kaggle: Open data analysis initiative https://www.kaggle.com/datasets

• Open AI: builds free software for training, benchmarking, and experimenting with AI. https://openai.com/

Heliophysics-Specific Software & Tools

• heliopython.org: A website to promote and facilitate the use and development of Python for Heliophysics, including an example gallery and great references

• sunpy.org: SunPy python-based solar software (see the SunPy Example Gallery for detailed examples & code)

• https://pythonhosted.org/SpacePy/: SpacePy python-based software for space sciences

• SPEDAS (Space Physics Environment Data Analysis Software) multi-mission analysis software http://spedas.org/

• PySAT: https://pysat.readthedocs.io/en/latest/ Python Satellite Data Analysis Toolkit (pysat) simplifies the process of using new instruments, reduces data management overhead, and enables the creation of instrument independent analysis routines.

• SpacePy: https://spacepy.github.io/ Python package for data analysis, modeling and visualization in space sciences.

• Peter Young's extremely useful tools and tips for solar data analysis: http://pyoung.org/

• Lynn Wilson's Wind mission software library: https://github.com/lynnbwilsoniii/wind_3dp_pros

• Kamodo: python-driven interface for space weather analysis, visualization, and knowledge discovery https://ccmc.gsfc.nasa.gov/Kamodo/

Machine & Deep Learning Education Resources:

Books

• Jake VanderPlas's Python Data Science "book" (written in python/Jupyter) , including scikit.learn tutorial and tools: https://github.com/jakevdp/PythonDataScienceHandbook

• The classic book "Artificial Intelligence: A Modern Approach" by Russell & Norvig has homeworks in multiple languages and great examples: http://aima.cs.berkeley.edu/

• Online textbook "Deep Learning" from MIT Press: http://www.deeplearningbook.org/

• Online textbook "Neural Networks and Deep Learning" http://neuralnetworksanddeeplearning.com/

• Reinforcement Learning: An Introduction

• helioml: An online machine learning textbook with specific examples and code for problems in Heliophysics (see the tutorial here)

Online Courses & Guides

• Stanford Machine Learning online course: https://www.coursera.org/learn/machine-learning Difficulty: ** Time commitment: 11 weeks, ~60 hours

• Deeplearning.ai: an online five-course "specialization" in various aspects of neural nets and deep learning. "Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization" is highly recommended.

• Udacity.com: A state-of-the-art data science learning environment that offers comprehensive "nanodegrees" in many fields

• Lynda/LinkedIn Learning: Thousands of mini-courses targetting skill development in data science

• "Machine Learning in Python step by step" by Jason Brownlee is a very gentle introduction: https://machinelearningmastery.com/machine-learning-in-python-step-by-step/

• Python for Data Science Bootcamp: https://www.udemy.com/python-for-data-science-and-machine-learning-bootcamp/

• Google Course on Deep Learning (TensorFlow Deep Learning Platform) https://research.googleblog.com/2016/01/teach-yourself-deep-learning-with.html

• Stanford course on Computer Vision and CNNs: http://cs231n.stanford.edu/2017/syllabus.html

• NVIDIA Deep Learning Resources (both online and in-person workshops): https://www.nvidia.com/en-us/deep-learning-ai/education/

• MIT Open Course on Machine Learning: https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-867-machine-learning-fall-2006/

• Basic machine learning course: https://www.kadenze.com/courses/machine-learning-for-musicians-and-artists/info

• Machine learning crash course with TensorFlow APIs: https://developers.google.com/machine-learning/crash-course/

Blogs, articles and references

• Kirk Borne, top internet digital influencer, provides regular updates, articles and advice on data science. Follow him at https://twitter.com/KirkDBorne or read his posts at https://kdnuggets.com

• https://towardsdatascience.com/ has many articles and examples written by experts in the field.

• "Hackernoon" regularly has great posts on machine learning topics: https://hackernoon.com/choosing-the-right-machine-learning-algorithm-68126944ce1f

• Datacamp online learning resources, also some good articles: https://www.datacamp.com/

Specific Applications in Machine Learning

RNNs

• Chistopher Olah's "Understanding LTSM RNNs" http://colah.github.io/posts/2015-08-Understanding-LSTMs/

• "Getting started with the Keras Sequential model" https://keras.io/getting-started/sequential-model-guide/

• Jason Brownlee's Excellent tutorial "Time Series Forecasting with the Long Short-Term Memory Network in Python" https://machinelearningmastery.com/time-series-forecasting-long-short-term-memory-network-python/

• Another good Brownlee tutorial: https://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-networks-python-keras/

• RNNs for n00bs: https://hackernoon.com/rnn-or-recurrent-neural-network-for-noobs-a9afbb00e860

Regression and PCA

• Gaussian Process Regression: Free and comprehensive book describing the theory of Gaussian Processes and applications to science and machine learning. Very often cited as the basis for many astrophysics and exoplanet applications of GP regression. Rasmussen & Williams 2006

Field-specific toolkits:

• EarthML: Earth Sciences Machine Learning package

Other Recommended Reading:

Statistics

• "Modern Statistical Methods for Astronomy" by Eric D. Feigelson and G. Jogesh Babu, Cambridge University Press, 2012

• "The Use and Misuse of Statistics in Space Physics," Patricia H. Reiff, J. Geomag. Geoelectr., 42, 1145-1174, 1990

• "An Introduction to Statistical Learning" by James et al., 2013 - Free online textbook

• "Elements of Statistical Learning" by Hastie et al., 2009

Data Representation & Representation Theory

• "The Visual Display of Quantitative Information" & book series by Edwin Tufte

• "Vision" by David Marr https://mitpress.mit.edu/books/vision

Bayesian Statistics & Networks

• https://towardsdatascience.com/introduction-to-bayesian-networks-81031eeed94e

Data Science General

• "50 Years of Data Science" by David Donoho https://www.tandfonline.com/doi/abs/10.1080/10618600.2017.1384734

• "Ushering in a New Frontier in Geospace Through Data Science," 2017 McGranaghan et al., https://agupubs.onlinelibrary.wiley.com/doi/full/10.1002/2017JA024835

Important Papers and Results in Heliophysics

• "Machine Learning Techniques for Space Weather" by Camporeale, Johnson & Wing, a great reference book describing many different principles, applications and results. Link

• Monica Bobra, "Predicting Solar Flares Using Machine Learning," https://zenodo.org/record/1241183#.Wy1JPBJJGHr

Seminars, Lectures, Discussions

(see the "Calendar" link for future seminars)

Misc.

• A very useful compilation of verification/validation and error analysis methods: http://www.cawcr.gov.au/projects/verification/

• "Data-Driven Documents" allows authors to design documents that can be manipulated based on data and interactive input: https://d3js.org/

• Learning TRIZ: Problem-solving strategies https://www.triz.co.uk/where/webinars

• TVMA (Toolkit for Multivariate Data Analysis) is a CERN-based machine learning environment that is a good example of tools developed specifically for scientific topics: https://root.cern.ch/tmva

• "This Week in Machine Learning" From Particle Physics to Audio AI with Scott Stephenson https://twimlai.com/from-particle-physics-to-audio-ai-with-scott-stephenson/

• BigML's list of public data sources for ML: https://blog.bigml.com/list-of-public-data-sources-fit-for-machine-learning/

• Arthur Shapiro's blog about visual and perception surprises: http://www.illusionsciences.com/

• Hone your skills with the challenges compiled by Peter Norvig! https://github.com/norvig/pytudes are "Python programs to practice or demonstrate skills." A great way to sharpen your programming skills quickly.

Fun Items

• "AI Fails": http://lewisandquark.tumblr.com/

• AI Paint New Colors: https://arstechnica.com/information-technology/2017/05/an-ai-invented-a-bunch-of-new-paint-colors-that-are-hilariously-wrong/

• The French Revolution happened by committee: https://www.csmonitor.com/Technology/2018/0516/When-the-humanities-meet-big-data

Getting Acquainted/Introductory Materials

• "Teachable machine" introduction to Machine Learning with no coding required! https://teachablemachine.withgoogle.com/

• Free, 101-level AI course: https://www.elementsofai.com/

• TensorFlow neural net playground: http://playground.tensorflow.org/

• Khan Academy Computer Science courses: https://www.khanacademy.org/computing/computer-science

• "Machine learning for artists" guides: http://ml4a.github.io/guides/