ML Surveys and Tools

Papers

1. Data-centric Artificial Intelligence: A Survey

2. A Survey on Transfer Learning

3. Machine Learning Operations (MLOps): Overview, Definition, and Architecture

4. A Survey on Federated Learning

5. A survey on semi-supervised learning

6. A Survey on Multi-Task Learning

7. Active Learning: A Survey

8. A Comprehensive Survey of Continual Learning: Theory, Method and Application

9. A reading survey on adversarial machine learning: Adversarial attacks and their understanding

10. Machine Unlearning: A Survey

11. Recent advances in decision trees: an updated survey

12. Taking the Human out of Learning Applications: A Survey on Automated Machine Learning

13. A Survey of Human-in-the-loop for Machine Learning

14. A Survey on Time-Series Pre-Trained Models

15. Adversarial Machine Learning A Taxonomy and Terminology of Attacks and Mitigations 

16. A Survey of Small Language Models

17. Questionable practices in machine learning 

18. A Comprehensive Survey on Transfer Learning

19. A Survey of Data Augmentation Approaches for NLP

20. A Survey of Transformers 

21. Word Embeddings: A Survey

22. A comprehensive survey of clustering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and future research prospects

23. A Comprehensive Survey on Data Augmentation

24. On the Foundations of Shortcut Learning

25. A Survey on Bias and Fairness in Machine Learning

26. A Comprehensive Survey of Continual Learning: Theory, Method and Application

27. "Forgetting" in Machine Learning and Beyond: A Survey 

Books

1. Machine Learning Engineering with Python

2. Designing Machine Learning Systems

Resources

1. Made with ML

ML Libraries, Tools and Platforms

Deep Learning Frameworks

• PyTorch: A flexible and dynamic framework for building custom AI models; widely used in research and production.

• TensorFlow: A comprehensive platform supporting the entire machine learning workflow, from prototyping to deployment.

• Keras: A high-level API running on top of TensorFlow, ideal for fast and easy neural network prototyping.

• MXNet: A deep learning framework that supports a flexible programming model for different deep learning tasks.

• Caffe: A deep learning library focused on speed and modularity, often used in computer vision.

Machine Learning Libraries

• Scikit-learn: A go-to library for data mining, predictive analysis, and classic machine learning algorithms.

• XGBoost: A powerful gradient boosting framework designed for structured/tabular data tasks.

• LightGBM: A gradient boosting framework optimized for speed and efficiency with large datasets.

• CatBoost: A gradient boosting library specifically designed to handle categorical features effectively.

• H2O.ai: An open-source platform for scalable machine learning, supporting distributed training.

Graph Neural Networks (GNNs)

• PyTorch Geometric (PyG): A popular library built on PyTorch for creating and training GNNs.

• Deep Graph Library (DGL): A framework designed for building and training GNNs, compatible with PyTorch, TensorFlow, and MXNet.

• GraphNets: A library from DeepMind for building GNNs with TensorFlow and Sonnet.

• Spektral: A Python library for GNNs built on TensorFlow and Keras.

• StellarGraph: A machine learning library for graph-based data using GNNs, built on TensorFlow.

• Karate Club: A Python library for unsupervised learning on graph-structured data.

• Neo4j Graph Data Science: A graph database platform that includes machine learning capabilities.

• PecanPy: A scalable Python library for embedding large-scale graphs.

Natural Language Processing (NLP) Tools

• Transformers: Pre-trained models for NLP tasks such as text generation, translation, and sentiment analysis.

• NLTK: A robust toolkit for text processing, including tokenization, tagging, and parsing.

• LangChain: A framework for building applications powered by large language models, integrating external resources like databases

• OpenAI: Offers cutting-edge AI models, including GPT, for language understanding and generation.

• LlamaIndex: Facilitates connecting language models with structured and unstructured external data sources.

• SpaCy: An industrial-strength NLP library with support for tokenization, named entity recognition, and dependency parsing.

• TextBlob: Simplifies NLP tasks like text classification, translation, and sentiment analysis.

• Gensim: A library for topic modeling and document similarity using techniques like word2vec and LDA.

Data Handling and Manipulation

• NumPy: Core library for numerical operations and efficient array handling in Python.

• Pandas: Essential for data wrangling, manipulation, and analysis of structured data.

• Dask: A parallel computing library for scaling NumPy, pandas, and scikit-learn workflows.

• Vaex: A library for fast, memory-efficient data manipulation of large datasets.

Visualization Tools

• Matplotlib: A foundational library for creating detailed and customizable visualizations.

• Seaborn: Enhances Matplotlib by simplifying statistical data visualization with elegant default styles.

• Plotly: An interactive visualization library for creating rich, web-based visualizations.

• Bokeh: A library for interactive visualizations, focusing on web applications.

Scientific Computing

• SciPy: A collection of advanced numerical routines for optimization, integration, and signal processing.

• SymPy: A symbolic mathematics library useful for algebraic computations and calculus.

• Numba: A JIT compiler that accelerates numerical computations with Python.

Computer Vision

• OpenCV: The industry-standard library for computer vision tasks like object detection, image processing, and video analysis.

• PyTorch Lightning: Simplifies the creation of deep learning models, particularly in computer vision.

• Fastai: A library for building computer vision and NLP models with PyTorch.

• Albumentations: A fast and flexible library for augmenting image data in computer vision tasks.

Reinforcement Learning

• Stable-Baselines3: A collection of pre-implemented reinforcement learning algorithms for research and development.

• Ray RLlib: A scalable library for reinforcement learning applications.

Automated Machine Learning (AutoML)

• Auto-sklearn: Automated machine learning built on scikit-learn.

• TPOT: A genetic programming-based framework for AutoML.

• DataRobot: A platform for automating the machine learning pipeline.

MLOps Platforms

• MLflow: A comprehensive platform for managing ML experiments, model deployment, and lifecycle tracking.

• Kubeflow: A Kubernetes-native platform for deploying and managing machine learning workflows at scale.

• Metaflow: A tool developed by Netflix for managing real-world machine learning workflows.

• Weights & Biases (W&B): Focused on experiment tracking, hyperparameter optimization, and collaboration.

• DVC (Data Version Control): A version control system tailored for machine learning projects.

Model Deployment and Serving

• TensorFlow Serving: A flexible system for serving TensorFlow models in production.

• TorchServe: A PyTorch-native tool for deploying and serving PyTorch models.

• Seldon Core: An open-source platform for deploying machine learning models on Kubernetes.

• FastAPI: A modern web framework for building APIs, commonly used for deploying ML models.

Pipeline Orchestration and Data Engineering

• Apache Airflow: A workflow orchestration tool for managing ML pipelines.

• Prefect: A modern dataflow automation platform that simplifies pipeline orchestration.

• Dagster: A data orchestrator for machine learning, analytics, and ETL pipelines.

Monitoring and Observability

• Evidently AI: A library for monitoring ML model performance and detecting data drift.

• WhyLabs AI Observatory: A platform for monitoring ML model health and data quality.

Automated Testing and CI/CD

• GitHub Actions: A popular tool for implementing CI/CD pipelines for ML projects.

• Jenkins: A flexible automation server for building and deploying ML models.

• Tekton: A Kubernetes-native framework for building CI/CD pipelines.

Cloud-Native MLOps Solutions

• AWS SageMaker: A cloud platform for building, training, and deploying ML models.

• Azure ML: Microsoft's MLOps platform for managing ML workflows.

• Google Vertex AI: Google's platform for end-to-end ML model lifecycle management.

Experimentation and Optimization

• Optuna: A framework for hyperparameter optimization.

• Hyperopt: A Python library for hyperparameter tuning.

Specialized AI Applications

• gorgias: A framework for conversational AI and chatbot development.

• PyCaret: A low-code library for automating machine learning workflows.

• Orange Data Mining: A visual programming tool for data analysis and machine learning.

Large-Scale Graph Processing

• NetworkX: A comprehensive library for creating, analyzing, and manipulating complex networks.

• GraphBLAS: A library for graph algorithms using linear algebra operations.

• Snap.py: A Python library for large-scale network analysis and graph processing.

Use Cases and Recommendations

• Deep Learning Projects: PyTorch, TensorFlow, and Transformers.

• Classic Machine Learning: Scikit-learn and XGBoost.

• NLP: OpenAI, LangChain, Transformers, NLTK.

• Data Preparation: NumPy, Pandas.

• Visualization: Matplotlib, Seaborn.

• Scientific Research: SciPy, Theano.

• Computer Vision: OpenCV.

Machine Unlearning

• • • Machine Unlearning in 2024

    • Machine Unlearning: A Comprehensive Survey

    • A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models

    • Digital Forgetting in Large Language Models: A Survey of Unlearning Methods

    • Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening\

    • Unlearning via Sparse Representations

    • Fast Yet Effective Machine Unlearning

    • Threats, Attacks, and Defenses in Machine Unlearning: A Survey

    • Researchers Develop New Technique to Wipe Dangerous Knowledge From AI Systems

    • Model Sparsity Can Simplify Machine Unlearning

    • Towards Source-Free Machine Unlearning

    • Selective Forgetting: Advancing Machine Unlearning Techniques and Evaluation in Language Models

    • Machine Unlearning

    • Towards Making Systems Forget with Machine Unlearning 

    • NeurIPS 2023 Machine Unlearning Challenge 

    • In-Context Unlearning: Language Models as Few Shot Unlearners

    • Many-shot jailbreaking

    • Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems

    • Guardrail Baselines for Unlearning in LLMs

    • Large Language Models Relearn Removed Concepts

    • Who's Harry Potter? Approximate Unlearning in LLMs

    • Large Language Model Unlearning

    • Model Editing with Canonical Examples

    • Erasing Concepts from Diffusion Models 

    • Locating and Editing Factual Associations in GPT 

    • MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions

    • Memory-Based Model Editing at Scale

    • Towards Unbounded Machine Unlearning

    • Knowledge Unlearning for Mitigating Privacy Risks in Language Models

    • KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment

    • Towards Adversarial Evaluations for Inexact Machine Unlearning

    • The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning

    • Who’s Harry Potter? Making LLMs forget

    • Unlearn What You Want to Forget: Efficient Unlearning for LLMs