Reading List on Large Language Models (LLM) and Generative AI

Masoud Makrehchi

In the previous post, I discussed my perspective on staying relevant in the era of Language Models (LLMs). Now, I’d like to share my reading list on LLMs. Please note that I haven’t read all of these papers yet, and the grouping may be inaccurate, with some overlap between groups. Nevertheless, I plan to gradually post my summaries on Medium. If you have read any of the following papers and have already summarized them, I would greatly appreciate it if you could send me the links to add to this list. Please be aware that the papers are not ordered based on any specific criterion.

How to read and review papers (Especially in AI, ML and NLP domains)

1.  How to Read a Paper

2. How to review a paper

3.  How to Read Research Papers: A Pragmatic Approach for ML Practitioners

4.  How to read Machine Learning and Deep Learning Research papers

5. Best Practices for Using AI When Writing Scientific Manuscripts

6. Approaching literature review for academic purposes: The Literature Review Checklist

Natural Language Processing (NLP)

1. Speech and Language Processing [1]

2. Foundations of Statistical Natural Language Processing

3. Neural Network Methods in NLP

4. Practical Natural Language Processing 

5. A Brief Timeline of NLP

6. Neural unsupervised domain adaptation in NLP — a survey

7. The State of the Art of Natural Language Processing—A Systematic Automated Review of NLP Literature Using NLP Techniques 

8. The Oxford Handbook of Computational Linguistics (2nd edn) 

Embedding and Language Modeling

1. Distributed Representations of Words and Phrases and their Compositionality

2. Efficient Estimation of Word Representations in Vector Space

3. A Neural Probabilistic Language Model

4. Beyond word embeddings: A survey

5. A Survey on Word Meta-Embedding Learning

6. Efficient Estimation of Word Representations in VectorSpace [1]. [2]. [3]. [4]. [5] [6]

7. The Illustrated Word2vec

8. Word2Vec Explained [1]

9. Introduction to autoencoders

10. A Survey On Neural Word Embeddings

11. Foundations of Vector Retrieval

12. Is Cosine-Similarity of Embeddings Really About Similarity?

Attention Mechanism and Transformers

1. The Annotated Transformer

2. Attention Is All You Need [1]

3. Some Intuition on Attention and the Transformer

4. Understanding the Attention Mechanism in Sequence Models

5. Transformers from Scratch [1] [2]

6. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

7. The Illustrated Transformer

8. Formal Algorithms for Transformers

9. LongNet: Scaling Transformers to 1,000,000,000 Tokens

10. AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing

11. A Survey of Transformers

Generative AI and Language Models

1. The Debate Over Understanding in AI’s Large Language Models

2. A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT

3. Choose Your Weapon: Survival Strategies for Depressed AI Academics

4. Improving Language Understanding by Generative Pre-Training [code]

5. Universal Intelligence: A Definition of Machine Intelligence

6. A Glimpse in ChatGPT Capabilities and its impact for AI research

7. Moving Beyond Islands of Experimentation to AI Everywhere

8. Public Perception of Generative AI on Twitter: An Empirical Study Based on Occupation and Usage

9. Capturing Humans’ Mental Models of AI: An Item Response Theory Approach

10. Neurosymbolic AI and its Taxonomy: a survey

11. Generative Agents: Interactive Simulacra of Human Behavior

12. Just Calm Down About GPT-4 Already

13. Transformative AGI by 2043 is <1% likely

14. What Will Working with AI Really Require?

15. Evaluating the Social Impact of Generative AI Systems in Systems and Society

16. Can Transformers Learn to Solve Problems Recursively?

17. Mindstorms in Natural Language-Based Societies of Mind

18. World Models

19. Deep learning, reinforcement learning, and world models

20. Learning to Predict Without Looking Ahead: World Models Without Forward Prediction

21. Towards Understanding Mixture of Experts in Deep Learning

22. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

23. An Introduction to Semantic Routing

24. The Seven Deadly Sins of Predicting the Future of AI

25. Moving Beyond Islands of Experimentation to AI Everywhere

26. Can AI invent?

27. Generative Agents: Interactive Simulacra of Human Behavior

28. What if Generative AI turned out to be a Dud? 

29. Consciousness in Artificial Intelligence: Insights from the Science of Consciousness

30. Cognitive Architectures for Language Agents

31. Inseq: An Interpretability Toolkit for Sequence Generation Models [code]

32. Planning for AGI and Beyond

33. The Hard Argument Against LLMs Being AGI

34. Generative Representational Instruction Tuning 

35. Self-Rewarding Language Models

36. Can AI Assistants Know What They Don't Know?

37. Artificial Intelligence Index Report 2024 

38. TimeGPT-1 (GPT for time series data)

Introduction to LLMs

1. Active Self-Supervised Learning: A Few Low-Cost Relationships Are All You Need

2. Natural Language Processing with Deep Learning

3. N-grams Language Models

4. A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Models

5. Sequence to Sequence Learning with Neural Networks [1]

6. TinyStories: How Small Can Language Models Be and Still Speak Coherent English?

7. A Survey on Evaluation of Large Language Models

8. Knowledge Distillation: A Survey

9. Learning by Distilling Context

10. A Survey of Large Language Models 

11. Large Language Models: A Survey

12. Self-Rewarding Language Models

Fundamentals of LLMs and Foundation Models

1. On the Opportunities and Risks of Foundation Models

2. Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

3. Talking About Large Language Models

4. Augmented Language Models: a Survey

5. Systematic Rectification of Language Models via Dead-end Analysis

6. Self-Refine: Iterative Refinement with Self-Feedback

7. A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?

8. Sparks of Artificial General Intelligence: Early experiments with GPT-4 (*)

9. GPT is becoming a Turing machine: Here are some ways to program it

10. Emergent Abilities of Large Language Models (*)

11. Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models (*)

12. HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace

13. GPT-4 Technical Report

14. A Comprehensive Overview of Large Language Models 

15. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?

16. Learning to Program with Natural Language

17. Language Model Cascades

18. Neural Networks are Decision Trees

19. ChatGPT is not all you need. A State of the Art Review of large Generative AI models

20. So what if ChatGPT wrote it?” _Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy

21. Pre-Trained Language Models and Their Applications

22. What’s the Meaning of Superhuman Performance in Today’s NLU?

23. Neural networks learn to magnify areas near decision boundaries

24. LIMA: Less Is More for Alignment

25. Gorilla: Large Language Model Connected with Massive APIs

26. Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions

27. Assistance with large language models

28. LaMDA: Language Models for Dialog Applications

29. What Is ChatGPT Doing … and Why Does It Work?

30. Retentive Network: A Successor to Transformer for Large Language Models

31. PaLM 2 Technical Report

32. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance

33. Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes

34. Emergent autonomous scientific research capabilities of large language models

35. Challenges and Applications of Large Language Models (*)

36. Llama 2: Open Foundation and Fine-Tuned Chat Models

37. Several categories of Large Language Models (LLMs): A Short Survey (*)

38. On the Origin of LLMs: An Evolutionary Tree and Graph for 15,821 Large Language Models

39. Patterns for Building LLM-based Systems & Products (*)

40. A jargon-free explanation of how AI large language models work

41. Open challenges in LLM research (**)

42. Are distributional representations ready for the real world? Evaluating word vectors for grounded perceptual meaning

43. The Dawn of LMMs: Preliminary Explorations with GPT-4V(vision)

44. Introducing Gemini: our largest and most capable AI model

45. Gemini: A Family of Highly Capable Multimodal Models (*)

46. Phi-2: The surprising power of small language models [1] 

47. Starling-7B: Increasing LLM Helpfulness & Harmlessness with RLAIF (*)

What are LLMs?

1. Language Models are Few-Shot Learners (*)

2. Large Language Models are not Models of Natural Language: they are Corpus Models (*)

3. Language Models can Solve Computer Tasks → RCI (Recursive Criticism and Improvement) (*)

4. Scaling Instruction-Finetuned Language Models

5. How Can We Know What Language Models Know?”

6. GPT Understands, too.

7. Language Models as Knowledge Bases?

8. Self-Supervised Contextual Data Augmentation for Natural Language Processing

9. Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference

10. Few-Shot Text Generation with Natural Language Instructions

11. When to use multi-task learning vs intermediate fine-tuning for pre-trained encoder transfer learning

12. What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization

13. Don’t stop pretraining: Adapt language models to domains and tasks

14. Parameter-Efficient Fine-Tuning Design Spaces

15. Improving language understanding by generative pre-training

16. Large Language Models are Built-in Autoregressive Search Engines

17. Large Language Models are Zero-Shot Rankers for Recommender Systems

18. Davinci the Dualist: the mind-body divide in large language models and in human learners

19. Large Language Models Humanize Technology

20. The False Promise of Imitating Proprietary LLMs

21. Large Language Models as Tool Makers

22. Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners. [Code] 

23. Can Transformers Learn to Solve Problems Recursively?

24. Large Language Models Encode Clinical Knowledge

25. Language Models as Knowledge Embeddings

26. Evolution through Large Models

27. Direct Preference Optimization: Your Language Model is Secretly a Reward Model

28. Language models are weak learners

29. KokoMind: Can LLMs Understand Social Interactions?

30. Could a Large Language Model be Conscious?

31. Understanding Large Language Models: The Physics of (Chat)GPT and BERT

32. Large Language Models as Optimizers

33. Theory of Mind May Have Spontaneously Emerged in Large Language Models

34. Toolformer: Language Models Can Teach Themselves to Use Tools

35. Large Language Models can Learn Rules

36. Task Contamination: Language Models May Not Be Few-Shot Anymore

37. Direct Preference Optimization: Your Language Model is Secretly a Reward Model

In-context learning

1. The Learnability of In-Context Learning

2. Larger language models do in-context learning differently

3. Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers

4. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work (S)

5. Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

6. What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning

7. Iterative Forward Tuning Boosts In-context Learning in Language Models

8. Learning to Retrieve In-Context Examples for Large Language Models (S)

9. In-context Autoencoder for Context Compression in a Large Language Model

10. What learning algorithm is in-context learning? Investigations with linear models

11. What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning

12. In-Context Learning with Many Demonstration Examples

13. Exploring Demonstration Ensembling for In-context Learning

14. LLM In-Context Recall is Prompt Dependent

Pre-Training, Fine-Tuning and Instruction-Tuning

1. Fine-Tuning Language Models with Just Forward Passes

2. LoRA: Low-Rank Adaptation of Large Language Models

3. How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources

4. On the Exploitability of Instruction Tuning (S)

5. Platypus: Quick, Cheap, and Powerful Refinement of LLMs

6. OctoPack: Instruction Tuning Code Large Language Models (S)

7. Finetuned language models are zero-shot learners.

8. DoRA: Weight-Decomposed Low-Rank Adaptation

9. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

10. Simple and Scalable Strategies to Continually Pre-train Large Language Models 

11. TeaMs-RL: Teaching LLMs to Teach Themselves Better Instructions via Reinforcement Learning 

12. Evolutionary Optimization of Model Merging Recipes

13. Make Your LLM Fully Utilize the Context

Prompt Engineering

1. Large Language Models Are Human-Level Prompt Engineers (S)

2. The Power of Scale for Parameter-Efficient Prompt Tuning

3. Learning to Compress Prompts with Gist Tokens

4. Prompting Is Programming: A Query Language For Large Language Models (S)

5. Reframing Instructional Prompts to GPTk’s Language (S)

6. Prefix-Tuning: Optimizing Continuous Prompts for Generation (S)

7. P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks 

8.  GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models

9. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts (S)

10. Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting (S)

11. Privacy-Preserving Prompt Tuning for Large Language Model Services (S)

12. Satisfiability-Aided Language Models Using Declarative Prompting 

13. Large Language Model Guided Tree-of-Thought (S)

14. Chain-of-Symbol Prompting Elicits Planning in Large Langauge Models (*) (S)

15. Chain-of-thought prompting for responding to in-depth dialogue questions with LLM (S)

16. Prompting with Pseudo-Code Instructions

17. HELMA: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models

18. TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks

19. Measuring and Narrowing the Compositionality Gap in Language Models

20. Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm (S)

21. MemPrompt: Memory-assisted Prompt Editing with User Feedback (S)

22. PromptChainer: Chaining Large Language Model Prompts through Visual Programming (S)

23. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances

24. A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT (S)

25. The Power of Scale for Parameter-Efficient Prompt Tuning (S)

26. Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts (S) 

27. PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting

28. AutoHint: Automatic Prompt Optimization with Hint Generation

29. Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models

30. Skeleton-of-Thought: Large Language Models Can Do Parallel Decoding

31. Graph of Thoughts: Solving Elaborate Problems with Large Language Models

32. Prompt Learning for News Recommendation (*) (S)

33. Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (*) (S)

34. Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation

35. Prompting open-source and commercial language models for grammatical error correction of English learner text

36. State Value Generation with Prompt Learning and Self-Training for Low-Resource Dialogue State Tracking

37. The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions

Evaluating LLMs

1. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments

2. SemEval-2017 Task 1: Semantic Textual Similarity — Multilingual and Cross-lingual Focused Evaluation

3. Bertscore: Evaluating text generation with Bert

4. Best practices for the human evaluation of automatically generated text

5. A survey of evaluation metrics used for NLG systems

6. Beyond accuracy: Behavioral testing of NLP models with CheckList

7. Bleu: a method for automatic evaluation of machine translation

8. Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalization

9. Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definition

10. Gemv2: Multilingual NLG benchmarking in a single line of code

11. Automatically constructing a corpus of sentential paraphrases

12. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

13. BARTScore: Evaluating Generated Text as Text Generation

14. Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction

15. Automatic Evaluation of Attribution by Large Language Models

16. Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study

17. Bridging History with AI A Comparative Evaluation of GPT 3.5, GPT4, and GoogleBARD in Predictive Accuracy and Fact Checking

18. Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation

19. ANALOGICAL — A New Benchmark for Analogy of Long Text for Large Language Models

20. TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models

21. Evaluating Large Language Models Trained on Code

22. Model evaluation for extreme risks

23. Large Language Models, scientific knowledge and factuality: A systematic analysis in antibiotic discovery

24. A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (*)

25. On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)

26. Measuring Attribution in Natural Language Generation Models (*)

27. DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models (*)

28. Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs (*)

29. Bring Your Own Data! Self-Supervised Evaluation for Large Language Models (*)

30. Exploring the Robustness of Large Language Models for Solving Programming Problems (*)

31. Rethink reporting of evaluation results in AI

32. Style Over Substance: Evaluation Biases for Large Language Models

33. Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment

34. A Survey on Evaluation of Large Language Models (*)

35. Style Over Substance: Evaluation Biases for Large Language Models

36. Entailment as Robust Self-Learner

37. How is ChatGPT’s behavior changing over time?

38. FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets

39. Model Card and Evaluations for Claude Models

40. Zero-shot NLG evaluation through Pairware Comparisons with LLMs

41. AgentBench: Evaluating LLMs as Agents

42. The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation

43. ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate

44. Evaluating Large Language Models on Controlled Generation Tasks

45. Benchmarking LLMs via Uncertainty Quantification

46. An Empirical Evaluation of LLMs for Solving Offensive Security Challenges

47. Evalverse: Unified and Accessible Library for Large Language Model Evaluation

48. Evaluating LLMs at Detecting Errors in LLM Responses

49. A User-Centric Benchmark for Evaluating Large Language Models

Reinforcement Learning with Human Feedback

1. Secrets of RLHF in Large Language Models Part I: PPO

2. Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision

3. Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback

4. RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment

5. Training language models to follow instructions with human feedback [*]

6. Fine-Grained Human Feedback Gives Better Rewards for Language Model Training

7. Reinforcement Learning with Human Feedback 

8. RLHF: Reinforcement Learning from Human Feedback 

9. Self-Refine: Iterative Refinement with Self-Feedback

10. Nash Learning from Human Feedback

11. Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs

Limitations and Risks of LLMs and the Mitigation Strategies

1. Exploring the Limits of Language Modeling

2. OpenAGI: When LLM Meets Domain Experts

3. Man vs the machine: The Struggle for Effective Text Anonymization in the Age of Large Language Models

4. To ChatGPT, or not to ChatGPT: That is the question!

5. GPT‑3: Its Nature, Scope, Limits, and Consequences

6. Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models

7. Learnings from Data Integration for Augmented Language Models

8. The political ideology of conversational AI: Converging evidence on ChatGPT’s pro-environmental, left-libertarian orientation

9. Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback

10. Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations

11. Algorithmic Content Moderation: Technical and Political Challenges in the Automation of Platform Governance

12. The Internet’s Hidden Rules: An Empirical Study of Reddit Norm Violations at Micro, Meso, and Macro Scales

13. All the News That’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation

14. The Radicalization Risks of GPT-3 and Advanced Neural Language Models

15. Defending Against Neural Fake News

16. Truth, Lies, and Automation

17. Regulatory Markets: The Future of AI Governance

18. Dissociating language and thought in large language models: a cognitive perspective

19. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature

20. GPT‑3: Its Nature, Scope, Limits, and Consequences

21. Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting (S)

22. Are Emergent Abilities of Large Language Models a Mirage? (*)

23. Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4

24. Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge

25. DeepTextMark: Deep Learning based Text Watermarking for Detection of Large Language Model Generated Text

26. Large Language Models Humanize Technology

27. Beyond the Safeguards: Exploring the Security Risks of ChatGPT

28. Dual Use Concerns of Generative AI and Large Language Models

29. Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting

30. Knowledge Refinement via Interaction Between Search Engines and Large Language Models

31. A Drop of Ink may Make a Million Think: The Spread of False Information in Large Language Models

32. BiAs Detection for Large Language Models in the context of candidate screening

33. Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility (*)

34. Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark

35. Machine-Made Media: Monitoring the Mobilization of Machine-Generated Articles on Misinformation and Mainstream News Websites

36. Regulating ChatGPT and other Large Generative AI Models

37. AI pioneer Yoshua Bengio: Governments must move fast to ‘protect the public’

38. Against LLM Maximalism

39. The Web Can Be Your Oyster for Improving Large Language Models

40. Large Language Models can be Guided to Evade AI-Generated Text Detection

41. Appraising the Potential Uses and Harms of LLMs for Medical Systematic Reviews

42. CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing

43. A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation

44. Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models

45. Do Large Language Models Know What They Don’t Know?

46. Multiscale Positive-Unlabeled Detection of AI-Generated Texts

47. Let’s Verify Step by Step

48. GPT-4 System Card

49. The Internal State of an LLM Knows When its Lying

50. Resolving Intent Ambiguities by Retrieving Discriminative Clarifying Questions

51. RARR: Researching and Revising What Language Models Say, Using Language Models

52. CLAM: Selective Clarification for Ambiguous Questions with Generative Language Models

53. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models

54. Taxonomy of Risks Posed by Language Models

55. Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey

56. On the Risk of Misinformation Pollution with Large Language Models

57. GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models

58. Risks and Benefits of Large Language Models for the Environment

59. Exploring the Impact of Data Poisoning Attacks on Machine Learning Model Reliability

60. Extracting Training Data from Large Language Models

61. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?

62. Language (Technology) is Power: A Critical Survey of “Bias” in NLP

63. Chatbots to ChatGPT in a Cybersecurity Space: Evolution, Vulnerabilities, Attacks, Challenges, and Future Recommendations

64. TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI (*)

65. Foundation Models and Fair Use

66. Just Calm Down About GPT-4 Already

67. Citation: A Key to Building Responsible and Accountable Large Language Models

68. How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench

69. Persistent Anti-Muslim Bias in Large Language Models

70. GeDi: Generative Discriminator Guided Sequence Generation

71. Plug and Play Language Models: A Simple Approach to Controlled Text Generation

72. Constitutional AI: Harmlessness from AI Feedback

73. Faith and Fate: Limits of Transformers on Compositionality

74. Lost in the Middle: How Language Models Use Long Contexts

75. LLM Cognitive Judgements Differ From Human

76. The Looming Threat of Fake and LLM-generated LinkedIn Profiles: Challenges and Opportunities for Detection and Prevention

77. Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implications

78. A Critical Review of Large Language Models: Sensitivity, Bias, and the Path Toward Specialized AI

79. Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text?

80. Three Bricks to Consolidate Watermarks for Large Language Models

81. Neural Authorship Attribution: Stylometric Analysis on Large Language Models

82. EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models

83. Mind your Language (Model): Fact-Checking LLMs and their Role in NLP Research and Practice

84. Unraveling the Risks: Cybersecurity and Large Language Models (LLMs)

85. Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning

86. An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning (*)

87. The Curse of Recursion: Training on Generated Data Makes Models Forget (*)

88. Emergent and Predictable Memorization in Large Language Models 

89. Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models

90. Constitutional AI: Harmlessness from AI Feedback

91. The Limitations and Ethical Considerations of ChatGPT

92. On the Societal Impact of Open Foundation Models 

93. Knowledge Conflicts for LLMs: A Survey

94. Steering LLMs Towards Unbiased Responses: A Causality-Guided Debiasing Framework 

95. Reverse Training to Nurse the Reversal Curse

Hallucination

1. A Survey of Hallucination in Large Foundation Models (*)

2. Large language models and the perils of their hallucinations

3. API-Bank: A BenchmarkforTool-AugmentedLLMs: mitigating LLM Hallucination

4. Hallucinations Could Blunt ChatGPT’s Success

5. Mitigating Language Model Hallucination with Interactive Question-Knowledge Alignment

6. Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models → Hallucination mitigation

7. On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?

8. Survey of Hallucination in Natural Language Generation

9. The Curious Case of Hallucinations in Neural Machine Translation

10. Retrieval Augmentation Reduces Hallucination in Conversation

11. Chain-of-Verification Reduces Hallucination in Large Language Models (*)

12. Chain-of-Verification Reduces Hallucination in Large Language Models

13. Zero-Resource Hallucination Prevention for Large Language Models

14. Chain-of-Verification Reduces Hallucination in Large Language Models

15. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

16. Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges

17. Hallucination is Inevitable: An Innate Limitation of Large Language Models

Red-Teaming LLM and Adversarial Attacks

1. Red-Teaming Large Language Models

2. Universal and Transferable Adversarial Attacks on Aligned Language Models. [code]

3. Jailbroken: How Does LLM Safety Training Fail?

4. Explore, Establish, Exploit: Red Teaming Language Models from Scratch

5. Seeing Seeds Beyond Weeds: Green Teaming Generative AI for Beneficial Uses

6. Red Teaming Language Model Detectors with Language Models

7. Query-Efficient Black-Box Red Teaming via Bayesian Optimization

8. Towards best practices in AGI safety and governance: A survey of expert opinion

9. Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

10. Can Large Language Models Change User Preference Adversarially?

11. LLM Hacker’s Handbook

12. LLM Sandboxing: Early Lessons Learned (AntiGPT)

13. Red Teaming Language Models with Language Models. [summary]

14. Universal Adversarial Triggers for Attacking and Analyzing NLP

15. Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned

16. A LLM Assisted Exploitation of AI-Guardian

17. LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?

18. Hacking deep learning: model inversion attack by example

19. Extracting Training Data from Large Language Models

20. Attacking Artificial Intelligence

21. Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies

22. GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher

23. LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked

24. On the Exploitability of Instruction Tuning

25. Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment

26. More than you’ve asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models (S)

27. Prompt Injection attack against LLM-integrated Applications

28. Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection [data poisoning]

29. Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks

30. Prompt Injection Attacks and Defenses in LLM-Integrated Applications

31. People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language Detection

32. Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game

33. "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models

34. AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications 

35. Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations

36. Adversarial Demonstration Attacks on Large Language Models

37. Red Teaming Visual Language Models

Reasoning with LLMs

1. Certified Reasoning with Language Models

2. Can Large Language Models Reason?

3. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

4.  Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks

5. Towards Reasoning in Large Language Models: A Survey (*)

6. Self-Consistency Improves Chain of Thought Reasoning in Language Models

7. Beyond Chain-of-Thought, Effective Graph-of Thought Reasoning in Large Language Models

8. Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations

9. Complexity-Based Prompting for Multi-Step Reasoning

10. Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models

11. . ReAct: Synergizing Reasoning and Acting in Language Models (*)

12. Faithful Reasoning Using Large Language Models

13. MathPrompter: Mathematical Reasoning using Large Language Models

14. Parsel: Algorithmic Reasoning with Language Models by Composing Decompositions

15. Chain-of-Thought Reasoning Without Prompting

16. Chain of Logic: Rule-Based Reasoning with Large Language Models

17. Large Language Models for Mathematical Reasoning: Progresses and Challenges

18. Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts

19. Least-to-most Prompting Enables Complex Reasoning In Large Language Models 

20. Teaching Large Language Models to Reason with Reinforcement Learning

21. Chain of Logic: Rule-Based Reasoning with Large Language Models

22. Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts

Data Generation using LLMs and LLM-Crowd-Sourcing

1. Retrieval-Augmented Generation for AI-Generated Content: A Survey

2. A general purpose, unsupervised reinforcement learning to training agents that can represent the environment. World models

3. DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort: Even with Language models, training data is still a big issue. Some models create training data with minimum human effort.

4. Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor

5. ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks

6. DATED: Guidelines for Creating Synthetic Datasets for Engineering Design Applications

7. Targeted Data Generation: Finding and Fixing Model Weaknesses

8. Automated Annotation with Generative AI Requires Validation

9. TinyStories: How Small Can Language Models Be and Still Speak Coherent English?

10. Contextualized Topic Coherence Metrics. [Code]

11. Power-up! What Can Generative Models Do for Human Computation Workflows?

12. Open-Source Large Language Models Outperform Crowd Workers and Approach ChatGPT in Text-Annotation Tasks

13. Generating Efficient Training Data via LLM-based Attribute Manipulation

14. LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs

15. Active Learning Principles for In-Context Learning with Large Language Models

16. Using ChatGPT for Annotation of Attitude within the Appraisal Theory: Lessons Learned

17. LLMs in the Loop: Leveraging Large Language Model Annotations for Active Learning in Low-Resource Languages

18. Synthetic Data and the Future of AI

Retrieval Augmented Generation (RAG)

1. Retrieval-Augmented Generation for AI-Generated Content: A Survey

2. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

3. Active Retrieval Augmented Generation

4. Retrieval Augmented Language Model Pre-Training

5. LeanDojo: Theorem Proving with Retrieval-Augmented Language Models. [Math]

6. Building RAG-based LLM Applications for Production (Part 1)

7. InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining

8. The Power of Noise: Redefining Retrieval for RAG Systems

Applications of LLMs

1. BloombergGPT: A Large Language Model for Finance

2. Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning

3. What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

4. Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding

5. ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark

6. Flamingo: a Visual Language Model for Few-Shot Learning

7. A Neural Lambda Calculus: Neurosymbolic AI meets the foundations of computing and functional programming

8. Template-Based Named Entity Recognition Using BART.

9. A First Look at LLM-Powered Generative News Recommendation

10. Large Language Model Programs

11. StructGPT: A General Framework for Large Language Model to Reason over Structured Data

12. AI-Augmented Surveys: Leveraging Large Language Models for Opinion Prediction in Nationally Representative Surveys

13. Life of PII — A PII Obfuscation Transformer

14. Chatting with GPT-3 for Zero-Shot Human-Like Mobile Automated GUI Testing

15. Maybe Only 0.5% Data is Needed: A Preliminary Exploration of Low Training Data Instruction Tuning

16. Text Classification via Large Language Models

17. Leveraging Large Language Models in Conversational Recommender Systems

18. CodeT5+: Open Code Large Language Models for Code Understanding and Generation

19. NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Models

20. PALR: Personalization Aware LLMs for Recommendation

21. Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach

22. Large Language Models Can Be Used To Effectively Scale Spear Phishing Campaigns

23. Dreams Are More “Predictable’’ Than You Think

24. MemoryBank: Enhancing Large Language Models with Long-Term Memory

25. CooK: Empowering General-Purpose Language Models with Modular and Collaborative Knowledge

26. A Preliminary Analysis on the Code Generation Capabilities of GPT-3.5 and Bard AI Models for Java Functions

27. Listen, Think, and Understand

28. Self-Agreement: A Framework for Fine-tuning Language Models to Find Agreement among Diverse Opinions

29. Writing your own book: A method for going from closed to open book QA to improve robustness and performance of smaller LLMs

30. Semantic Anomaly Detection with Large Language Models

31. LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention

32. Mathematical Capabilities of ChatGPT

33. SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks

34. Breaking Language Barriers with a LEAP: Learning Strategies for Polyglot LLMs

35. ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing

36. ModuleFormer: Learning Modular Large Language Models From Uncurated Data

37. Tree of Thoughts: Deliberate Problem Solving with Large Language Models

38. Exploring the MIT Mathematics and EECS Curriculum Using Large Language Models

39. MathPrompter: Mathematical Reasoning using Large Language Models

40. Textbooks Are All You Need

41. Language to Rewards for Robotic Skill Synthesis

42. Making Pre-trained Language Models Better Few-shot Learners

43. InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback

44. Unveiling the Potential of Sentiment: Can Large Language Models Predict Chinese Stock Price Movements?

45. Chinese Fine-Grained Financial Sentiment Analysis with Large Language Models

46. Revolutionizing Cyber Threat Detection with Large Language Models

47. On the Uses of Large Language Models to Interpret Ambiguous Cyberattack Descriptions

48. Full Automation of Goal-driven LLM Dialog Threads with And-Or Recursors and Refiner Oracles

49. Large Language Models as Sous Chefs: Revising Recipes with GPT-3

50. LLM-assisted Generation of Hardware Assertions

51. Voyager: An Open-Ended Embodied Agent with Large Language Models

52. DR. AI: How ChatGPT and other AI tools will change healthcare

53. Large Language Models Enable Few-Shot Clustering

54. Evolution through Large Models

55. Text Alignment Is An Efficient Unified Model for Massive NLP Tasks

56. Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review

57. Building Cooperative Embodied Agents Modularly with Large Language Models

58. Utilizing ChatGPT Generated Data to Retrieve Depression Symptoms from Social Media

59. Abstractions, Scenarios, and Prompt Definitions for Process Mining with LLMs: A Case Study

60. Generative Job Recommendations with Large Language Model

61. Recommender Systems in the Era of Large Language Models (LLMs)

62. Can LLMs be Good Financial Advisors?: An Initial Study in Personal Decision Making for Optimized Outcomes

63. Named entity recognition using GPT for identifying comparable companies

64. Investigating ChatGPT’s Potential to Assist in Requirements Elicitation Processes

65. LLM Powered Autonomous Agents

66. Intelligent Mutations in Genetic Programming: OpenAI Proposes Evolution Through Large Models

67. To Infinity and Beyond: SHOW-1 and Showrunner Agents in Multi-Agent Simulations

68. Matching Patients to Clinical Trials with Large Language Models

69. Holistic Exploration on Universal Decompositional Semantic Parsing: Architecture, Data Augmentation, and LLM Paradigm

70. How to use LLMs for Text Analysis

71. Leveraging Large Language Models (LLMs) for Process Mining (Technical Report)

72. Enhancing Job Recommendation through LLM-based Generative Adversarial Networks

73. Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models

74. Large Language Models for Information Retrieval: A Survey

75. Generative Interpretation

76. Dynamic Planning with a LLM

77. Discovering Insights Beyond the Known: A Dialogue Between GPT-4 Agents from Adam and Eve to the Nexus of Ecology, AI, and the Brain

78. A Survey on Large Language Model based Autonomous Agents

79. Context Aware Query Rewriting for Text Rankers using LLM

80. In-Context Learning for Text Classification with Many Labels

81. TopicGPT: A Prompt-based Topic Modeling Framework

82. A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases

83. The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4

84. Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4

85. Large Language Model for Table Processing: A Survey

86. Self-Retrieval: Building an Information Retrieval System with One Large Language Mode

87. Approaching Human-Level Forecasting with Language Models

88. LLM Agent Operating System

Shortcut learning in LLMs

1. Shortcut Learning of Large Language Models in Natural Language Understanding: A Survey (*)

2. Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Models: shortcut learning

3. Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning

4. Shortcut Learning of Large Language Models in Natural Language Understanding

LLM and Toxicity

1. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models.

2. Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicity

3. Reducing Toxicity in Language Models

4. Challenges in Detoxifying Language Models

5. HateCheck: Functional Tests for Hate Speech Detection Models

Applications in Legal Domain

1. Generative Interpretation

2. Legal Prompt Engineering for Multilingual Legal Judgement Prediction

3. Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Texts

4. A Short Survey of Viewing Large Language Models in Legal Aspect

5. Legal Prompting: Teaching a Language Model to Think Like a Lawyer

6. A Brief Report on LawGPT 1.0: A Virtual Legal Assistant Based on GPT-3

7. Anonymity at Risk? Assessing Re-Identification Capabilities of Large Language Models [legal]

8. Large Language Models as Fiduciaries: A Case Study Toward Robustly Communicating With Artificial Intelligence Through Legal Standards

Applications in Software Engineering and Coding

1. Introducing Code Llama, a state-of-the-art large language model for coding

2. CodeCoT and Beyond: Learning to Program and Test like a Developer  [code]

3. Scope is all you need: Transforming LLMs for HPC Code [code] 

4. CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors: Code Generation and CODEX

5. VeriGen: A Large Language Model for Verilog Code Generation

6. On the Use of GPT-4 for Creating Goal Models:  An Exploratory Study 

7. Language Models of Code are Few-Shot Commonsense Learners.: Code Generation and CODEX

8. Enabling Programming Thinking in Large Language Models Toward Code Generation: Code Generation and CODEX

9. Enabling Programming Thinking in Large Language Models Toward Code Generation

10. Think Outside the Code: Brainstorming Boosts Large Language Models in Code Generation

11. The potential of LLMs for coding with low-resource and domain-specific programming languages

12. Software Testing with Large Language Model: Survey, Landscape, and Vision

13. Analysis of ChatGPT on Source Code

14. Structured Chain-of-Thought Prompting for Code Generation

15. So Much for ‘Learn to Code’ [job]

16. Large Language Models for Software Engineering: Survey and Open Problems

17. A comparison of Human, GPT-3.5, and GPT-4 Performance in a University-Level Coding Course

Question-Answering

1. PDFTriage: Question Answering over Long, Structured Documents

2. Evaluating Open-Domain Question Answering in the Era of Large Language Models

3. Integrating ChatGPT with internal knowledge base and question-answer platform

4. Towards Expert-Level Medical Question Answering with Large Language Models

5. Entity-Based Knowledge Conflicts in Question Answering

6. Large Language Models Need Holistically Thought in Medical Conversational QA

7. Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves

8. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions 

Summarization

1. Summarization is (Almost) Dead

2. Rouge: A package for automatic evaluation of summaries

3. Improved Beam Search for Hallucination Mitigation in Abstractive Summarization

4. Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articles

5. When Large Language Models Meet Citation: A Survey

6. Investigating Gender Bias in News Summarization

7. ODSum: New Benchmarks for Open Domain Multi-Document Summarization

LLM applications in biomedical, healthcare and pharma

1. Utilizing Large Language Models for Natural Interface to Pharmacology Databases

2. ChatGPT-powered Conversational Drug Editing Using Retrieval and Domain Feedback

3. Towards Generalist Biomedical AI

4. A Platform for the Biomedical Application of Large Language Models

5. Large language model AI chatbots require approval as medical devices

6. Dangerous chatbots: Prof. Stephen Gilbert calls for AI chatbots to be approved as medical devices

7. AI chatbots not yet ready for clinical use

8. Comparative Analysis of Drug-GPT and ChatGPT LLMs for Healthcare Insights: Evaluating Accuracy and Relevance in Patient and HCP Contexts

9. From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery

10. Utilizing Large Language Models for Natural Interface to Pharmacology Databases

11. Matching Patients to Clinical Trials with Large Language Models

12. Distilling Large Language Models for Biomedical Knowledge Extraction: A Case Study on Adverse Drug Events

13. Learning to Generate Novel Scientific Directions with Contextualized Literature-based Discovery

14. CancerGPT: Few-shot Drug Pair Synergy Prediction using Large Pre-trained Language Models

15. Generative AI in Life Sciences: Use Cases & Examples in 2023

16. How to Safely Integrate Large Language Models Into Health Care

17. MedEdit: Model Editing for Medical Question Answering with External Knowledge Bases

18. Towards Accurate Differential Diagnosis with Large Language Models

19. Rationalism in the face of GPT hypes: Benchmarking the output of large language models against human expert-curated biomedical knowledge graphs

20. Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data

Graphs + LLMs

1. Natural Language is All a Graph Needs

2. Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs

3. Can Language Models Solve Graph Problems in Natural Language? (*)

4. DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphs

5. Neural Embeddings for Protein Graphs

6. Talk Like a Graph: Encoding Graphs for Large Language Models

7. Large Language Models on Graphs: A Comprehensive Survey

Knowledge Graphs + LLMs

1. Language Models are Open Knowledge Graphs

2. Large Language Models and Knowledge Graphs: Opportunities and Challenges

3. Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding → Hallucination mitigation and knowledge graph

4. Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphs

5. LLM-assisted Knowledge Graph Engineering: Experiments with ChatGPT

6. Large Language Models and Knowledge Graphs: Opportunities and Challenges

7. LLMs4OL: Large Language Models for Ontology Learning

8. Generating Faithful Text From a Knowledge Graph with Noisy Reference Text

9. Knowledge Graph Completion Models are Few-shot Learners: An Empirical Study of Relation Labeling in E-commerce with LLMs

10. Unifying Large Language Models and Knowledge Graphs: A Roadmap (*)

11. GraphCare: Enhancing Healthcare Predictions with Open-World Personalized Knowledge Graphs

12. Language Models sounds the Death Knell of Knowledge Graphs

13. GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graph

14. Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning

15. Head-to-Tail: How Knowledgeable are Large Language Models (LLM)? A.K.A. Will LLMs Replace Knowledge Graphs?

16. Can Language Models Learn Analogical Reasoning? Investigating Training Objectives and Comparisons to Human Performance

17. RelBERT: Embedding Relations with Language Models 

18. Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey

19. On the Evolution of Knowledge Graphs: A Survey and Perspective

20. Two Heads Are Better Than One: Integrating Knowledge from Knowledge Graphs and Large Language Models for Entity Alignment

21. Knowledge Fusion of Large Language Models

Intent Classification

1. Exploring Zero and Few-shot Techniques for Intent Classification

2. Can ChatGPT Detect Intent? Evaluating Large Language Models for Spoken Language Understanding

3. Prompt Learning With Knowledge Memorizing Prototypes For Generalized Few-Shot Intent Detection

Scaling LLMs, Compute Cost and SLMs

1. Training Compute-Optimal Large Language Models

2. Scaling Language Models: Methods, Analysis & Insights from Training Gopher

3. Scaling Transformer to 1M tokens and beyond with RMT

4. An empirical analysis of compute-optimal large language model training

5. Reducing Activation Recomputation in Large Transformer Models

6. Mixed Precision Training

7. The Power of Scale for Parameter-Efficient Prompt Tuning

8. OPT: Open Pre-trained Transformer Language Models

9. LLaMA: Open and Efficient Foundation Language Models

10. Scalable Training of Language Models using JAX pjit and TPUv4

11. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

12. On Hate Scaling Laws For Data-Swamps

13. LLMeBench: A Flexible Framework for Accelerating LLMs Benchmarking

14. Numbers every LLM Developer should know

15. The Cost of Training NLP Models: A Concise Overview

16. Counting the Cost of Training Large Language Models

17. LLM in a flash: Efficient Large Language Model Inference with Limited Memory 

18. SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling

19. Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models

20. The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

21. OpenELM: An Efficient Language Model Family with Open Training and Inference Framework

22. Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

Generative AI: Plagiarism and Education

1. Detecting LLM-Generated Text in Computing Education: A Comparative Study for ChatGPT Cases

2. Large language models challenge the future of higher education

3. Generative AI: Implications and Applications for Education

4. Can AI-Generated Text be Reliably Detected?

5. OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples

6. Limits of Detecting Text Generated by Large-Scale Language Models

7. How Useful are Educational Questions Generated by Large Language Models?

LLM (Machine) Unlearning

1. A Survey of Machine Unlearning

2. Machine Unlearning

3. Zero-Shot Machine Unlearning

4. Measuring and Modifying Factual Knowledge in Large Language Model

5. One-Shot Machine Unlearning with Mnemonic Code

6. Machine Unlearning: A Survey (*)

7. Machine Unlearning: its nature, scope, and importance for a “delete culture”

8. In-Context Unlearning: Language Models as Few Shot Unlearners

9. Privacy Adhering Machine Un-learning in NLP

10. Rethinking Machine Unlearning for Large Language Models

Responsible AI, LLM ethics and AI regulation

1. Do Large GPT Models Discover Moral Dimensions in Language Representations? A Topological Study Of Sentence Embeddings

2. The Ethics of AI Value Chains: An Approach for Integrating and Expanding AI Ethics Research, Practice, and Governance

3. Ethical and social risks of harm from Language Models

4. CoCoMo Computational Consciousness Modeling for Generative and Ethical AI

5. Ethical and social risks of harm from Language Models

6. The Dark Side of ChatGPT: Legal and Ethical Challenges from Stochastic Parrots and Hallucination

7. The Ethics of AI Value Chains: An Approach for Integrating and Expanding AI Ethics Research, Practice, and Governance

8. Evaluating the Moral Beliefs Encoded in LLMs

9. Language (Technology) is Power: A Critical Survey of “Bias” in NLP (*)

10. Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets

11. Demographic Dialectal Variation in Social Media: A Case Study of African-American English

12. Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English

13. 70. Large Language Models are not Fair Evaluators

14. Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc

15. Frontier AI Regulation: Managing Emerging Risks to Public Safety

16. TrustLLM: Trustworthiness in Large Language Models

LLM Agents

1. Shall We Talk: Exploring Spontaneous Collaborations of Competing LLM Agents

LLaMA

1. How Good Are Low-bit Quantized LLaMA3 Models? An Empirical Study

LMM: Large Multimodal Models

1. Chain-of-Thought Prompt Distillation for Multimodal Named Entity and Multimodal Relation Extraction

2. NExT-GPT: Any-to-Any Multimodal LLM

3. Contextual Object Detection with Multimodal Large Language Models

4. The Multimodal and Modular Ai Chef: Complex Recipe Generation From Imagery

5. MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action

6. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity

7. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity

8. The Dawn of LMMs: Preliminary Explorations with GPT-4V(vision)

9. AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling

What is next?

1. What Should Data Science Education Do with Large Language Models

2. Choose Your Weapon: Survival Strategies for Depressed AI Academics

3. International AI institutions 

4. Cognition is All You Need -- The Next Layer of AI Above Large Language Models

5. Silicon Valley is pricing academics out of AI research

6. First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models

7. Now, Later, and Lasting: Ten Priorities for AI Research, Policy, and Practice

Books

1. Hands-On Large Language Models

Resources and Blogs

1. Falcon-40b LLM

2. Langchain

3. 5 Free Resources to Master LLMs

4. Open LLM-Perf Leaderboard

5. Chip Huyen's blog

6. Anti-hype LLM reading list

7. GPT in 60 Lines of NumPy

8. Jay Alammar's blog

9. Blogposts for Generative AI

10. Prompt Engineering Institute

11. Clinical Language Models

12. Want to run a large language model on your laptop? It's possible now! - [Part 1]

13. LLM Gaurd: red-teaming and prompt injection  

14. Word2Vec Embeddings from the ground up, for the ML-Adjacent

15. Top 10 List of Large Language Models Reshaping the Open-Source Arena in 2023

16. LLM Fine Tuning Guide for Enterprises in 2023

17. Large Language Models and the Future of Custom, Fine-tuned LLMs

18. Understanding the Power of Transformers: A Guide to Sentence Embeddings in Spark NLP

19. Selecting Large Language Model Customization Techniques

20. Prompt Engineering: In-Context Learning

21. Claude vs ChatGPT for Data Science: A Comparative Analysis

22. Mastering Customer Segmentation with LLM

23. 12 GPT-4 Open-Source Alternatives

24. What is GPT-4 and Why Does it Matter?

25. Using LLMs for Search with Dense Retrieval and Reranking

26. Introducing Chat with Retrieval-Augmented Generation (RAG)

27. A Very Gentle Introduction to Large Language Models without the Hype

28. Prompt Engineering

29. Building LLM applications for production (blog post)

30. Fine-Tuning Large Language Models (LLMs)

31. Chain-of-Thought Prompting — Improve Accuracy by Getting LLMs to Reason

32. Word2Vec code 

33. Distilling step-by-step: Outperforming larger language models with less training data and smaller model sizes

34. LLM Hacking: Prompt Injection Techniques

35. Research Papers for NLP Beginners

36. Prompt Injection Threat is Real, Will Turn LLMs into Monsters

37. Generative AI — Protect your LLM against Prompt Injection in Production

38. Best practices for prompt engineering with OpenAI API 

39. LLM Vulnerabilities 

40. The Internet’s New Favorite AI Proposes Torturing Iranians and Surveilling Mosques

41. Researchers found a command that could ‘jailbreak’ chatbots like Bard and GPT

42. Cybersecurity experts are warning about a new type of AI attack

43. Prompt injection explained, with video, slides, and a transcript

44. The ELI5 Guide to Prompt Injection: Techniques, Prevention Methods & Tools

45. Prompt injection: What’s the worst that can happen?

46. Build and run llama2 LLM locally

47. Announcing LangSmith, a unified platform for debugging, testing, evaluating, and monitoring your LLM applications

48. ChainForge: an open-source visual programming environment for prompt engineering

49. LLM Collection

50. Getting started with Llama 

51. Chat jailbreak (prompt injection)

52. Building Transformer Models with Attention Crash Course. Build a Neural Machine Translator in 12 Days

53. Llamaindex

54. Introducing Claude 2.1

55. Intro to Large Language Models (talk)

56. Using Machine Learning to Reduce Toxicity Online

57. llamafile is the new best way to run a LLM on your own computer

58. A practical guide to deploying Large Language Models Cheap, Good *and* Fast

59. How to secure your RAG application against LLM prompt injection attacks

60. CPT4All: A free-to-use, locally running, privacy-aware chatbot.

61. MLC LLM

62. LM studio

63. Short courses on Generative AI

64. Model alignment protects against accidental harms, not intentional ones

65. Shervin Amidi: Recurrent Neural Networks cheatsheet

66. NLP is more than just LLMs

67. Planning red teaming for large language models (LLMs) and their applications

68. Announcing Purple Llama: Towards open trust and safety in the new world of generative AI

69. 5 ways to deploy your own large language model

70. Open AI prompt engineering guide 

71. How to Chunk Text Data — A Comparative Analysis

72. Pre-training vs Fine-Tuning vs In-Context Learning of Large Language Models

73. Prompt engineering: openAI guide

74. ChatGPT is going to change education, not destroy it

75. Merge Large Language Models with mergekit 

76. MLOps roadmap 2024

77. Understanding Deep Learning

78. A Comprehensive Study of Trustworthiness in Large Language Models

79. Hallucinating Law: Legal Mistakes with Large Language Models are Pervasive

80. RAG makes LLMs better and equal

81. Understanding LoRA: Low-rank Adaption of Large Language Models

82. The Shift from Models to Compound AI Systems 

83. CS25: Transformers United V4 (Stanford Transformer course)

84. Louis Lebbos's reading list

85. Tiny but mighty: The Phi-3 small language models with big potential

86. Measuring  trends in AI 

87. Unlearning Copyrighted Data From a Trained LLM – Is It Possible?

88. How AI copyright lawsuits could make the whole industry go extinct

89. Getting Started with GPT-4 API