Welcome to our research portal! 

Our goals for this year's cohort include:

• Learn foundations of AI/ML and agentic techniques

• Appreciate the advances in LLMs and become familiar with toolchains for using them as consumers

• Understand what is algorithmic bias, and in particular bias in AI/ML/LLMs

• Study a few major algorithmic faux pas made by well intentioned engineers.  We will study (possibly) unintentional consequences of algorithms leading to unfair biases.

• Hypothesize how to mitigate as much as possible biases inherent in computer algorithms and data sets, as as to minimize harm to society.

We intend for you to engage in serious scientific investigation in groups, and to learn how to leverage each group member in your research.  No major research is done alone in the real world.  If you are interested in more specificity, here are our minimal research expectations for this group.

Research Assistant: this term, we have the pleasure of having Ananya G. and Theodore M. be our research assistants.   They will be sharing their current research with you as well as trying to help each of your group research effort. 

2025 Fall Schedule

Our first meeting will be Sep 27 (Saturday) to welcome the fall cohorts of researchers!

2025/09/27

• OpenAI Agents Framwork & Guardrails

• repo: github.com/philmui/research2025

• recording: [link]

Topics that we will try to cover through our weekend meetings:

• Agent Frameworks: 

  • OpenAI agents

  • CrewAI agents

  • Amazon Strands

  • LlamaIndex Agents

  • Langchain Agents

  • Google ADK agents

• Agent Safety & Guardrails

• Multi-Agents Orchestration

• Agent Optimization

• Knowledge Graphs

• Using VLM to outperform OCR

• Evaluation: how to use common datasets for validating agents

2025/10/04

• Input & output guardrails

• code: [link]

• recording: [link]

• Readings:

  • Trailhead: Define the Agent Guardrails [link]

  • Boyuan Zheng, Zeyi Liao, Scott Salisbury, Zeyuan Liu, Michael Lin, Qinyuan Zheng, Zifan Wang, Xiang Deng, Dawn Song, Huan Sun, Yu Su.  "WebGuard: Building a Generalizable Guardrail for Web Agents" arXiv:2507.14293, July 18, 2025 [link] [github]

2025/10/18

• Agent Graphs

• slides: [link]

• recording: [link]

• Readings:

  • OpenAI: Introducing Agent Kit [link]

  • ElevenLabs: Workflows [link]

  • P. Mui. "Agentforce’s Agent Graph: Toward Guided Determinism with Hybrid Reasoning."  Salesforce Engineering Blog, October 20, 2025.  [link]

2025/10/25

• Poster & Oral Presentation Notes [link]

• LangChain & LangSmith

• code: [link]

• recording: [link]

• Readings:

  • LangChain v1.0 [link]

2025/11/02

• Multi-Agents Orchestration

• slides: [link]

• code: [link]

• recording: [link]

• Readings:

  • Zhipeng Hou, Junyi Tang, Yipeng Wang.  “HALO: Hierarchical Autonomous Logic-Oriented Orchestration for Multi-Agent LLM Systems.” arXiv:2505.13516 [cs.MA], May 17, 2025. [arxiv]

2025/11/09

• MMLU Evaluation

• Multi-Agents Orchestration (con't)

• code: [link]

• recording: [link]

• Readings:

  • (same HALO paper)

  • MMLU (Massive Multitask Language Understanding) Evaluation Benchmark [link]

  • T. Mui. "What 15 years of Daily opinion pieces reveal about diversity", Stanford Daily Op-Ed, Nov 5, 2025. [link]

2025/11/16

• SCCUR poster & talk schedule:

  • [schedule]

  • Date: Nov 22 (Saturday) @ CSU Channel Islands

• recording: [link]

2025/11/23 (NO MEETING)

• See you in CSU Channel Islands for SCCUR

  • [schedule]

2025/11/30 (Thanksgiving Weekend)

• Building Multi-agents

• code: [link]

• recording: [link]

• If enough people are interested, we will meet to show off the final HALO Orchestrator implementation & MMLU Evaluation!

2025/12/06

• Smart Router Agent building

• code: [link]

• recording: [link]

• Readings:

  • Feijie Wu, Zitao Li, Fei Wei, Yaliang Li, Bolin Ding, Jing Gao, "Talk to Right Specialists: Routing and Planning in Multi-agent System for Question Answering", arXiv:2501.07813, January 14, 2025. [link]

2025/12/13

• Multi-agent Orchestration (con't)

• Readings:

  • Junwei Yu, Yepeng Ding, Hiroyuki Sato.  DynTaskMAS: A Dynamic Task Graph-driven Framework for Asynchronous and Parallel LLM-based Multi-Agent Systems, arXiv:2503.07675, December 2, 2025. [link]

    • demonstrates that dynamic task graph orchestration achieves 21-33% execution time reduction and near-linear throughput scaling up to 16 concurrent agents. 

  • Philip Drammeh, Multi-Agent LLM Orchestration Achieves Deterministic, High-Quality Decision Support for Incident Response, arXiv:2511.15755, November 19, 2025. [link]

    • shows the quality case: multi-agent orchestration achieves 100% actionable recommendations versus 1.7% for single-agent, with zero quality variance -- enabling SLA commitments impossible with simpler approaches. 

  • Jiaqi Wu, Qinlao Zhao, Zefeng Chen, Kai Qin, "GAP: Graph-based Agent Planning with Parallel Tool Use and Reinforcement Learning." arXiv:2510.25320, October 29, 2025. [link]

  • Anna Alexandra Grigoryan, "Why Do Multi-Agent LLM Systems Fail?", Medium, April 30, 2025. [link]

  • ByteByteGo, How Anthropic Built a Multi-Agent Research System, ByteByteGo, September 16, 2025. [link]

Current Research Projects:

Reference Materials

• "Outline Goals" for a Conference Abstract Submission (link)