Deep Hierarchical Variational Autoencoders for World Models in Reinforcement Learning
A VAE-based “world model” learns a compact latent representation of the environment so an agent can train efficiently with fewer real interactions. This work explores NVAE-style hierarchical VAEs to improve representation quality for complex visual environments.
Comparative Study of World Models, NVAE-Based Hierarchical Models, and NoisyNet-Augmented Models in CarRacing-V2 (ICLR 2025 workshop paper)
An experimental comparison of (i) standard World Models, (ii) NVAE-based hierarchical world models, and (iii) NoisyNet-augmented exploration. Highlights trade-offs among reward performance, stability, and compute, showing how stronger latent models and exploration noise affect learning.
Tags: VAE, Hierarchical VAE (NVAE), Model-Based RL, Exploration
Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning
A unified approach combining DRL + VAE + active learning to detect anomalies with minimal labeled data. Uses sequential modeling (LSTM) and leverages VAE reconstruction behavior as part of the anomaly signal.
Tags: VAE, Anomaly Detection, RL, Active Learning, Time Series
Dynamic and Adaptive Feature Generation with LLMs
Feature generation transforms raw data into an optimized feature space for downstream modeling, but many automated methods lack explainability, generality, and flexibility. This project introduces an LLM-driven approach that uses feature-generating prompts to produce dynamic, adaptive features with improved interpretability. The method is designed to generalize across data types and tasks, and experiments show consistent gains over prior automated feature engineering baselines.
Tags: Large Language Models (LLMs), Automated Feature Engineering, Feature Generation, Representation Learning, Interpretability
Publication:
Zhang, Xinhao, Jinghan Zhang, Banafsheh Rekabdar, Yuanchun Zhou, Pengfei Wang, and Kunpeng Liu. “Dynamic and Adaptive Feature Generation with LLM.” AAAI (2024).
LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection
A framework that integrates LLM-based potential functions for reward shaping with RL, plus VAE-enhanced dynamic reward scaling and active learning with label propagation. The RL agent uses LLM-derived semantic rewards for exploration while VAE reconstruction error contributes unsupervised anomaly signals; evaluated on benchmarks including Yahoo-A1 and SMD.
Semantic Reward Shaping: LLMs in Reinforcement Learning for Time Series Anomaly Detection(anonymous/preprint/submission version)
A submission-friendly version of the same core idea: LLMs provide semantic reward shaping, and VAEs provide an unsupervised anomaly component, improving learning under limited labels.
Tags: LLMs, Reward Shaping, RL, VAE, Time Series
Golchin, Bahareh, Banafsheh Rekabdar, and Danielle Justo. “LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection.” Accepted for publication in Proceedings of the IEEE International Conference on Semantic Computing (ICSC), 2025.
Noisy-Defense Variational Auto-Encoder (ND-VAE): An Adversarial Defense Framework to Eliminate Adversarial Attacks
A defense mechanism that combines strengths of NVAE and Defense-VAE, trained with noisy images to remove adversarial perturbations while preserving details. Evaluated on MNIST and Fashion-MNIST, including strong performance in generalizing across attacks.
Biologically Inspired Variational Auto-Encoders for Adversarial Robustness
Introduces a biologically inspired “sleep phase” in a VAE-based defense (Defense-VAE-Sleep) to improve generalization and robustness. Experiments include datasets like CelebA, MNIST, and Fashion-MNIST.
Tags: VAE, Robustness, Adversarial Defense, Purification
GAN-based Defense Mechanism against Word-level Attacks for Arabic Transformer-based Model (results pending update)
A GAN-based (InfoGAN-style) defense strategy designed to improve robustness of transformer models for Arabicagainst word-substitution attacks, using generated perturbations and discrimination to improve resilience.
Tags: GAN, Robustness, NLP Security, Low-Resource NLP, Transformers
Sample Generation with CVAE + GAN
Explores combining a Conditional VAE (CVAE) and GAN to model complex, high-dimensional distributions relevant to planning/robotics—aiming to generate feasible samples conditioned on task constraints.
Tags: CVAE, GAN, Robotics, Generative Modeling
OSA-Diff: An Origin Sampling Based Adversarial Attack Using Diffusion Models
This project explores how diffusion models (DDPMs) can be used not only for generation/denoising, but also to create end-to-end adversarial perturbations. The core idea is an origin-sampling–based reconstruction process that injects carefully designed perturbations during the denoising trajectory, producing high-success adversarial examples that can remain visually subtle. The method is designed to be more computationally efficient than typical diffusion training while still converging to high-quality outputs.
Tags: Diffusion Models, DDPM, Adversarial Attacks, Security, Robustness