SPARSE RECOVERY 

STATISTICAL GUARANTEES FOR LEARNING WITH NON-LINEAR FOURIER FEATURES

Random features (RFs) framework enables nonlinear representation of input signals in a systematic manner and provides a computationally attractive alternative to kernel based methods. Algorithms that utilize random features  have shown remarkable performance in a wide range of real-world data regressions/classification applications. Our results provide theoretical insights on why good estimation performance can be obtained even with a small number of features in these applications. 

A. Özçelikkale, Sparse Recovery with Non-Linear Fourier Features, ICASSP2020. 

DOUBLE-DESCENT IN LASSO AND BASIS PURSUIT 

We present a novel analysis of feature selection in linear models by the convex framework of least absolute shrinkage operator (LASSO) and basis pursuit (BP). Our analysis pertains to a general overparametrized scenario. When the numbers of the features and the data samples grow proportionally, we obtain precise expressions for the asymptotic generalization error of LASSO and BP. Considering a mixture of strong and weak features, we provide insights into regularization trade-offs for double descent for l1-norm minimization.

D. Bosch,  Ashkan Panahi,  A. Özçelikkale, Devdatt Dubhashi, Random Features Model with General Convex Regularization: A Fine Grained Analysis with Precise Asymptotic Learning Curves, Proc.  of The 26th International Conference on Artificial Intelligence and Statistics, (AISTATS2023), PMLR 206:11371-11414, 2023

D. Bosch, Ashkan Panahi, A. Özçelikkale, "Double Descent in Feature Selection: Revisiting LASSO and Basis Pursuit." (Accepted to ICML 2021 Workshop Overparameterization: Pitfalls & Opportunities)

SPARSE MODELS AND THE MMSE ESTIMATION

We model the low degree of freedom of the family signals we are interested in through a covariance matrix model. We focus on the  unitary transformation that relates the canonical signal domain and the measurement domain.  We investigate the error performance, both in the average, and also in terms of guarantees that hold with high probability. We explore connections to compressive sensing, The concept of coherence of random fields as defined in optics and applications in energy harvesting.

A. Özçelikkale, S. Yüksel, and H. M. Ozaktas, “Unitary Precoding and Basis Dependency of MMSE Performance for Gaussian Erasure Channels”, IEEE Trans. Information Theory , vol. 60, no. 11, pp. 7186-7203, Nov. 2014.

A. Özçelikkale, T. McKelvey, and M. Viberg, “Remote Estimation of Correlated Sources under Energy Harvesting Constraints”, IEEE Transactions on Wireless Communications, vol. 17, pp. 5300–5313, Aug. 2018

A. Özçelikkale, T. McKelvey, and M. Viberg, ``Performance Bounds for Remote Estimation with an Energy Harvesting Sensor'',  Proc. IEEE Int. Symp. Information Theory (ISIT), pp. 460-464, July 2016.