RESEARCH

LEARNING OVER NETWORKS

Distributed learning provides an attractive framework for scaling the learning task by sharing the computational load over multiple nodes in a network. Despite the vast interest in distributed learning and central importance of generalization performance in data science, generalization performance of distributed approaches is not well understood. We address this gap by focusing on the setting where the model is partitioned over a network of nodes.

We have showed how the generalization error depends heavily on the partitioning of the model parameters among the nodes. Our results highlighted a typically overlooked relationship between the training and generalization error in distributed learning. In particular, distributed learning schemes can significantly amplify the gap between the training error and the generalization error: A distributed solution with a training error that is on the same level as that of the centralized solution is not guaranteed to have a generalization error that is as low as that of the centralized solution. These results are directly connected to double descent curves which illustrate that the relationship between the number of data points and the assumed model dimension (corresponding to the size of the partial model in a node in the distributed learning setting) can significantly affect the generalization error.

Publications:

M. Hellkvist, A. Özçelikkale, A. Ahlén, Distributed Continual Learning with CoCoA in High-dimensional Linear Regression, in IEEE Trans. on Signal Processing, vol. 72, pp. 1015-1031, 2024, doi: 10.1109/TSP.2024.3361714

M. Hellkvist, A. Özçelikkale, A. Ahlén, Linear Regression with Distributed Learning: A Generalization Error Perspective IEEE Trans.on Signal Processing, vol. 69, pp. 5479-5495, 2021

M. Hellkvist, A. Özçelikkale, A. Ahlén, Continual Learning with Distributed Optimization:Does COCOA Forget? . Preprint, 2023

M. Hellkvist, A. Özçelikkale, A. Ahlén, Generalization Error for Linear Regression under Distributed Learning, Proc. IEEE International Workshop on Signal Processing Advances in Wireless Communications, (SPAWC), 2020

ACTIVE INFERENCE

Active Inference (ActInf) is a neuroscience inspired framework that is originally used to explain how biological agents learn and act in dynamic environments. It is based on minimizing a free energy bound on Bayesian surprise. Goal-directed behavior is elicited by introducing prior beliefs on the underlying generative model. We are exploring connections of ActInf to standard control and extend ActInf theory in multiple directions, such as chance constraints.

Publications:

T. van de Laar, A. Özçelikkale, H. Wymeersch , Application of the Free Energy Principle to Estimation and Control, IEEE Trans. on Signal Processing, 2021

T. van de Laar, I. Senoz, Ayça Özçelikkale, H. Wymeersch, Chance-Constrained Active Inference, IEEE Neural Computation, 2021

SPARSE RECOVERY

STATISTICAL GUARANTEES FOR LEARNING WITH NON-LINEAR FOURIER FEATURES

Random features (RFs) framework enables nonlinear representation of input signals in a systematic manner and provides a computationally attractive alternative to kernel based methods. Algorithms that utilize random features have shown remarkable performance in a wide range of real-world data regressions/classification applications. Our results provide theoretical insights on why good estimation performance can be obtained even with a small number of features in these applications.

A. Özçelikkale, Sparse Recovery with Non-Linear Fourier Features, ICASSP2020.

DOUBLE-DESCENT IN LASSO AND BASIS PURSUIT

We present a novel analysis of feature selection in linear models by the convex framework of least absolute shrinkage operator (LASSO) and basis pursuit (BP). Our analysis pertains to a general overparametrized scenario. When the numbers of the features and the data samples grow proportionally, we obtain precise expressions for the asymptotic generalization error of LASSO and BP. Considering a mixture of strong and weak features, we provide insights into regularization trade-offs for double descent for l1-norm minimization.

D. Bosch, Ashkan Panahi, A. Özçelikkale, Devdatt Dubhashi, Random Features Model with General Convex Regularization: A Fine Grained Analysis with Precise Asymptotic Learning Curves, Proc. of The 26th International Conference on Artificial Intelligence and Statistics, (AISTATS2023), PMLR 206:11371-11414, 2023

D. Bosch, Ashkan Panahi, A. Özçelikkale, "Double Descent in Feature Selection: Revisiting LASSO and Basis Pursuit." (Accepted to ICML 2021 Workshop Overparameterization: Pitfalls & Opportunities)

SPARSE MODELS AND THE MMSE ESTIMATION

We model the low degree of freedom of the family signals we are interested in through a covariance matrix model. We focus on the unitary transformation that relates the canonical signal domain and the measurement domain. We investigate the error performance, both in the average, and also in terms of guarantees that hold with high probability. We explore connections to compressive sensing, The concept of coherence of random fields as defined in optics and applications in energy harvesting.

A. Özçelikkale, S. Yüksel, and H. M. Ozaktas, “Unitary Precoding and Basis Dependency of MMSE Performance for Gaussian Erasure Channels”, IEEE Trans. Information Theory , vol. 60, no. 11, pp. 7186-7203, Nov. 2014.

A. Özçelikkale, T. McKelvey, and M. Viberg, “Remote Estimation of Correlated Sources under Energy Harvesting Constraints”, IEEE Transactions on Wireless Communications, vol. 17, pp. 5300–5313, Aug. 2018

A. Özçelikkale, T. McKelvey, and M. Viberg, ``Performance Bounds for Remote Estimation with an Energy Harvesting Sensor'', Proc. IEEE Int. Symp. Information Theory (ISIT), pp. 460-464, July 2016.

WIRELESS SYSTEMS WITH ENERGY HARVESTING CAPABILITIES: SENSING AND COMMUNICATIONS TRADE-OFFS

Energy harvesting (EH) solutions offer a promising framework for future wireless sensing systems. Instead of completely relying on a fixed battery or power from the grid, nodes with EH capabilities can collect energy from the environment, such as solar power or power from mechanical vibrations. In addition to enabling energy autonomous sensing systems, EH capabilities also offer prolonged network life-times and enhanced mobility for the nodes in the network.

One of the key issues in the design of EH systems is the intermittent nature of the energy supply. In a traditional device, the energy that can be used for any sensor task whether it is sensing or communications has either a fixed known value for each task or there is a total energy constraint. In contrast, for an EH node, the energy available for each task depends on the energy used in previous transmissions and energy that may be available in the future.

Our approach: We adopt a cross-layer approach where the sensing and communications tasks for a sensing system with EH capabilities are optimized jointly.

My EU Marie Skłodowska-Curie Project: "GRENHAS: Green and Smart Communications with Energy Harvesting: A Signal Processing Approach".

Publications:

A. Özçelikkale, T. McKelvey, and M. Viberg, ``Transmission strategies for remote estimation with an Energy Harvesting Sensor'', IEEE Transactions on Wireless Communications, vol. 16, no. 7, pp. 4390-4403, July 2017

R. Du, A. Özçelikkale, C. Fischione, and M. Xiao, ``Towards Immortal Wireless Sensor Networks by Optimal Energy Beamforming and Data Routing”, IEEE Transactions on Wireless Communications, vol. 17, pp. 5338–5352, Aug. 2018.

A. Özçelikkale, M.Koseoglu, M. Srivastava and Anders Ahlén, "Deep reinforcement learning based energy beamforming for powering sensor networks", IEEE International Workshop on Machine Learning for Signal Processing (MLSP), 2019 (See also here)

A. Özçelikkale, M.Koseoglu,and M. Srivastava, “Optimization vs. reinforcement learning for wirelessly powered sensor networks", IEEE International Workshop on Signal Processing Advances in Wireless Communications, (SPAWC) June 2018. (Invited Paper)

A. Özçelikkale, T. McKelvey, and M. Viberg, ``Performance Bounds for Remote Estimation with an Energy Harvesting Sensor'', Proc. IEEE Int. Symp. Information Theory (ISIT), pp. 460-464, July 2016.

A. Özçelikkale, T. McKelvey, and M. Viberg, ``Transmission strategies for remote estimation under energy harvesting constraints'', Proc. European Conference on Signal Processing (EUSIPCO), pp. 572-576, Aug. 2016.

R. Du, A. Özçelikkale, C. Fischione, and M. Xiao, “Optimal Energy Beamforming and Data Routing for Immortal Wireless Sensor Networks", 2017 IEEE International Conference on Communications (ICC), 2017.

ENERGY HARVESTING: TRANSMISSION STRATEGIES FOR DEVICES WITH WIRELESS POWER TRANSFER CAPABILITIES

Efficient usage of energy resources is a growing concern in today's communication systems. Solutions that consider wireless power transfer (WPT) offer an attractive alternative. In systems with WPT capabilities, the two tasks, information and power transfer is done simultaneously in a wireless medium and simultaneous information and power transfer strategies are developped. The optimal transmission strategies for these two tasks are different, hence novel transmission strategies have to be designed in order to be able to do these two tasks most efficiently. In addition to efficient usage of energy sources, WPT also reduces the dependence on batteries and the grid and brings flexibility in a wide range of applications including wireless sensor networks and smart homes.

Our approach: Practical receiver structures with linear filtering, low complexity designs such as linear precoders, power allocation methods are important ingredients in our work. The resulting solutions complement the existing information theoretic solutions, and will contribute to creating future green and smart communication systems.

My EU Marie Skłodowska-Curie Project: "GRENHAS: Green and Smart Communications with Energy Harvesting: A Signal Processing Approach".

Publications:

A. Özçelikkale, and T. M. Duman, “Linear Precoder Design for Simultaneous Information and Energy Transfer over Two-User MIMO Interference Channels”, IEEE Transactions on Wireless Communications, vol. 14, no. 10, pp. 5836-5847, Oct. 2015.

A. Özçelikkale, T. McKelvey, and M. Viberg, “Wireless Information and Power Transfer in MIMO Channels under Rician fading”, Proc. 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3187-3191, April 2015.

A. Özçelikkale, T. McKelvey, and M. Viberg, ``Simultaneous information and power transfer with transmitters with hardware impairments'', 2016 International Symposium on Wireless Communication Systems (ISWCS), pp.114 -118, Sept. 2016.

X. Xu, A. Özçelikkale, T. McKelvey, and M. Viberg, “Simultaneous Information and Power Transfer under a Non-Linear RF Energy Harvesting Model", 2017 IEEE International Conference on Communications (ICC) -Workshop on Emerging Energy Harvesting Solutions for 5G Networks, 2017.

R. Du, A. Özçelikkale, C. Fischione, and M. Xiao, “Towards Immortal Wireless Sensor Networks by Optimal Energy Beamforming and Data Routing”, IEEE Transactions on Wireless Communications, vol. 17, pp. 5338–5352, Aug. 2018.

SAMPLING RATE AND ACCURACY TRADE-OFFS IN SIGNAL RECOVERY

In contrast to common practice which often treats sampling and quantization separately, we explicitly focus on the interplay between limited spatial resolution and limited amplitude accuracy. We show that in certain cases, sampling at rates different than the Nyquist rate is more efficient when limited accuracy of the samples are also taken into account. We find the optimal sampling rates, and the resulting trade-off curves between the error and the accuracy of the measurements.

Publications:

A. Özçelikkale and H. M. Ozaktas, “Beyond Nyquist Sampling: A Cost-Based Approach,” J. Opt. Soc. Am. A, vol.30, no.4, pp.645-655, Apr. 2013.

A. Özçelikkale and H. M. Ozaktas, “Optimal representation of non-stationary random fields with finite numbers of samples: A linear MMSE framework,” Digital Signal Process., vol.23, no.5, pp.1602-1609, Sep. 2013.

A. Özçelikkale and H. M. Ozaktas, “Representation of optical fields using finite numbers of bits,” Opt. Lett., vol. 37, pp. 2193–2195, June 2012.

A. Özçelikkale, H. M. Ozaktas, and E. Arıkan, “Signal recovery with cost constrained measurements,” IEEE Trans. Signal Process., vol. 58, no. 7, pp. 3607–3617, Jul. 2010.

TRELLIS-BASED APPROACHES FOR COMMUNICATIONS WITH SHORT LENGTH CODES

Delay intolerant applications, for instance various machine to machine communications scenarios require communication with short-length frames. On the other hand, traditional code design approaches focus on the asymptotic regime where the frame length is assumed to very large. Hence these approaches have very limited applicability in the short length communication scenarios. To address this issue, we focus on the trellis-based codes where systematic design and performance evaluation can be done for short block lengths.

Publications:

A. Özçelikkale, and T. M. Duman, “Lower Bounds on the Error Probability of Turbo Codes”, Proc. 2014 IEEE Int. Symp. Information Theory, pp. 3170-3174, June 2014.

A. Özçelikkale, and T. M. Duman, “Short Length Trellis-Based Codes for Gaussian Multiple-Access Channels”, IEEE Signal Process. Lett., vol. 21, no. 10, pp. 1177-1181, Oct. 2014.