Understanding Auto regression in machine learning

Introduction

Laymen explanation

Classical models such as autoregression (AR) exploit the inherent characteristics of a time series, leading to a more concise model. This model impact future values based on past values and so it is an important approach for forecasting. If you are interested to understand this approach, then this document help.

Technical explanation

In an autoregression model, we forecast the variable of interest using a linear combination of past values of the variable. The term autoregression indicates that it is a regression of the variable against itself.

Thus, an autoregressive model of order p can be written as

Properties of auto-regression model

In an AR process, a one-time shock affects values of the evolving variable infinitely far into the future. For example, consider the AR(1) model . A non-zero value for at say time t=1 affects

by the amount . Then by the AR equation for in terms of , this affects by the amount . Then by the AR equation for in terms of , this affects by the amount

. Continuing this process shows that the effect of never ends,

AR in Neural Network

Left model represents AR-equivalent neural network without hidden layers (simplest form of AR-Net).

Right model represents AR-inspired neural network with n hidden layers (general AR-Net).

Reference

https://en.wikipedia.org/wiki/Autoregressive_model

https://ai.facebook.com/blog/ar-net-a-simple-autoregressive-neural-network-for-time-series/

https://images.app.goo.gl/wpL9hGdQSc4gm7wz8