Vector Auto Regression (VAR)
VAR is a multivariate time series model. It takes multiple time series variables and makes prediction for every variable.
e.g. if you have variable x1, x2 and target y, it actually considers the y as a variable as well.
So now there are 3 time series x1, x2, x3(i.e. y)
Each variable is modeled as a linear combination of past values ot itself and the other variables.
For example, considerring 2 step lagged values of 3 variables x1-x3, each of the variable has a linear regression formula as below:
x1(t) = a1 + b1,11 * x1(t-1) + b1,12 * x1(t-2) + b1,21 * x2(t-1) + b1,22 * x2(t-2) + b1,31 * x3(t-1) + b1,32 * x3(t-2) + e1
x2(t) = a2 + b2,11 * x1(t-1) + b2,12 * x1(t-2) + b2,21 * x2(t-1) + b2,22 * x2(t-2) + b2,31 * x3(t-1) + b2,32 * x3(t-2) + e2
x3(t) = a3 + b3,11 * x1(t-1) + b3,12 * x1(t-2) + b3,21 * x2(t-1) + b3,22 * x2(t-2) + b3,31 * x3(t-1) + b3,32 * x3(t-2) + e3
Here a1-3 are intercepts, e1-3 are errors or noises, the bi,jk is parameter of a lagged variable value.
Each variable's lagged values are contributing to a variable's predicted value in a linear combination way.
Because every variable is contributing to the prediction of every variable, the prediction and variables impact each other.
The VAR requires the time series to be stationary so make sure taking differencing and other mechanisms to make them stationary first.
Granger's Causality Test
There is a causation test (Granger's Causality Test) to check if one timeseries is useful for forecasting another time series, ie.influence check. What it does is simply testing the null hypothesis that the coefficients of lagged values in the formula above is zero, then calculates the significance of changes with and without the variable (coefficient equals / not equals 0), if the p-value of is less than the significance level 0.05 then the removing the variable causes significant change, and the variable is influential. By the Granger's Causality test, it knows what variables to be used in the formula. E.g. X1 might be only influenced by X2 without X3. In python, it can use statsmodels.tsa.stattools.grangercausalitytests for the test.
Cointegration Test
This is another test to check if two time series have a significant connection.
Firstly, 'Order of integration' means the number of differencing required to make a time series stationary. If you have two or more time series, and there is a linear combination of the two series, which has a smaller 'order of integration' than that of the individual series, i.e. it takes less differencing to become stationary for the linear combination, then the combination of the two series is said to be 'cointegrated'. When two series are cointegrated, they have a long run and statistically significant relationship. In python, it can use statsmodels.tsa.vector_ar.vecm.coint_johansen for the test.
Augmented Dickey-Fuller Test (ADF Test)
This is for testing if the series is stationary, if not, take differencing to achieve stationary. But in fact, it may never achieve real stationary.
Lag Order
How many lagged steps to use?
A statistical way is to fit the VAR model with lag = 1,2,3 ... and check the AIC (Akeike information)
Choose the lag order with the lowest AIC before it increses.
Always check if the residusls of the timeseries still have correlation to the prediction target. If yes, there is still some pattern or information can be extracted further from the residuals.