Other Automatic ARIMA modeling softwares:
Some of the softwares that can be invoked for automatic ARIMA modeling are GRETL, TRAMO/SEATS, X-12 ARIMA and AUTOBOX. etc.
RATIONALE FOR AAM USING EXPERT SYSTEM (TSE-AX):
Following factors motivated the development of ARIMA forecasting software using expert system:
Firstly, the most widely used “Box and Jenkins” manual forecasting methodology was too intricate and required expertise for producing good forecast.
Secondly, forecasting involving large number of data series were too difficult to handle manually.
Finally, even most of the softwares used up till then for forecasting suffered from the disadvantage that these would neither convey the logics and reasoning used or intermediate results to the users nor did they warrant the quality of final model chosen for forecasting leading them to be termed as “black boxes”.
In contrast to these shortcomings of previously used methods, TSE-AX software has the following favorable characteristics lending it an edge as a forecasting method:
It keeps the user informed about the steps followed, intermediate results and final results.
The user can alter or change the model according to the information available to him.
It can be adjusted according to the expertise of the user.
It regards only the most “substantive models” for lag structures.
It works efficiently in case of large number of time series data.
An additional and very significant step in building an ARIMA model in TSE-AX is intervention analysis for appropriate treatment of “outliers”
METHODOLOGY (for TSE 2.3 version)
AAM1 and AAM2: The only difference between the methodologies of AAM1 and AAM2 is an additional step of intervention analysis in the latter with the view to spot the outliers in the data and treat those appropriately in order to increase the accuracy of results.
Unautomated Steps:
The user has to first attain know-how as to the properties of the time series to be utilized for forecasting purpose without the help of the system. That is, the information regarding the periodicity and the chosen sample are needed to be given to the system by the user. Therefore, series can be modified or altered to suit the purpose e.g. to eliminate some parts of the series etc, by the user.
Outlier Detection: operation is performed after differencing and transformation of the series and before specification of the model. The test is carried out in order to find out if there are extreme values close to the forecasting origin which suggest problems for extrapolative forecasting. For example if there are T values in a time series then the test would be used to discover the outliers at the observation no T-1, T-2 and T-s where ‘s’ represents seasonal periodicity. Contextual information also called as ‘domain knowledge’ can prove extremely helpful in picking out these outliers resulting from or implying some shocks thereby bringing remarkable improvement in forecasts.
Automated Steps
PRELIMINARIES
The “Selection Order” differs from “Application Order” of automated steps. That is, selection order would be:
To choose a difference to be applied to the series
Secondly, to perform the “Box-Cox” transformation”.
These two steps are carried out to attain the stationary version of the relevant series.
Then, to select the intervention to be applied.
While application sequence goes as:
First to perform intervention, then transformation and finally differences.
a) Choice of a difference and a Seasonal difference:
The ‘autocorrelation function (CACF)’ and ‘partial autocorrelation function (CPACF)’ are used as measures of over differencing while ‘Kruskal and Wallis test’ (KW) is the recommended criteria for seasonality.
WHAT IS OVERDIFFERENCING:
If the lag one autocorrelation coefficient of backward difference series comes out to be greater than that of the lag one autocorrelation coefficient of the original series, then such results signals the problem of ‘over differencing’.
Heuristics used for the choice of differences:
Following steps are followed and results displayed accordingly, to analyze the series for the choice of differences: (Melards &Pasteels 1998 for formulas)
Heuristic#1 for “Seasonal Differences”: Invokes KW test and CACF and requires a positive seasonality test and no evidence of over differencing as a pre-requisite for using ‘seasonal differences’.
Heuristic#2 for “Non-seasonal Differences”: That is to select the series from among the original series (yt), the backward difference of the original series and second order backward differences of the original series(n2>yt) the one for which there is no evidence of over differencing.
Heuristic#3 for “Smooth (artificial) Series”: If the series is smooth giving an indication of its being an artificial series, then its DGP is figured out and conveyed to the user.
Heuristic#4: This finds out whether the recommendations for the choice of differences of CACF criteria and CPACF criteria coincide or there is a difference between the two.
Other Information Displayed:
The number of the lags of each of the series understudy, for which autocorrelation (rk) and partial autocorrelation (pk) are significantly different from zero.
The number of lags for which no explanation can be given or which cannot be interpreted with reference to the dynamics of the model.
b) Choice of a transformation:
The series understudy is analyzed for variance stationarity by using ‘tau test’ of rank correlation coefficients. The coefficients are computed among ‘time’ and ‘scale parameter’ of the aptly differenced time series called as ‘residual time series’. The scale parameter is defined as the interquartile range i.e. the difference between the third and the first quartiles and is computed for each year. If the null hypothesis of tau=0 is rejected then the square root transformations are applied to the series and tau coefficient is computed again while in case of significant rank correlation coefficient second time logarithmic transformations are applied. The level of significance can be chosen by the user.
This method of series transformation is immune to outliers.
c) Intervention Analysis:
Intervention analysis refers to analyzing the impact of external events, shocks or structural change in time series data also termed as “event study”. In other words, the results of outliers in the time series data whose timings are known are evaluated through intervention analysis.
Various forms of intervention analysis performed in the computer program named “ANSECH’ used to build up the expert system are:
ü Pulse Interventions or ‘additive outlier’ (AO):
If the affects of the shock manifested as ‘outlier’ is realized or existed only for one period then, pulse intervention variable is used to model their impact.
While the outliers whose time of occurrence are unknown and its impacts occur only for one time period, such outliers are detected and removed via ‘additive outlier’ models.
ü Level shift (LS):
Level shift outlier model is added to the process of outlier detection in case of permanent shift in the mean function due to the outlier.
ü Compensation:
Compensation filters are used to remove corner outliers typically applied in case of image compression.
ü Ramp:
Ramp refers to the analysis of impact of an intervention which linearly alters the mean function of the post-intervention time series.
ü Temporary change (TC):
When the outlier brings a temporary shift in mean function then temporary change outlier model is incorporated in the outlier detection procedure.
While another one called as ‘Innovation Outliers” is not used in the specific computer program used.
SPECIFICATION
a) Autoregressive Specification Procedure:
It is a single step and two step process for non-seasonal and seasonal models respectively.
STEPS:
First stage comprises fitting of 4 models in case of non-seasonal models while eight models are to fitted for seasonal models i.e.
Models Fitted In Case Of Both Seasonal & Non-Seasonal Models:
white noise
AR (1) [first order autoregressive model]
AR(2) [Second order autoregressive model]
AR(3) [Third order autoregressive model]
Additional Models Fitted In Case Of Seasonal Models:
SAR(1) [First order seasonal autoregressive model]
AR(1) and SAR(1) [Multiplicative autoregressive model with both AR and SAR polynomials of degree 1]
AR(2) and SAR(1) [Multiplicative autoregressive model with AR polynomial of degree 2 and SAR polynomial of degree 1 of order n*s where s=1= seasonal period]
AR(3) and SAR(1) [Multiplicative autoregressive model with AR and SAR polynomials of degree 3 and 1 respectively of order n*s
where s=1= seasonal period]
At second stage autocorrelations of the residuals of each model are examined. Among all the four models in non-seasonal case and eight models for seasonal analysis, if some model is available which has no significant lag autocorrelation of residuals then that model will be selected and the specification procedure terminates afterwards. Otherwise, the model with smallest number of autocorrelation lag cut off will be selected. For example ‘k’ is the largest lag with significant autocorrelation and ‘K’ is the largest lag for which some interpretation is possible then if ‘k<K,’ moving average polynomial of the selected model will be of degree ‘k’ of order 1. Moreover, a SMA(1) term is also incorporated into the model in the event of significant autocorrelation for lags s,2s, 3s…. ks.On the other hand, if ‘K<k’, then “mixed strategy” is the proposed method for model specification.
b) Mixed Strategy:
The mixed strategy is composed of three steps:
STEPS:
In the first step again four and eight autoregressive models are fitted for non-seasonal and seasonal case respectively.
In the second step moving average terms for all the models being fitted are decided instead of focusing on only the model exhibiting smallest lag number of significant residual autocorrelation. Criteria for the choice of plausible models among all the fitted model in case of mixed strategy is ‘Ljung-Box Q statistic’ of order m symbolized as ‘Qm’, where m is a function of n which represents the length of the time series.
Third step makes up the final model selection being done on the basis of ‘Bayesian Information Criterion’ (BIC). However, if no model comes out to be good on the BIC scale then the model with minimum Qm statistic is selected.
ADEQUACY AND MODEL CHECKING
Model Checking:
Firstly, the ‘convergence of the optimization process of the likelihood function’ is analyzed.
Secondly, ‘stationarity’ and ‘invertibility’ of the roots of AR and MA polynomials are checked.
In case of convergence of likelihood function second condition is always satisfied in the ANSECH program because these are the restrictions imposed for model fitting.
Testing Of Residuals:
Randomness of the data to which the sample understudy belongs is checked through ‘Ljung and Box statistic Qm’ where m represents lags e.g. 6, 12, 18 … for monthly data.
Autocorrelation (rk) and partial autocorrelations (pk) of the residuals are also tested individually by means of ‘Bartlett test’.
Outliers among the residual series are picked out.
CRITERIA FOR MODEL ADEQUACY:
Ljung-Box test should not reject the null hypothesis of randomness of the data at lags m=6, 12, 18…. upto the maximum <=n/5 (for monthly data) where n= total number of data values, at 5% level of significance.
Maximum of three significant autocorrelation and partial autocorrelation coefficients are compatible with the idea of adequacy of the model being analyzed.
NEW SPECIFICATION
In case an adequate good fit model for the data is not figured out through the two stages “autoregressive specification procedure’ or ‘three stage mixed strategy’ then the TSE-AX does not go further to fit complex models involving higher order polynomials. Moreover, the final interpretations and qualifications are left to the user of the program.
APPENDIX:
LJUNG-BOX TEST: checks all the autocorrelations taken together of the sample understudy for equal to zero. It's null hypothesis is :
H0 : the data is random OR the correlations in the population from which the sample has been drawn are zero (implying that any positive correlation in the sample studied is a random event)
BARTLETT TEST: test the hypothesis that sample autocorrelation coefficients are statistically different from zero. Since sample understudy is just a subset of a population it is highly probable that its correlation coefficients come out to be significantly different from zero which might convey wrong signals about fit of the ARIMA model. Therefore to overcome this problem the variance of the sample autocorrelations are analyzed on the basis of ‘Bartlett Theorem’ which says that if the residuals are generated by a random process i.e. white noise process then its sample autocorrelations are normally distributed with mean zero and variance 1/T or standard deviation =1/√T where t denotes the size of the data series studied.
References:
Ord, Keith and Lowe, Sam (1996) “Automatic Forecasting”, The American Statistician, Vol. 50, No. 1 (Feb., 1996), pp. 88-94
Melard, G and Pasteels, J.-M. (2000) “Automatic ARIMA modeling including interventions, using time series expert software”, International Journal of Forecasting, Vol. 16, No. 4
Wei,William W.S (2006), Time Series Analysis. Univariate and Multivariate Methods. Second Edition. Pearson Education
TSE 2.2 version can be downloaded from here.
One of the methods of forecasting from ARIMA models utilized in M-3 competition was referred to as “Time Series Expert Software abbreviated as TSE-AX” based on “EXPERT SYSTEM TECHNOLOGY”
AUTOMATIC ARIMA MODELING WITH & WITHOUT INTERVENTION ANALYSIS USING TSE-AX SOFTWARE
INTRODUCTION:
AAM stands for “Automatic ARIMA modeling” while AAM1 and AAM2 are the variants of AAM denoting:
Automatic ARIMA modeling disregarding outliers, and
Automatic ARIMA modeling with intervention analysis, respectively.
termed as “AAM strategies”
Automatic forecasting entails making forecasts for future values for a given time series with the aid of a software package without much interference on the part of the user. (Ord, Keith; Lowe, Sam 1996)