## IntroductionTime series provide the opportunity to forecast future values. Based on previous values, time series can be used to forecast trends in economics, weather, and capacity planning, to name a few. The specific properties of time-series data mean that specialized statistical methods are usually required. In this tutorial, we will aim to produce reliable forecasts of time series. We will begin by introducing and discussing the concepts of autocorrelation, stationarity, and seasonality, and proceed to apply one of the most commonly used method for time-series forecasting, known as ARIMA. One of the methods available in Python to model and predict future points of a time series is known as SARIMAX, which stands for Seasonal AutoRegressive Integrated Moving Averages with eXogenous regressors. Here, we will primarily focus on the ARIMA component, which is used to fit time-series data to better understand and forecast future points in the time series. ## PrerequisitesThis guide will cover how to do time-series analysis on either a local desktop or a remote server. Working with large datasets can be memory intensive, so in either case, the computer will need at least 2GB of memory to perform some of the calculations in this guide. To make the most of this tutorial, some familiarity with time series and statistics can be helpful. For this tutorial, we’ll be using Jupyter Notebook to work with the data. If you do not have it already, you should follow our tutorial to install and set up Jupyter Notebook for Python 3. ## Step 1 — Installing PackagesTo set up our environment for time-series forecasting, let’s first move into our local programming environment or server-based programming environment:
From here, let’s create a new directory for our project. We will call it
This tutorial will require the Like with other Python packages, we can install these requirements with
At this point, we're now set up to start working with the installed packages. ## Step 2 — Importing Packages and Loading DataTo begin working with our data, we will start up Jupyter Notebook:
To create a new notebook file, select New > Python 3 from the top right pull-down menu: This will open a notebook. As is best practice, start by importing the libraries you will need at the top of your notebook:
We have also defined a We'll be working with a dataset called "Atmospheric CO2 from Continuous Air Samples at Mauna Loa Observatory, Hawaii, U.S.A.," which collected CO2 samples from March 1958 to December 2001. We can bring in this data as follows:
Let's preprocess our data a little bit before moving forward. Weekly data can be tricky to work with since it’s a briefer amount of time, so let's use monthly averages instead. We’ll make the conversion with the
Let's explore this time series e as a data visualization:
Some distinguishable patterns appear when we plot the data. The time series has an obvious seasonality pattern, as well as an overall increasing trend. To learn more about time series pre-processing, please refer to "A Guide to Time Series Visualization with Python 3," where the steps above are described in much more detail. Now that we've converted and explored our data, let's move on to time series forecasting with ARIMA. ## Step 3 — The ARIMA Time Series ModelOne of the most common methods used in time series forecasting is known as the ARIMA model, which stands for AutoregRessive Integrated Moving Average. ARIMA is a model that can be fitted to time series data in order to better understand or predict future points in the series. There are three distinct integers ( `p` is the*auto-regressive*part of the model. It allows us to incorporate the effect of past values into our model. Intuitively, this would be similar to stating that it is likely to be warm tomorrow if it has been warm the past 3 days.`d` is the*integrated*part of the model. This includes terms in the model that incorporate the amount of differencing (i.e. the number of past time points to subtract from the current value) to apply to the time series. Intuitively, this would be similar to stating that it is likely to be same temperature tomorrow if the difference in temperature in the last three days has been very small.`q` is the*moving average*part of the model. This allows us to set the error of our model as a linear combination of the error values observed at previous time points in the past.
When dealing with seasonal effects, we make use of the The seasonal ARIMA method can appear daunting because of the multiple tuning parameters involved. In the next section, we will describe how to automate the process of identifying the optimal set of parameters for the seasonal ARIMA time series model. ## Step 4 — Parameter Selection for the ARIMA Time Series ModelWhen looking to fit time series data with a seasonal ARIMA model, our first goal is to find the values of We will use a "grid search" to iteratively explore different combinations of parameters. For each combination of parameters, we fit a new seasonal ARIMA model with the
We can now use the triplets of parameters defined above to automate the process of training and evaluating ARIMA models on different combinations. In Statistics and Machine Learning, this process is known as grid search (or hyperparameter optimization) for model selection. When evaluating and comparing statistical models fitted with different parameters, each can be ranked against one another based on how well it fits the data or its ability to accurately predict future data points. We will use the The code chunk below iterates through combinations of parameters and uses the
Because some parameter combinations may lead to numerical misspecifications, we explicitly disabled warning messages in order to avoid an overload of warning messages. These misspecifications can also lead to errors and throw an exception, so we make sure to catch these exceptions and ignore the parameter combinations that cause these issues. The code above should yield the following results, this may take some time:
The output of our code suggests that ## Step 5 — Fitting an ARIMA Time Series ModelUsing grid search, we have identified the set of parameters that produces the best fitting model to our time series data. We can proceed to analyze this particular model in more depth. We'll start by plugging the optimal parameter values into a new
The When fitting seasonal ARIMA models (and any other models for that matter), it is important to run model diagnostics to ensure that none of the assumptions made by the model have been violated. The
Our primary concern is to ensure that the residuals of our model are uncorrelated and normally distributed with zero-mean. If the seasonal ARIMA model does not satisfy these properties, it is a good indication that it can be further improved. In this case, our model diagnostics suggests that the model residuals are normally distributed based on the following: -
In the top right plot, we see that the red `KDE` line follows closely with the`N(0,1)` line (where`N(0,1)` ) is the standard notation for a normal distribution with mean`0` and standard deviation of`1` ). This is a good indication that the residuals are normally distributed. -
The qq-plot on the bottom left shows that the ordered distribution of residuals (blue dots) follows the linear trend of the samples taken from a standard normal distribution with `N(0, 1)` . Again, this is a strong indication that the residuals are normally distributed. -
The residuals over time (top left plot) don't display any obvious seasonality and appear to be white noise. This is confirmed by the autocorrelation (i.e. correlogram) plot on the bottom right, which shows that the time series residuals have low correlation with lagged versions of itself.
Those observations lead us to conclude that our model produces a satisfactory fit that could help us understand our time series data and forecast future values. Although we have a satisfactory fit, some parameters of our seasonal ARIMA model could be changed to improve our model fit. For example, our grid search only considered a restricted set of parameter combinations, so we may find better models if we widened the grid search. ## Step 6 — Validating ForecastsWe have obtained a model for our time series that can now be used to produce forecasts. We start by comparing predicted values to real values of the time series, which will help us understand the accuracy of our forecasts. The
The code above requires the forecasts to start at January 1998. The We can plot the real and forecasted values of the CO2 time series to assess how well we did. Notice how we zoomed in on the end of the time series by slicing the date index.
Overall, our forecasts align with the true values very well, showing an overall increase trend. It is also useful to quantify the accuracy of our forecasts. We will use the MSE (Mean Squared Error), which summarizes the average error of our forecasts. For each predicted value, we compute its distance to the true value and square the result. The results need to be squared so that positive/negative differences do not cancel each other out when we compute the overall mean.
The MSE of our one-step ahead forecasts yields a value of However, a better representation of our true predictive power can be obtained using dynamic forecasts. In this case, we only use information from the time series up to a certain point, and after that, forecasts are generated using values from previous forecasted time points. In the code chunk below, we specify to start computing the dynamic forecasts and confidence intervals from January 1998 onwards.
Plotting the observed and forecasted values of the time series, we see that the overall forecasts are accurate even when using dynamic forecasts. All forecasted values (red line) match pretty closely to the ground truth (blue line), and are well within the confidence intervals of our forecast.
Once again, we quantify the predictive performance of our forecasts by computing the MSE:
The predicted values obtained from the dynamic forecasts yield an MSE of 1.01. This is slightly higher than the one-step ahead, which is to be expected given that we are relying on less historical data from the time series. Both the one-step ahead and dynamic forecasts confirm that this time series model is valid. However, much of the interest around time series forecasting is the ability to forecast future values way ahead in time. ## Step 7 — Producing and Visualizing ForecastsIn the final step of this tutorial, we describe how to leverage our seasonal ARIMA time series model to forecast future values. The
We can use the output of this code to plot the time series and forecasts of its future values.
Both the forecasts and associated confidence interval that we have generated can now be used to further understand the time series and foresee what to expect. Our forecasts show that the time series is expected to continue increasing at a steady pace. As we forecast further out into the future, it is natural for us to become less confident in our values. This is reflected by the confidence intervals generated by our model, which grow larger as we move further out into the future. ## ConclusionIn this tutorial, we described how to implement a seasonal ARIMA model in Python. We made extensive use of the Here are a few other things you could try: - Change the start date of your dynamic forecasts to see how this affects the overall quality of your forecasts.
- Try more combinations of parameters to see if you can improve the goodness-of-fit of your model.
- Select a different metric to select the best model. For example, we used the
`AIC` measure to find the best model, but you could seek to optimize the out-of-sample mean square error instead.
For more practice, you could also try to load another time series dataset to produce your own forecasts. |