The Big Book Of Stock Trading Strategies

The purpose of stock market investment is to obtain more profits. In recent years, an increasing number of researchers have tried to implement stock trading based on machine learning. Facing the complex stock market, how to obtain effective information from multisource data and implement dynamic trading strategies is difficult. To solve these problems, this study proposes a new deep reinforcement learning model to implement stock trading, analyzes the stock market through stock data, technical indicators and candlestick charts, and learns dynamic trading strategies. Fusing the features of different data sources extracted by the deep neural network as the state of the stock market, the agent in reinforcement learning makes trading decisions on this basis. Experiments on the Chinese stock market dataset and the S&P 500 stock market dataset show that our trading strategy can obtain higher profits compared with other trading strategies.

At present, an increasing number of studies implement dynamic trading strategies based on deep reinforcement learning. Reinforcement learning gains increasing attention after AlphaZero defeated humans [7], has the ability of independent learning and decision-making, and has been successfully applied in the field of game playing [8, 9], unmanned driving [10, 11], and helicopter control [12]. Reinforcement learning solves the sequential decision-making problem, which can be applied to stock trading to learn dynamic trading strategies. Nevertheless, reinforcement learning lacks the ability to perceive the environment. The combination of deep learning and reinforcement learning (i.e., deep reinforcement learning) solves this problem and has more advantages when it has the decision-making ability of reinforcement learning and perception ability of deep learning.

Download Zip 🔥 https://urluso.com/2xYdbK 🔥

To a deeper analysis of the stock market and learn the optimal dynamic trading strategy, this study proposes a deep reinforcement learning model and integrates multisource data to implement stock trading. Through the analysis of stock data, technical indicators, and candlestick charts, we obtain a deeper feature representation of the stock market, which is conducive to learning the optimal trading strategy. Besides, the setting of the reward function in reinforcement learning cannot be ignored. In stock trading, investment risk should be paid attention to while considering returns and reasonably balance risk and returns. Sharpe ratio (SR) represents the profit that can be obtained under certain risks [23]. In this study, the reward function takes investment risk into consideration and combines SR and profit rate (PR) as the reward function to promote the learning of optimal trading strategies.

To verify the effectiveness of the trading strategy learned by our proposed model, we compare it with other trading strategies based on practical trading data. For stocks with different trends, our trading strategy obtained higher PR and SR, which has better robustness. In addition, we conduct ablation experiments, and the experimental results show that the trading strategy learned from analyzing the stock market based on multisource data is better than those learned from analyzing the stock market based on a single data source. The main contributions of this paper are as follows:(i)A new deep reinforcement learning model is proposed to implement stock trading and integrate the stock data and candlestick charts to analyze the stock market, which is more helpful to learn the optimal dynamic trading strategy.(ii)A new reward function is proposed. In this study, investment risk is taken into account, and the sum of SR and PR is taken as the reward function.(iii)The experimental results show that the trading strategy learned from the deep reinforcement learning model proposed in this paper can obtain better profits for stocks with different trends.

In recent years, a mass of machine learning methods has been implemented in stock trading. Investors make trading decisions based on their judgment of the stock market. However, due to the influence of many factors, they cannot make correct trading decisions based on the changes in the stock market in time. Compared with traditional trading strategies, machine learning methods can learn trading strategies by analyzing information related to the stock market and discovering profit patterns that people do not know about without professional financial knowledge, which have more advantages.

Reinforcement learning can be used to implement stock trading by self-learning and autonomous decision-making. Chakole et al. [31] used Q-learning algorithm [32] to find the optimal trading strategy, in which the unsupervised learning method K-means and candlestick chart were, respectively, used to represent the state of the stock market. Deng et al. [33] proposed a model Deep Direct Reinforcement Learning and added fuzzy learning, which is the first attempt to combine deep learning and reinforcement learning in the field of financial transactions. Wu et al. [34] proposed a long short-term memory based (LSTM-based) agent that could perceive stock market conditions and automatically trade by analyzing stock data and technical indicators. Lei et al. [35] proposed a time-driven feature aware jointly deep reinforcement learning model (TFJ-DRL), which combines gated recurrent unit (GRU) and policy gradient algorithm to implement stock trading. Lee et al. [36] proposed HW_LSTM_RL structure, which first used wavelet transforms to remove noise in stock data, then based on deep reinforcement learning to analyze the stock data to make trading decisions.

Existing studies on stock trading based on deep reinforcement learning mostly analyze the stock market through a single data source. In this study, we propose a new deep reinforcement learning model to implement stock trading, and analyze the state of the stock market through stock data, technical indicators, and candlestick charts. In our proposed model, firstly, different deep neural networks are used to extract the features of different data sources. Secondly, the features of different data sources are fused. Finally, reinforcement learning makes trading decisions according to the fused features and continuously optimizes trading strategies according to the profits. The setting of reward function in reinforcement learning cannot be ignored. In this study, the SR is added to the reward function setting, and the investment risk is taken into account while considering the profits.

We propose a new deep reinforcement learning model and implement stock trading by analyzing the stock market with multisource data. In this section, first, we introduce the overall deep reinforcement learning model, then the feature extraction process of different data sources is described in detail. Finally, the specific application of reinforcement learning in stock trading is introduced.

Implementing stock trading based on deep reinforcement learning and correctly analyzing the state of the stock market is more conducive to learning the optimal dynamic trading strategy. To obtain the deeper feature representation of the stock market state and learn the optimal dynamic trading strategy, we fuse the features of stock data, technical indicators, and candlestick charts. Figure 1 shows the overall structure of the model.

The deep reinforcement learning model we propose can be divided into two modules, the deep learning module for extracting features of different data sources and the reinforcement learning module for making trading decisions. Candlestick charts features are extracted by the CNN and bidirectional long short-term memory (BiLSTM); stock data and technical indicators are as the input of the LSTM network for feature extraction. After extracting the features of different data sources, contacting the features of the different data sources to implement feature fusion, the fused features can be regarded as the state of the stock market, and the reinforcement learning module makes trading decisions on this basis. In addition, in the reinforcement learning module, the algorithms used are Dueling DQN [37] and Double DQN [38].

The purpose of this study is to obtain a deeper feature representation of the stock market environmental state through the fusion of multisource data to learn the optimal dynamic trading strategy. Although raw stock data can reflect changes in the stock market, they contain considerable noise. To reduce the impact of noise and perceive the changes of the stock market more objectively and accurately, relevant technical indicators are used as one of the data sources for analyzing the stock market in this study. Candlestick charts can reflect the changes in the stock market from another perspective. This paper fuses the features of the candlestick charts.

Due the noise in stock data, we use relevant technical indicators to reduce the impact of noise. The technical indicators reflect the changes in the stock market from different perspectives. In this paper, stock data and technical indicators are used as inputs to the LSTM network to better capture the main trends of stocks. The raw stock data we use include opening price, closing price, high price, low price, and trading volume. The technical indicators used in this paper are the MACD, EMA, DIFF, DEA, KDJ, BIAS, RSI, and WILLR. The indicators are calculated by mathematical formulas based on stock prices and trading volumes [39], as reported in Table 1.

To facilitate subsequent calculations, we perform missing value processing on the stock data. First, the stock data is cleaned, and the missing data are supplemented with 0. In addition, the input of the neural network must be a real value, so we replaced the NaNs in the stock data and technical indicators with 0. Data with different value ranges may show gradient explosion during neural network training [42]. To prevent this problem, we normalize the stock data and technical indicators; normalization is performed to transform the data to a fixed interval. In this work, the stock data and technical indicators of each dimension are normalized, and the data are converted into ranges [0, 1]. The normalization formula is as follows:where represents the original data, and represent the minimum and maximum values of the original data, respectively, represents the normalized data. The neural network structure for extracting stock data and technical indicators is shown in Figure 2. LSTM network is a variant of recurrent neural network (RNN), and its unit structure is shown in Figure 3. LSTM solves the problem of gradient disappearance and gradient explosion in the long sequence training process. In the LSTM network, f, i, and represent a forget gate, an input gate, and an output gate, respectively. A forget gate is responsible for removing information from the cell state. The input gate is responsible for the addition of information to the cell state. The output gate decides which next hidden state should be selected. is the state of the memory cell at time t; is the value of the candidate state of the memory cell at time ; and tanh are the sigmoid and tanh activation functions, respectively; W and b represent the weight and deviation matrix, respectively; is the input vector; is the output vector; in this paper, is the data after the contacting of stock data and technical indicators, and other specific calculation formulas are as follows: be457b7860