An optimized LSTM network for improving arbitrage spread forecasting using ant colony cross-searching in the K-fold hyperparameter space

Related work

Hyperparameter tuning is the process of finding the optimal combination of hyperparameters that maximizes the output accuracy. There are several methods for hyperparameter tuning in artificial neural network (ANN), such as grid search (Bergstra & Bengio, 2012), Bayesian optimization (Snoek, Larochelle & Adams, 2012), gradient-based optimization (Maclaurin, Duvenaud & Adams, 2015), or random search based on metaheuristic algorithms (MAs).

Traditional grid search is a common method used for hyperparameter tuning in the development of time series forecasting models. However, it can pose some challenges. If the grid points are too sparse, the optimal search outcome may deviate significantly from the true optimal solution. Conversely, a densely populated grid could significantly increase the computational demands of the experiment. Therefore, selecting the appropriate grid granularity and extent is critical to maintaining a balance between grid density and computational load. However, individual subjective factors often influence the process of achieving this balance.

For the deficiencies of traditional grid search, population-based MAs have been widely used for combinatorial optimization problems. These algorithms generate optimal solutions by exploring a new region in the search space via an iterative process. Grey Wolf Optimizer (Mirjalili, Mirjalili & Lewis, 2014), Particle Swarm Optimization (Kennedy & Eberhart, 1995), Cuckoo Search Algorithm (Gandomi, Yang & Alavi, 2013), and Sparrow Search Algorithm (Xue & Shen, 2020) are well-known examples of this class of MAs. The Ant Colony Optimization (ACO) is a population-based metaheuristic algorithm proposed by Dorigo, Maniezzo & Colorni (1996). It emulates the behavior of ant colonies as they seek the shortest path during their food-foraging process and has parallel computing, strong robustness, and easy combination with other algorithms. To better handle complex optimization problems, the ACO algorithm has been enhanced in three main areas. Firstly, the search, update, and coordination mechanisms have been improved in the feedback mechanism of pheromones. Engin & Güçlü (2018) proposed a crossover and mutation mechanism to solve shop scheduling. Zhao, Zhang & Zhang (2020) presented an adaptive reference point mechanism to enhance the optimization ability. Secondly, to address the shortcomings of the algorithms themselves, a hybrid ACO with other algorithms was proposed. Wang & Han (2021) presented a hybrid algorithm with symbiotic organism search, ACO, and local optimization strategy. Finally, some parameters of the ACO algorithm were adaptively tuned. Yang et al. (2016) developed an adaptive parameter adjustment by taking the differences among niches into consideration, and a differential evolution mutation operator for ants.

Neural network hyperparameter tuning can be considered a black-box optimization problem. When applying ACO to black-box optimization problems, the difficulty in evaluating the search space makes the selection of the population size of the ACO algorithm a challenge. Additionally, the revised method of continuously exploring new areas encounters challenges in determining the appropriate number of iterations due to the heightened randomness of the global search. This article introduces a novel method of hyperparameter tuning based on the principles of the traditional ant colony algorithm. It allows for the automatic adjustment of hyperparameters in ANN models across a wide range of options. In our algorithm, the basic unit of the search space is an interval block instead of a coordinate point. We limit the total number of basic units in the searchable space to ensure that the difficulty of the search process does not outweigh the capabilities of the pheromone search mechanism. During the iterative process, each search unit eliminates unpromising regions using the worst elimination rule. The size of the interval block decreases as the search progresses until it reaches a critical point, indicating the end of the search. The critical point here is determined by the precision of the hyperparameters. If the size of the region eliminated in each interval block update is fixed, the number of iterations also needs to increase. Therefore, the demand for the number of populations in the original algorithm has been replaced with a demand for the number of iterations. Our approach utilizes the reduction process of the interval block as a reference, making it easier to select the number of iterations. More details about the search space and search rules will be elaborated in ‘Search space’ and ‘Search rules’.

Search space

The KCS algorithm uses a transformation mechanism to convert the search space into a search environment where interval blocks serve as the basic unit. Since the size of K and the number of hyperparameters m are fixed values, the original search space of any size will be transformed into a new search space of a finite size in each iteration. The performance of the search algorithm is only affected by the distribution of the new space. Here the new search environment will be explored by ACO algorithm. Unpromising regions are excluded, and the remaining regions will again generate a new search environment. As the iterative process proceeds, the size of the basic unit of the search environment will continue to shrink and the average fitness level of the entire search environment will be optimised.

The Split-Range1 algorithm and Split-Range2 algorithm will perform the conversion process. Split-Range1 will handle initialization and spatial updates under the Optimal Selection Rule, while Split-Range2 will handle spatial updates under the Worst Elimination Rule. The basic search unit in the new search environment is denoted as $B=\left\{L_{1j_1},L_{2j_2},\dots ,L_{m{j}_m}|j_i\in 1,2,\dots ,K\right\}$ and referred to as an interval block. The Split-Range1(2) algorithm divides the range of values L_i(i = 1, 2, …, m) of each hyperparameter into K parts, creating m × K subintervals, which make up the K^m basic units of the new search environment.

The conversion process of the Split-Range1(2) algorithm is shown in Algorithm 1 .

Search rules

Assuming that the initial pheromone concentration of each subinterval is τ(0), and the number of ants searching the interval blocks is n. The optimization process in each iteration primarily relies on the principles of the ant colony algorithm and involves the following three types of updates:

(1) Update of the pheromone matrix

The pheromone matrix serves as a comprehensive record of search information across the entire search space. It is influenced by both positive and negative feedback mechanisms of pheromones.

In the case of positive feedback, whenever an ant completes a search of an interval block, it releases a corresponding amount of pheromones according to Eq. (3). The amount of pheromones released reflects the quality of the ants’ search results. (3)${\displaystyle \Delta {\tau }_{ij}=\sum _{k=1}^n\frac{Q\cdot I{S}_{ij}\left(k\right)}{Fi{t}_k},\left(k=1,2,\dots ,n\right)}$ where Δτ_ij represents the pheromone increment of the j- th subinterval for the i- th hyperparameter, the constant Q is the adjustment factor of the pheromone increment, and Fit_k is the quantified indicator of the search results of the k- th ant. When L_ij is a part of the interval block searched by the k th ant, $I{S}_{ij}\left(k\right)=1$, otherwise $I{S}_{ij}\left(k\right)=0$.

In the case of negative feedback, the pheromones on each subinterval will evaporate over time.

The specific update formula for the pheromone matrix is as follows: (4)${\displaystyle {\tau }_{ij}\left(t+1\right)=\left(1-\rho \right){\tau }_{ij}\left(t\right)+\Delta {\tau }_{ij},\left(0<\rho <1\right)}$ where ρ is the evaporation factor, and ${\tau }_{ij}\left(t\right)$ is the pheromone concentration on L_ij during the t- th iteration.

(2) Update of the selection probability matrix

The selection probability matrix is updated along with the pheromone matrix, and the calculation formula is as follows: (5)${\displaystyle p_{ij}\left(t\right)=\frac{{\left({\tau }_{ij}\left(t\right)\right)}^{\alpha }}{\sum _{j=1}^K{\left({\tau }_{ij}\left(t\right)\right)}^{\alpha }},\left(i=1,2,\dots ,m\right)}$ where $p_{ij}\left(t\right)$ signifies the likelihood of a subinterval being chosen in the t- th iteration. The sensitivity of the ants is modulated by $\alpha \left(\alpha \in N^{+}\right)$, known as the sensitivity factor. A larger value of the sensitivity factor indicates a heightened sensitivity of the ants to variations in pheromone concentration. Based on the selection probability matrix, we choose the interval block that each ant will search in the next iteration by roulette.

(3) Update of the search space

The update of the search space involves two mechanisms (as shown in Fig. 3): optimal selection and worst elimination.

Optimal selection rule: When the selection probability of a subinterval exceeds the threshold P_best, it will become the only available value range for the corresponding hyperparameter and forms a new search space with the value ranges of other hyperparameters.

Worst elimination rule: When the selection probability of a subinterval A is less than the threshold P_worst, if the value range of subinterval A is adjacent to the neighboring subinterval B, we retain a quarter of the interval length of A that is close to B, otherwise, discard subinterval A entirely. Note that the selection probability of A is the minimum of K probability values for a hyperparameter.

Based on the above search rules, the search process of the KCS algorithm is shown in Algorithm 3 .

LSTM network

Our LSTM network is depicted in Fig. 4. It comprises an LSTM layer and a fully connected (FC) network. The number of neurons in each layer serves as a hyperparameter and is determined through the KCS optimization algorithm. Each LSTM unit is built using the traditional ‘gate’ structure. Their internal mechanism is described by the following equations: (6)${\displaystyle \text{Input Gate}:I_t=sigmoid\left(w_i\cdot \left[d_t,h_{t-1}\right]+b_i\right)}$ (7)${\displaystyle \text{Output Gate}:O_t=sigmoid\left(w_o\cdot \left[d_t,h_{t-1}\right]+b_o\right)}$ (8)${\displaystyle \text{Forget Gate}:f_t=sigmoid\left(w_f\cdot \left[d_t,h_{t-1}\right]+b_f\right)}$ (9)${\displaystyle \text{Cell state}:C_t=f_t\cdot C_{t-1}+I_t\cdot tanh\left(w_c\cdot \left[d_t,h_{t-1}\right]+b_c\right)}$ (10)${\displaystyle h_t=O_t\cdot tanh\left(C_t\right)}$ where weight matrixes (w_i, w_o, w_f, w_c) and bias vectors (b_i, b_o, b_f, b_c) of I_t, O_t, f_t and C_t are obtained through a process of training on the designated training set. h_t represents the extracted feature of each LSTM unit and will be used as the input feature for the next time step. Note that only the h_t from the final time step is utilized as the extracted feature of the LSTM layer. This feature is subsequently input into a straightforward FC network and transformed into the predicted value (${\hat{y}}_t$).

Fitness function

In numerous models for hyperparameter optimization, the mean squared error (MSE) or mean absolute error (MAE) of the validation set is typically employed as the fitness function of the model. However, to mitigate the risk of model overfitting, we design a novel fitness function: (11)${\displaystyle Fit=\frac{1}{n-start}\sum _{i=start}^{n}Loss\text{\_}fu{n}_{val}^i\cdot \left|Loss\text{\_}fu{n}_{val}^i-Loss\text{\_}fu{n}_{train}^i\right|}$ where $Loss\text{\_}fu{n}_{val}^i$ is the loss function value of the model on the validation set after training for i epochs. The efficacy of the newly proposed fitness function will be substantiated through the experimental results presented in ‘Experiment’.

Steps of the KCS-LSTM

The flowchart of the KCS-optimized LSTM neural network is shown in Fig. 5. The main steps are as follows:

Step 1. Preprocessing. Preprocess the historical data, including completing the division of training set, validation set, and test set; eliminate the dimensional differences between data through normalization.

Step 2. Initialization. Set the number of populations, K value, number of algorithm iterations, etc. Then, initialize the search space and the pheromone matrix.

Step 3. Termination condition. The termination condition is defined as follows:

1. The process ends when the maximum number of iterations is reached.

2. It also ends when the early termination criteria are met. This occurs when the size of the subinterval is less than or equal to the precision of its corresponding hyperparameter.

At these points, the KCS optimization process concludes. The representative value of the interval block exhibiting the highest pheromone concentration is then returned as the final result of the hyperparameter optimization. Subsequently, the process advances to Step 9. If not, continue with the KCS optimization.

Step 4. Optimal selection. The optimal selection rule is implemented. The objects of conditional judgment do not include probability values where the size of the corresponding subinterval is less than or equal to the precision of the hyperparameter. Finally, it returns the updated search space, pheromone matrix, and selection probability matrix.

Step 5. Worst elimination. The worst elimination rule is implemented. The objects of conditional judgment do not include probability values where the size of the corresponding subinterval is less than or equal to the precision of the hyperparameter. Finally, it returns the updated search space, pheromone matrix, and selection probability matrix.

Step 6. Search plan. Based on the selection probability matrix and roulette, determine the interval blocks that each ant will search in this iteration. The search plan is represented as: (12)${\displaystyle A_k={\left[I{S}_{ij}\left(k\right)\right]}_{m\times K},\left(i=1,2,\dots ,m;j=1,2,\dots ,K;k=1,2,\dots ,n\right).}$

Moreover, a set of representative values is calculated for the interval blocks to be searched. In this article, the median of the subinterval is used as the representative value. These values serve as hyperparameters, playing a crucial role in both the initialization and the training processes of the model.

Step 7. Calculate fitness. Under the given hyperparameters, the sliding window mechanism is employed to extract the feature data and the label data, represented as: ${\displaystyle D_{feature}=\left\{d_1,d_2,\dots ,d_t,\dots ,d_{L_{in}}|d_t=x_{t1},x_{t2},\dots ,x_{t{M}_{in}}\right\},}$ ${\displaystyle D_{targets}=\left\{d_{L_{in}+1}=x_{t^{\prime }{i}^{{}_1}},x_{t^{\prime }{i}^2},\dots ,x_{t^{\prime }{i}^{{}_{M_{out}}}}|i^j\in \left\{1,2,\dots ,M_{in}\right\},t^{\prime }=L_{in}+1\right\}}$ where L_in is the length of the input window for the time series, M_in is the number of input features, and M_out is the number of features to be predicted. D_feature are substituted into Equations (6)–(10). The extracted features are then transformed into predicted values using a fully connected network. Next, the loss function is calculated, and the model parameters are updated through backpropagation. Finally, the loss function values of the training set and validation set throughout the entire training process are substituted into Eq. (11). We obtain the fitness corresponding to the given hyperparameters.

Step 8. Update. The pheromone matrix is updated according to Eqs. (3) and (4). Subsequently, the selection probability matrix is recalculated using Eq. (5). The process then returns to step 3.

Step 9. Prediction model. The hyperparameters obtained from step 3 are employed to define and train the neural network model. Subsequently, the final predictive model is generated.

Experimental environment

All the models were trained/tested in the same experimental conditions, and detailed configuration is given in Table 5.

The processing of data

We have selected 1-minute cycle K-line fitting price spread data as the experimental data to test the performance of the proposed forecasting model. And the price spread data for each period (1 min) consists of eight features, including the opening price spread, the highest price spread, the lowest price spread, the closing price spread, MACD, DEA, DIF, and the price spread fluctuation (See ‘Data description’). Among these features, the closing price spread is our target for prediction. In all experiments, the datasets (190,000 min) have been split into a training set (128,000 min), a validation set (30,000 min) and a test set (32,000 min) in chronological order.

Each data is normalized to the range of 0 to 1 when the characteristic data is input into the artificial neural network. It reduces the effects of noise, ensuring that neural networks update parameters efficiently and speed up the training of the network. We use the following formula for normalization. (13)${\displaystyle \overline{x}\left(t\right)=\frac{x\left(t\right)-x_{\min }}{x_{\max }-x_{\min }}}$ where x_min and x_max are the minimum and maximum values of each feature in the training set respectively. Since the data is normalized during the model training phase, the output of the test set can be restored by the formula $x\left(t\right)=x{\left(t\right)}^{\prime }\left(x_{max}-x_{min}\right)+x_{min}$, where $x{\left(t\right)}^{\prime }$ is the output value of the forecasting model.

Metrics

Four error measures are adopted to assess the performance of the proposed models with more accuracy, including mean square error (MSE), symmetric mean absolute percentage error (sMAPE) and root relative square error (RSE). They are calculated as follows: (14)${\displaystyle e_{MAE}=\frac{1}{n}\sum _{t=1}^n\left|{\hat{y}}_t-y_t\right|}$ (15)${\displaystyle e_{MSE}=\frac{1}{n}\sum _{t=1}^n{\left({\hat{y}}_t-y_t\right)}^2}$ (16)${\displaystyle e_{sMAPE}=\frac{100\text{\%}}{n}\sum _{t=1}^n\frac{\left|{\hat{y}}_t-y_t\right|}{\left(\left|y_t\right|+\left|{\hat{y}}_t\right|\right)/2}}$ (17)${\displaystyle e_{RSE}=\sqrt{\frac{\sum _{t=1}^n{\left(y_t-{\hat{y}}_t\right)}^2}{\sum _{t=1}^n{\left(y_t-\overline{y}\right)}^2}}.}$

In the above formula, y_t represents the original value of the moment t, ${\hat{y}}_t$ represents the predicted value of the moment t, and n is the total number of test samples. If the values of MAE, MSE, sMAPE, and RSE are smaller, the deviation between the predicted value and the original value is also smaller.

Besides the aforementioned indicators, we utilized two conventional evaluation metrics defined as:

• Empirical Correlation Coefficient (CORR): (18)${\displaystyle e_{corr}=\frac{\sum _{t=1}^n\left({\hat{y}}_t-\overline{\hat{y}}\right)\left(y_t-\overline{y}\right)}{\sqrt{\sum _{t=1}^n{\left({\hat{y}}_t-\overline{\hat{y}}\right)}^2}\sqrt{\sum _{t=1}^n{\left(y_t-\overline{y}\right)}^2}}}$ which is used to test the degree of association between two variables,

• Symbol Accuracy (SA): (19)${\displaystyle e_{SA}=\frac{100\text{\%}}{n-1}\sum _{t=1}^{n-1}{z}_t}$ where $z_t=\left\{\begin{array}{cc}1,\hfill & if\left(y_{t+1}-y_t\right)\left({\hat{y}}_{t+1}-y_t\right)>0\hfill \\ 0,\hfill & if\left(y_{t+1}-y_t\right)\left({\hat{y}}_{t+1}-y_t\right)<0\hfill \end{array}\right.,\overline{y}=\frac{1}{n}{\sum }_{t=1}^n{y}_t,\overline{\hat{y}}=\frac{1}{n}{\sum }_{t=1}^n{\hat{y}}_t$ . For both CORR and SA, a higher value indicates better performance.

Model construction and hyperparameter tuning

All of our experiments use the Adam optimizer to optimize the parameters of model. The mean square error (MSE) is chosen as the loss function of model, and the batch size is 512.

In our KCS-LSTM network, the LSTM model includes an LSTM layer and a fully connected layer, beyond the input and output layers. To ensure a solid fit between the LSTM model and our data, the KCS-LSTM network optimizes the hyperparameters of the LSTM model using the KCS algorithm. We analyzed some research article (Greff et al., 2016; Ding & Qin, 2020) on the selection of hyperparameters for the LSTM model. The learning rate and the number of parameters in the model are crucial hyperparameters, as confirmed by our experimental results. Therefore, we ultimately decided to select five hyperparameters as optimization targets, including the initial learning rate, the number of neurons in each hidden layer (LSTM and FC), the time step of the input window, and epoch. In addition, considering the constraints of the experimental conditions, we have defined a suitable range for hyperparameter search: the learning rate is between [0.001,0.01], the number of neurons in both the LSTM and Fully Connected layers is between [1,150], the time step is between [2,60], and the epoch is between [10,100]. More details of network components are provided in Table 6.

Figure 6 provides a visualization of the search space alterations throughout the iterative process. The x-axis represents the iteration count, with the current hyperparameter value range being recorded every fifth iteration. The y-axis, represented by the shaded region, corresponds to the feasible value interval. Table 7 provides a quantitative representation of the aforementioned content, with data recorded every tenth iteration. As depicted in Fig. 6 and Table 7, the optimization process for the search space is primarily driven by the worst elimination rule, with the optimal selection rule playing a secondary role in inducing changes. It can effectively mitigate the potential negative impact on search results that could arise from the weak representativeness of interval representative values.

Figure 7 illustrates the changing distribution of fitness values for the sample points throughout the iterative process. Each subplot’s caption indicates the iteration count and the number of sample points included. Note that each iteration consists of 20 sample points. Fitness values that exceed the range of the x-axis are not shown in the figure. Furthermore, to facilitate representation, a logarithmic scaling transformation has been applied to the fitness values. As illustrated in Fig. 7, in the first iteration of the random search, the fitness of the 20 sample points has only 70% of the sample values within [−3.3, −2.7], whose distribution center is around −3. The remaining 30% of the sample values are higher than −2.7. As the search process progresses, the center of the fitness distribution of the sample points, obtained by semi-random sampling relying on pheromones, shifts towards a smaller value of −3.2. The distribution transitions from being sparse to becoming more concentrated. It demonstrates the KCS algorithm’s effectiveness in optimizing the search space.

Table 8 shows the optimal hyperparameters obtained for the KCS-LSTM network.