An efficient hybrid differential evolution-golden jackal optimization algorithm for multilevel thresholding image segmentation

Exploration phase

In the exploration phase, the golden jackal utilizes its unique ability to locate and stalk the prey. Male and female jackals hunt for prey together. The mathematical model of behavior is described as: (9)${\displaystyle P_1\left(t\right)=P_M\left(t\right)-E\cdot |P_M\left(t\right)-rl\cdot Prey\left(t\right)|,}$ (10)${\displaystyle P_2\left(t\right)=P_{FM}\left(t\right)-E\cdot |P_{FM}\left(t\right)-rl\cdot Prey\left(t\right)|,}$ (11)${\displaystyle E=1.5\times E_0\times \left(1-t/T\right),}$ (12)${\displaystyle rl=0.05\times LF,}$ where P_M(t) and P_FM(t) represent the positions of the male and female jackals, respectively. Prey(t) represents the position vector of the prey, E denotes the prey’s escape energy, E₀ represents the initial escape energy in [-1,1], t is the current iteration, T is the maximum iteration, rl is a random vector, and LF denotes the Lévy flight function (Heidari et al., 2019).

Finally, the positions of the golden jackal are updated with the positions of the male and female jackals, represented as: (13)${\displaystyle P\left(t+1\right)=0.5\times \left(P_1\left(t\right)+P_2\left(t\right)\right).}$

Exploitation phase

When the prey is tracked and pursued by golden jackals, the prey’s evasion energy degrades rapidly, and a pair of jackals surround the prey. After encircling the prey, the jackals pounce and capture it. The behavior of jackals can be described as follows. (14)${\displaystyle P_1\left(t\right)=P_M\left(t\right)-E\cdot |rl\cdot P_M\left(t\right)-Prey\left(t\right)|,}$ (15)${\displaystyle P_2\left(t\right)=P_{FM}\left(t\right)-E\cdot |rl\cdot P_{FM}\left(t\right)-Prey\left(t\right)|.}$ Finally, the positions of the golden jackal are also updated using the mean position with Eq. (13).

Mutation operation

In the population, mutation vectors are generated through the mutation operation. The commonly used mutation operator is denoted as: (16)${\displaystyle V_i\left(t\right)=X_{best}\left(t\right)+FM\cdot \left(X_{r1}\left(t\right)-X_{r2}\left(t\right)\right),}$ where FM is the scaling control parameter in [0, 2], X_best(t) represents the individual vector for t iterations, r₁ and r₂ are randomly selected values from the population.

Crossover operation

After generating the mutant vector, a crossover operation is performed on the source vector and its corresponding mutation vector to generate the crossover vector. This process is mathematically represented as: (17)${\displaystyle U_i\left(t\right)=\left\{\begin{array}{c}{V}_i\left(t\right),if\left(rand\le CR\right)\hfill \\ X_i\left(t\right),otherwise\hfill \end{array}\right.,}$ where CR denotes a crossover factor in [0, 1].

Selection operation

If the fitness value of the crossover vector is better than that of the source individual vector, the individual vector is updated to the crossover vector. Otherwise, the individual vector remains unchanged. The selection operation is denoted as: (18)${\displaystyle X_i\left(t+1\right)=\left\{\begin{array}{c}{U}_i\left(t\right),f\left(U_i\left(t\right)\right)<f\left(X_i\left(t\right)\right)\hfill \\ X_i\left(t\right),otherwise\hfill \end{array}\right.,}$ where f(⋅) represents the fitness function.

Test functions and experiment settings

Multiple experiments are performed using the CEC2021 benchmark function (Lou et al., 2024; Mohamed et al., 2022; Wang et al., 2023b) to verify the DEGJO algorithm’s performance. Table 2 displays the detailed information on the benchmark functions. For the previously mentioned algorithm, the population number, dimension number, and maximum iteration number are set to 30, 10, and 200, respectively.

Analysis of the results of the CEC2021 benchmark functions

To analyze the results, two measures of average fitness (Avg) and standard deviation (Std) are utilized. After 20 independent runs, the measurement results of different MA algorithms are depicted in Table 3. Figure 2 illustrates the convergence curves of the DEGJO and other MA algorithms.

Table 3 illustrates that the DEGJO algorithm obtains the best results in the test functions compared to the other algorithms. Specifically, for F2, F3, F4, F6, and F8, both the DEGJO and HGJO algorithms achieve the minimum theoretical values, outperforming the results of the other algorithms. Although the results of the DEGJO algorithm are not optimal in F1, F7, F9, and F10, there are still significant improvements with the GJO and HGJO algorithms. These results highlight the superior search capabilities of the proposed algorithm.

As observed in Fig. 2, the DEGJO algorithm exhibits excellent convergence ability. In the unimodal function F1, the convergence accuracy of the DEGJO algorithm is better than other algorithms and the HGJO algorithm. For the basic function F2–F4, the DEGJO algorithm shows good searching ability and has almost the same convergence performance as the HGJO algorithm. For the functions F5–F7 and F8–F10, the DEGJO algorithm consistently outperforms the other MA algorithms in terms of search results.

Test images

To assess the adaptability, the DEGJO is employed for image segmentation. Experiments are performed on six test images used in the literature (Houssein et al., 2021b; Wang & Song, 2022). These images are selected from the Open Standard Test Dataset and Berkeley Segmentation Dataset (Martin et al., 2001), as depicted in Fig. 3.

Figure 3 shows the benchmark images used for the testing along with their corresponding histograms. These histograms reveal variations of gray-level intensity in different images, which facilitate the verification of the robustness and applicability of the proposed method on different datasets.

Evaluation metrics

For image thresholding segmentation of the DEGJO algorithm, multiple evaluation metrics are required to assess the performance of the proposed algorithm. These evaluation metrics are described as follows.

1. Average (Avg) and standard deviation (Std) values of the fitness function

The convergence accuracy of the algorithm is influenced by the fitness value. Accurate image segmentation thresholds can be obtained by iterating through the minimization of the fitness value. Therefore, the efficiency of the algorithm is assessed using Avg and Std of the fitness function.

2. The best threshold values

Optimization algorithms are employed for segmentation images, where the optimal thresholds are derived through iterative optimization of the algorithm. The selection of the best threshold is crucial for image segmentation, as it directly affects the quality of the segmentation.

3. Peak Signal-to-Noise Ratio (PSNR)

The PSNR denotes the peak signal-to-noise ratio of the source I_sou and the segmented image I_seg and is closely related to the quality of the segmented image (Houssein et al., 2022). The PSNR can be calculated as: (22)${\displaystyle PSNR=10log10\left(\frac{255^2}{Mea{n}_{SE}}\right),}$ where Mean_SE is the mean square error.

4. Structural Similarity Index (SSIM)

The SSIM represents the similarity between the I_sou and I_seg images (Houssein et al., 2021b). The higher the value of SSIM, the higher the similarity between the source and the segmented image, which means that the segmentation of the image is more effective. The SSIM can be calculated as: (23)${\displaystyle SSIM\left(x,y\right)=\frac{\left(2{\mu }_{sou}{\mu }_{seg}+c_1\right)\left(2{\sigma }_{ss}+c_2\right)}{\left({\mu }_{sou}^2+{\mu }_{seg}^2+c_1\right)\left({\sigma }_{sou}^2+{\sigma }_{seg}^2+c_2\right)},}$ where μ_sou and μ_seg are the mean intensity of the I_sou and I_seg images, respectively. σ_sou and σ_seg represent the standard deviation of the I_sou and I_seg images, respectively. σ_ss is the covariance. c₁ and c₂ are the constant numbers.

5. Feature Similarity Index (FSIM)

The FSIM denotes the similarity between the I_sou and I_seg images (Aziz, Ewees & Hassanien, 2017). A high FSIM value denotes good performance of the image segmentation method. The calculation formula is denoted as follows: (24)${\displaystyle \text{FSIM}=\frac{\sum _{x\in \Omega }{S}_L\left(x\right)P{C}_{max}\left(x\right)}{\sum _{x\in \Omega }P{C}_{max}\left(x\right)},}$ (25)${\displaystyle S_L\left(x\right)=S_{PC}\left(x\right)S_G\left(x\right),}$ (26)${\displaystyle S_{PC}\left(x\right)=\frac{2P{C}_1\left(x\right)P{C}_2\left(x\right)+T_1}{P{C}_1^2\left(x\right)+P{C}_2^2\left(x\right)+T_1},}$ (27)${\displaystyle S_G\left(x\right)=\frac{2G_1\left(x\right)G_2\left(x\right)+T_2}{G_1^2\left(x\right)+G_2^2\left(x\right)+T_2},}$ (28)${\displaystyle P{C}_{max}\left(x\right)=max\left\{P{C}_1\left(x\right),P{C}_2\left(x\right)\right\},}$ where T₁ and T₂ are constant numbers. PC(x) denotes the phase congruence, and G(x) denotes the magnitude of the image gradient.

Analysis of the segmented image results

The DEGJO algorithm is employed to search for the optimal multilevel threshold, with an objective function of cross-entropy minimization. For each algorithm, the N is selected as 30, T is chosen as 100, and the threshold levels are set to 3, 5, and 8, respectively. To reduce the randomness of the algorithms, each algorithm is run independently 20 times. The experimental results of the DEGJO and other MA algorithms are displayed in Tables 4–8.

Table 4 shows the Avg and Std of the cross-entropy values for the DEGJO and other algorithms. The best values are highlighted in bold. Compared to DE, GWO, WOA, GJO and HGJO algorithms, the DEGJO algorithm obtains the best average values in most images at all levels.

Table 5 depicts the best threshold values with the DE, GWO, WOA, GJO, HGJO, and DEGJO algorithms. Generally, most algorithms exhibit similar values when the threshold levels are 3 and 5, however, significant differences emerge in the threshold values obtained when the threshold level is 8, particularly with the HGJO algorithm.

Table 6 displays the mean PSNR values of the algorithm mentioned above. The Optimal values are marked in bold. In most images, the segmented image using the DEGJO algorithm has a higher PSNR value than other algorithms. However, for some images, other algorithms perform better than the DEGJO algorithm.

Table 7 compares the mean SSIM values derived from the performance of various algorithms. The best results are highlighted in bold, indicating superior segmentation of the original image. It can be observed that the DEGJO algorithm achieves significant results across most of the images.

Table 8 shows the mean FSIM values by employing the proposed DEGJO and other MA algorithms. The optimal values are highlighted in bold, indicating superior segmentation quality, with higher values reflecting better performance of the thresholding method.

Finally, Figs. 4–9 depict the segmentation images using the DEGJO approach at three different levels (k = 3, 5, and 8), alongside their respective histograms. In these figures, the optimal thresholds are indicated by red vertical lines. As shown in Figs. 4–9, the contrast quality of the images improves significantly with the increasing number of thresholds.