Solving high-dimensional global optimization problems via solution space restructuring with neural network

doi:10.1007/s10483-026-3393-7

Abstract

Abstract:

It is well-known that appropriately determining the solution space and initializations is crucial for obtaining high-quality optimal solutions for optimization problems, considering both gradient-based and gradient-free techniques. However, the general framework for adaptively dealing with a particular optimization problem is commonly overlooked and thus still hidden in the literature. To overcome this limitation, a new approach assisted by the neural network (NN) is proposed for solving high-dimensional optimization issues. By restructuring the search space to optimize the objective function via a nonlinear mapping constructed by an autoencoder (AE), the surrogate solution space is constructed by a network training process and dynamically oriented to the optimal solution of the optimization issue. To enhance the optimization efficiency and address non-smooth problems, the classical metaheuristic grey wolf optimizer (GWO) and the adaptive moment estimation (Adam) are sequentially employed to complement the disadvantages of the constituted models. The effectiveness of the proposed approach is validated by solving a set of mathematical functions with 1 000-dimensional and three large-scale truss design optimization problems. Several numerical experiments show that the solution space is reduced in terms of both size and complexity based on the restructuring procedure, in which the global optimal solution is still conserved, leading to better optimization efficiency when solving optimization problems with complex search domains with large dimensions. In addition, the hybrid optimizer has also been proven to be more effective when combined with the restructuring technique owing to the use of the Adam algorithm in the second phase.

Key words: high-dimensional optimization, solution space restructuring, adaptive moment estimation (Adam), grey wolf optimizer (GWO), neural network (NN)

2010 MSC Number:

O234

N. VO, T. LE-DUC, H. TANG, H. NGUYEN-XUAN, S. H. LEE, J. H. LEE. Solving high-dimensional global optimization problems via solution space restructuring with neural network. Applied Mathematics and Mechanics (English Edition), 2026, 47(6): 1383-1400.

TrendMD

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: https://www.amm.shu.edu.cn/EN/10.1007/s10483-026-3393-7

https://www.amm.shu.edu.cn/EN/Y2026/V47/I6/1383

Figures/Tables 9

Fig. 1

Algorithm 1

Pseudocode of SSR-hybrid"

1: Initialize the input X;

2: Set the architecture of SSR,

N N (X, θ);

3: Set the optimizer for SSR;

4: Initialize set of

N N

’s parameters;

5: if GWO phase then

⊳

Only enable this phase when using hybrid approach

6: Optimize θ by GWO;

7: Obtain the best parameter in GWO phase

θ GWO *

with elitist selection operator;

8: end if

9: if Adam phase then

10:

θ : = θ GWO *;

11: while

epoch ⩽ N epochs;

12: Restructure solution space

X ′ = N N (X, θ);

13: Calculate fitness value by using the objective function

f (X ′);

14: Calculate and smooth the constraint by using Eq. (20);

15: Total loss is obtained by

L = f (X ′) + g ˜ (X ′)

as Eq. (22);

16: Optimize θ by Adam;

17:

epoch ← epoch + 1;

18: end while

19: Obtain the final parameter

θ *

by Eq. (15) with identity selection operator;

20: end if

21: Optimal solution

X * = N N (X, θ *) .

Algorithm 1

Fig. 2

Fig. 3

Table 1

Details of unimodal and multimodal mathematical benchmark functions"

Issue	Function	Name	D	Range	Optimal
Unimodal	f₁	Schwefel 2.22	$[100 − 1 000]$	$[− 10, 10]$	0
Unimodal	f₂	Rosenbrock	$[100 − 1 000]$	$[− 30, 30]$	0
Multimodal	f₃	Michalewicz	$[100 − 1 000]$	$[− π, π]$	–
	f₄	Schewefel 2.26	$[100 − 1 000]$	$[− 500, 500]$	$− 418 × D$
	f₅	Rastrigin	$[100 − 1 000]$	$[− 5.12, 5.12]$	0

Table 1

Fig. 4

Table 2

Problem definitions and settings of truss design optimization problems"

Problem	D	A	$σ max$	p
120-bar	120	$[1, 129.3]$	40	1
354-bar	354	$[1.07, 21.3]$	40	1
942-bar	942	$[1, 200]$	25	10³

Table 2

Fig. 5

Table 3

References 27

[1]	LE-DUC, T., NGUYEN, Q. H., LEE, J., and NGUYEN-XUAN, H. Strengthening gradient descent by sequential motion optimization for deep neural networks. IEEE Transactions on Evolutionary Computation, 27(3), 565–579 (2023)
[2]	LE-DUC, T., LEE, S., NGUYEN-XUAN, H., and LEE, J. A hierarchically normalized physics-informed neural network for solving differential equations: application for solid mechanics problems. Engineering Applications of Artificial Intelligence, 133, 108400 (2024)
[3]	LWAKATARE, L. E., RAJ, A., CRNKOVIC, I., BOSCH, J., and OLSSON, H. H. Large-scale machine learning systems in real-world industrial settings: a review of challenges and solutions. Information and Software Technology, 127, 106368 (2020)
[4]	OMIDVAR, M. N., LI, X. D., and YAO, X. A review of population-based metaheuristics for large-scale black-box global optimization: part I. IEEE Transactions on Evolutionary Computation, 26(5), 802–822 (2022)
[5]	BELLMAN, R. Adaptive Control Processes: A Guided Tour, Princeton University Press, New Jersey, 85–99 (1961)
[6]	LIU, G. Y., CHEN, Y. N., XIE, Z., and GENG, X. D. GA\SQP optimization for the dimensional synthesis of a delta mechanism based haptic device design. Robotics and Computer-Integrated Manufacturing, 51, 73–84 (2018)
[7]	TANG, H. and LEE, J. Adaptive initialization LSHADE algorithm enhanced with gradient-based repair for real-world constrained optimization. Knowledge-Based Systems, 246, 108696 (2022)
[8]	MIRJALILI, S., MIRJALILI, S. M., and LEWIS, A. Grey wolf optimizer. Advances in Engineering Software, 69, 46–61 (2014)
[9]	SADRANI, M., TIRACHINI, A., and ANTONIOU, C. Bus scheduling with heterogeneous fleets: formulation and hybrid metaheuristic algorithms. Expert Systems with Applications, 263, 125720 (2025)
[10]	SI, T., BAGCHI, J., and MIRANDA, P. B. C. Artificial neural network training using metaheuristics for medical data classification: an experimental study. Expert Systems with Applications, 193, 116423 (2022)
[11]	BARUA, S. and MERABET, A. Lévy arithmetic algorithm: an enhanced metaheuristic algorithm and its application to engineering optimization. Expert Systems with Applications, 241, 122335 (2024)
[12]	TANG, H., NGUYEN, N. V., LEE, S., and LEE, J. An analyzer-surrogate-hybrid optimization framework for three-dimensional functionally graded material distribution. Computers & Structures, 302, 107472 (2024)
[13]	LE-DUC, T., NGUYEN-XUAN, H., and LEE, J. Sequential motion optimization with short-term adaptive moment estimation for deep learning problems. Engineering Applications of Artificial Intelligence, 129, 107593 (2024)
[14]	STEIN, M. Large sample properties of simulations using Latin hypercube sampling. Technometrics, 29(2), 143–151 (1987)
[15]	CAFLISCH, R. E. Monte Carlo and quasi-Monte Carlo methods. Acta Numerica, 7, 1–49 (1998)
[16]	TOKDAR, S. T. and KASS, R. E. Importance sampling: a review. Wiley Interdisciplinary Reviews: Computationl Statistics, 2(1), 54–60 (2010)
[17]	LONG, W., JIAO, J. J., LIANG, X. M., and TANG, M. Z. Inspired grey wolf optimizer for solving large-scale function optimization problems. Applied Mathematical Modelling, 60, 112–126 (2018)
[18]	WANG, J., LIN, D. K., ZHANG, Y. Z., and HUANG, S. G. An adaptively balanced grey wolf optimization algorithm for feature selection on high-dimensional classification. Engineering Applications of Artificial Intelligence, 114, 105088 (2022)
[19]	KINGMA, D. P. and BA, J. Adam: a method for stochastic optimization. International Conference on Learning Representations, California (2015)
[20]	BAYDIN, A. G., PEARLMUTTER, B. A., RADUL, A. A., and SISKIND, J. M. Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research, 18(1), 5595–5637 (2017)
[21]	ABADI, M., BARHAM, P., CHEN, J., CHEN, Z., DAVIS, A., DEAN, J., DEVIN, M., GHEMAWAT, S., IRVING, G., ISARD, M., KUDLUR, M., LEVENBERG, J., MONGA, R., MOORE, S., MURRAY, D. G., STEINER, B., TUCKER, P., VASUDEVAN, V., WARDEN, P., WICKE, M., YU, Y., and ZHENG, X. Q. Tensorflow: a system for large-scale machine learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, Georgia, 265–283 (2016)
[22]	PASZKE, A., GROSS, S., MASSA, F., LERER, A., BRADBURY, J., CHANAN, G., KILLEEN, T., LIN, Z., GIMELSHEIN, N., ANTIGA, L., DESMAISON, A., KOPF, A., YANG, E., DEVITO, Z., RAISON, M., TEJANI, A., CHILAMKURTHY, S., STEINER, B., FANG, L., BAI, J., and CHINTALA, S. Pytorch: an imperative style, high-performance deep learning library. Proceedings of the 33rd International Conference on Neural Information Processing Systems, New York, 8026–8037 (2019)
[23]	PALUSZEK, M. and THOMAS, S. Practical MATLAB Deep Learning, Apress, California (2020)
[24]	RUDER, S. An overview of gradient descent optimization algorithms. ArXiv Preprint, (2016)
[25]	EIBEN, A. E. and SMITH, J. E. Introduction to Evolutionary Computing, Springer Berlin, Heidelberg, 79–98 (2015)
[26]	GURROLA-RAMOS, J., HERNANDEZ-AGUIRRE, A., and DALMAU-CEDENO, O. COLSHADE for real-world single-objective constrained optimization problems. 2020 IEEE Congress on Evolutionary Computation (CEC), IEEE, Glasgow, 1–8 (2020)
[27]	KUMAR, A., WU, G. H., ALI, M. Z., MALLIPEDDI, R., SUGANTHAN, P. N., and DAS, S. A test-suite of non-convex constrained optimization problems from the real-world and some baseline results. Swarm and Evolutionary Computation, 56, 100693 (2020)

Problem	Average time/s
Problem	GWO	Adam	COLSHADE	SSR-GWO	SSR-Adam	SSR-hybrid
120-bar	22.42	56.03	19.07	9.14	17.07	44.90
354-bar	72.29	131.66	92.66	17.42	53.47	75.41
942-bar	281.26	280.74	314.30	56.67	202.68	256.30