In this paper, we first establish a model of ultra-large-scale integrated circuits and study the model architecture from basic circuit units to complex circuit units. Then, the circuit optimization problem is mathematically analyzed, and the unconstrained and constrained parametric optimization problems with electrical parameters are investigated. Reinforcement learning is introduced to a reasonably one-to-one correspondence between the parametric optimization problem and the environment in reinforcement learning, which transforms the ordinary optimization problem into a task of reinforcement learning and realizes the optimization of electrical parameters in integrated circuit design. Finally, the effect of optimizing the electrical parameters of the method in this paper is evaluated. In the case of 200 DPPM, 300 DPPM, and 400 DPPM, the number of censored test parameters of this paper’s method is distributed in the range of (10,15), while the number of censored test parameters of the other methods are in the interval of (2,10), and this paper’s method outperforms the other methods. This study has an important reference value to improve the efficiency, reliability, and performance of integrated circuit design, and can provide a reference for the design of integrated circuits.