The study of the overburden rock transportation law of different mining thicknesses can provide a theoretical basis for the actual site of the mine and ensure the safety of coal mining. On the basis of analyzing the deformation characteristics of the overburden rock, the article studies the relevant factors affecting the mobile deformation of the overburden rock and gives the nonlinear mechanical, mathematical model of the overburden rock movement in the quarry, i.e., the yield failure criterion, the GHB flow law, and plasticity increment theory. Based on the nonlinear mechanics theory, the generalized Kelvin model was established, and its constitutive equations were transformed by the Laplace transformation method, and the viscoelastic finite element numerical simulation model was constructed by combining with MIDAS GTS NX software. When the mining depth is 150 meters, and the thickness of the mining is 9 meters, the maximum tensile stress of the quarry overburden is 274.28MPa, which will damage the movement and deformation of the overburden. When the mining thickness increases from 3 meters to 9 meters, the maximum value of settlement and horizontal displacement of the overburden is 0.038 meters. There is a significant positive correlation between the mining thickness and the movement and deformation of the overburden, i.e., the greater the mining thickness, the more serious the movement and deformation of the overburden. In order to ensure the safety of mining work, it is essential to establish a reasonable mining thickness in the actual operation.