Image inpainting aims to fill the undetectable domain and has been studied using deep learning in recent years. This study investigates smoothness-informed convolutional neural network models for image inpainting. The total variation (TV) is considered and local smoothness constraints are also explored in this study. The local smoothness constraint is conducted by the fidelity of the low-order derivatives on mostly connected parts of the given image at training stage. Unlike most neural network-based inpainting methods using numerous images for training, only a single local image containing the domain to be filled is required for the whole training here. The convolutional neural network accepts the image and is trained using detectable data. Computational results indicate that the local smoothness constraint can conduct a more satisfactory inpainting in comparison to usual TV-based one. We also demonstrate how a deep learning approach is used to solve the Euler-Lagrange equation-based inpainting.