Aiming at the complexity of the problems of risk prediction and drug cost and utility in pharmacoeconomics, this paper proposes the application of the Markov decision model to pharmacoeconomics, and based on this, the solution method of pharmacoeconomic optimization is proposed. After summarizing the advantages and purposes of Markov’s application in pharmacoeconomics, the Markov decision process is established from four aspects: state of the world, action, transfer, and benefit function. The Lagrangian function is constructed with the expectation of the maximum long-term drug benefit. The solution problem is converted into an unconstrained problem, and the objective solution is carried out using reinforcement learning methods. The improved algorithm’s convergence is examined. It is found that the optimized Markov decision-making algorithm obtains a cumulative payoff value of 85, which is 25 higher than that obtained by the Markov decision-making algorithm, and the Markov decision-making model is more effective in evaluating the economics of drugs or treatment measures. Evaluating the long-term benefits of therapeutic measures on cost, survival, and quality of life in sick populations has promising applications as well.