Background: This study aims to create a machine learning model that produces explainable, interpretable, and trustable predictions for diabetes using an XAI approach. Objective: In the study, we have utilized an earlier approach to implementing explainable Machine Learning. Methods: In order to apply XAI technique, we follow a brief version of CRISP-DM. (i) Data Understanding, (ii) Data Preparation, (iii) Model Planning and Building (iv) SHAP Implementation for Interpretability. Results: Global interpretability shows us that two major contributors are symptoms of Polydipsia and Polyuria. An algorithm doesn't "know" prior information, which is highly specific domain knowledge. Local interpretability-based single-instance explanation showed decent multivariate reasoning capability. If the reasoning was based on a simple univariate approach, positive polyuria alone should result in a high probability of positive model output, considering the positive SHAP value of polyuria. Conclusion: The model output results 99.7% confidence to be classified as negative makes much sense since polyuria is also a common symptom of many different situations, such as diabetes insipidus, Kidney disease, Liver failure, Medications that include diuretics, Chronic diarrhea, Cushing's syndrome, Psychogenic polydipsia, Hypercalcemia, Pregnancy.