Sclerosis detection using brain magnetic resonant imaging (MRI) images is challenging task. With the promising results for variety of applications in terms of classification accuracy using of deep neural network models, one can use such models for sclerosis detection. The features associated with sclerosis is important factor which is highlighted with contrast lesion in brain MRI images. The sclerosis classification initially can be considered as binary task in which the sclerosis segmentation can be avoided for reduced complexity of the model. The sclerosis lesion show considerable impact on the features extracted using convolution process in convolution neural network models. The images are used to train the convolutional neural network composed of 35 layers for the classification of sclerosis and normal images of brain MRI. The 35 layers are composed of combination of convolutional layers, Maxpooling layers and Upscaling layers. The results are compared with VGG16 model and results are found satisfactory and about 92% accuracy is seen for validation set.