Now a days, as the large amount of annotated medical data has been growing quickly, and giving more attention for the deep learning-based approaches and having a lot of achievement in the medical segmentation field, such as CAD and other things. A hierarchical representation of data in medical image recognition problems is able to learn by using Deep learning when it is used in biologically-inspired architectures. This helps it learn how to distinguish between different image types. In other words, if the discriminative info is only found in small parts of the image, an existing classic deep learning framework may still have problems finding them without local-level annotations. In this paper, we show how to use “a Novel multi-Phasebased deep learning framework” to find local discriminative details for medical image segmentation that can be found in large-scale output space.