A system that classifies different types of fruits and identifies the quality of fruits will be of value in various areas, especially in the mass production of fruit products. Automatic classification of fresh and rotten fruits plays a substantial role in agriculture as well as the food industry. Traditional methods for fruit spoilage detection are manual, inaccurate, time-consuming, laborious, and subjective. So, this paper presents a new approach for automatic classification of fruit quality using deep learning, which is mainly focused on image resizing and classification of fruits ripeness. This paper uses bicubic interpolation approaches for image resizing to compute the loss. Then, the automatic fruit classification (different sizes concerned) is achieved by Convolutional Neural Network (CNN) with the VGG16 model. It even updates the parameters by concentrating channels with respect to Red, Green, and Blue (RGB) to identify as well as classify the images of ripened and rotten. The suggested approach is implemented on the python platform. The evaluation of CNN with VGG16 using SGD algorithm for accurate classification is determined by parameters like accuracy curve and loss curve of training dataset as well as validation dataset with different numerical output.