Comparative Study on Various Adaptive Learn-Rate Optimisers on Image Tasks

Abstract

This study aims to evaluate the impact of utilising different adaptive learn-rate optimization tools on image classification tasks, as well as the effect of new features on accuracy and computational time. The study compares the performance of an Adam, RMSprop and AMSGrad optimizer on various image recognition tasks (digit recognition, image classification). The study evaluates the computational time for each optimiser as well as several benchmarks of the model’s resultant accuracy, being the F1-Score and the mean squared error. This study aims to compare the practical benefit of newer iterations of adaptive learn-rate algorithms in terms of computational time and accuracy of prediction in image classification tasks. The content of the study has been taken from MNIST and CIFAR datasets and used to train models for all three algorithms. The results of this study indicate that the RMSProp optimiser has a generally higher compute speed and accuracy when compared to the Adam and AMSGrad models.