Iterative error re-establishment to improve neural network cost function for imbalanced data learning

Abstract

A new training algorithm to enhance the accuracy of minority class in imbalanced data learning problem was proposed. This algorithm is based on the observation that the cause of lower accuracy is due to the domination of the error terms, i.e. the square of difference between the target and the actual output, computed by those data in majority class in the cost function. To resolve this domination, our cost function is re-established at each epoch based on the errors of the data in minority and majority classes. Any datum whose corresponding term in the cost function produces an error less than 0.05 is removed from cost function. Otherwise, it is put back into the cost function. Our algorithm adopting multilayer perceptron and Levenberg-Marquardt (LM) as the learning algorithm was compared with classical LM and the recent algorithm RAMOBoost on 15 well-known benchmarks. The experimental results of our approach produced higher accuracy than the other approaches in 13 cases with faster training speed.