Early Stopping Criteria for Levenberg-Marquardt Based Neural Network Training Optimization

  • Authors

    • Azizah Suliman
    • Batyrkhan Omarov
    2018-12-09
    https://doi.org/10.14419/ijet.v7i4.36.25382
  • Early Stop Condition, Levenberg-Marquardt Method, Neural Network, Overtraining.

  • In this research we train a direct distributed neural network using Levenberg-Marquardt algorithm. In order to prevent overtraining, we proposed correctly recognized image percentage based on early stop condition and conduct the experiments with different stop thresholds for image classification problem. Experiment results show that the best early stop condition is 93% and other increase in stop threshold can lead to decrease in the quality of the neural network. The correct choice of early stop condition can prevent overtraining which led to the training of a neural network with considerable number of hidden neurons.

     

     

  • References

    1. [1] Omarov, B., Suliman, A., Kushibar, K. Face recognition using artificial neural networks in parallel architecture. Journal of Theoretical and Applied Information Technology 91 (2), pp. 238-248. (2016). Islamabad

      [2] Omarov, B., Suliman, A., Tsoy, A. Parallel backpropagation neural network training for face recognition. Far East Journal of Electronics and Communications. Volume 16, Issue 4, December 2016, Pages 801-808. (2016)

      [3] A. Altayeva, B. Omarov, H.C. Jeong, Y.I. Cho. Multi-step face recognition for improving face detection and recognition rate. Far East Journal of Electronics and Communications 16(3), pp. 471-491, 2016

      [4] Lutz P. 1998. Early Stopping-But When? Neural Networks: Tricks of the Trade. London, UK: Springer-Verlag

      [5] Marquardt D. 1963. An Algorithm for Least-Squares Estimation of Nonlinear Parameters SIAM Journal on Applied Mathematics. T. 11. â„– 2. C. 431-441

      [6] Pinz A. 2016. Human, car, other object database, Electronic resource, http://cvrg.iyte.edu.tr/datasets.htm.

      [7] Xu J., Ho D.W.C., Zheng Y. 2004. A Constructive Algorithm for Feedforward Neural Networks Control Conference. Shanghai: Inst, of Syst. Sei., East China Normal Univ., C. 659-664.

      [8] Sattar, M.A., Achanta, S. “Development and validation of a simple method for simultaneous estimation of memantine and donepezil in pharmaceutical dosage forms by using RP-HPLCâ€, (2018) International Journal of Pharmaceutical Research, 10 (2), pp. 155-166.

      [9] V. Franc and J. Cech, Learning CNNs for Face Recognition from Weakly Annotated Images, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), Washington, DC, 2017, pp. 933-940. doi: 10.1109/FG.2017.115, 2017

  • Downloads

  • How to Cite

    Suliman, A., & Omarov, B. (2018). Early Stopping Criteria for Levenberg-Marquardt Based Neural Network Training Optimization. International Journal of Engineering & Technology, 7(4.36), 1194-1198. https://doi.org/10.14419/ijet.v7i4.36.25382