Modified Cascade-2 Algorithm with Adaptive Slope Sigmoidal Function
Article Sidebar
Section
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Main Article Content
Abstract
Cascade-2 algorithm is a variant of well-known cascade-correlation algorithm that is widely investigated constructive training algorithm for designing cascade feed forward neural networks. This paper proposes a modified Cascade-2 algorithm with adaptive slope sigmoid function (MC2AASF). The algorithm emphasizes on architectural adaptation and functional adaptation during learning. This algorithm is a constructive approach of designing cascade architecture. To achieve functional adaptation, the slope of the sigmoid function is adapted during training. One simple variant is derived from MC2AASF is where the slope parameter of sigmoid function used at the hidden layers’ nodes is fixed to unity. Both the variants are compared to each other on five function approximation tasks. Simulation results show that adaptive slope sigmoid function presents several advantages over standard fixed shape sigmoid function, resulting in increasing flexibility, smoother learning, better generalization performance and better convergence
Article Details
References
[1] T. Y. Kwok, D. Y. Yeung, (1997) “Constructive Algorithms for Structure Learning in feedforward Neural Networks for Regression Problems,” IEEE Transactions on Neural Networks, vol. 8, no. 3, pp 630-645.
[2] T.Y. Kwok, D.Y.Yenug, (1997) “Objective functions for training new hidden units in constructive neural networks,” IEEE Transactions on Neural Networks, vol. 8, no. 5, pp 1131-1148.
[3] J. J. T. Lahnajarvi, M. I. Lehtokangas, and J. P. P. Saarinen, (2002) “Evaluation of constructive neural networks with cascaded architectures,” Neurocomputing, vol. 48, pp 573-607.
[4] L. Ma and K. Khorasani, (2004) “New training strategies for constructive neural networks with application to regression problems,” Neurocomputing, vol. 17, pp 589-609.
[5] S. E. Fahlman and C. Lebiere, (1990) “The cascade correlation learning architecture,” Advances in Neural Information Processing
System 2, D. S. Touretzky, Ed. CA: Morgan Kaufmann, pp 524-277.
[6] T. Ash, (1989) “Dynamic node creation in backpropagation networks,” Connection Science, vol. 1, no. 4, pp 365-375.
[7] L. Prechelt, (1997)"Investigation of the cascor family of learning algorithms,” Neural Networks
10 (5), pp 885-896.
[8] S. E. Fahlman and J. A. Boyan, (1994) “The Cascade 2 Learning Architecture,” Technical Report(forthcoming), CMU-CS-94-100, Carnegie
Mellon University
[9] M. C. Nechyba and Y. Xu, “Neural network approach to control system identification with variable activation functions,” IEEE International Symposium on Intelligent Control, Columbus, Ohio, USA (1994).
[10] J. N. Hwang, S. Shien and S. R. Lay, (1996) “The Cascade – Correlation Learning: A Projection Pursuit Learning Perspective,” In IEEE Transactions on Neural Networks, vol. 7,
no. 2.
[11] T. Yamada, T. Yabuta, (1992) “Remarks on a neural network controller which uses an auto tuning method for nonlinear functions,” IJCNN, Vol. 2, pp 775-780.
[12] Z. Hu and H. Shao, (1992) “The study of neural network adaptive control systems,” control and Decision, vol. 7, pp 361-366.
[13] C. T. Chen and W. D. Chang, (1996), “A feedforward neural network with function shape auto tuning,” Neural Networks, Vol. 9, issue (4), pp 627-641.
[14] S. Xu and M. Zhang, (2001), “A novel adaptive activation function,” In Proc. Int. Joint Conf. Neural Networks, vol. 4, pp 2779-2782.
[15] P. Chandra ,Y. Singh, (2004),“An activation function adapting training algorithm for sigmoidal feedforward networks,” Neurocomputing, pp 429-437.
[16] S. K. Sharma and P. Chandra.,(2010). "An adaptive slope sigmoidal function cascading neural networks algorithm", Emerging Trends in Engineering and Technology (ICETET), 2010 3rd International Conference on IEEE.