Modified Cascade-2 Algorithm with Adaptive Slope Sigmoidal Function

Cascade-2 algorithm is a variant of well-known cascade-correlation algorithm that is widely investigated constructive training algorithm for designing cascade feed forward neural networks. This paper proposes a modified Cascade-2 algorithm with adaptive slope sigmoid function (MC2AASF). The algorithm emphasizes on architectural adaptation and functional adaptation during learning. This algorithm is a constructive approach of designing cascade architecture. To achieve functional adaptation, the slope of the sigmoid function is adapted during training. One simple variant is derived from MC2AASF is where the slope parameter of sigmoid function used at the hidden layers’ nodes is fixed to unity. Both the variants are compared to each other on five function approximation tasks. Simulation results show that adaptive slope sigmoid function presents several advantages over standard fixed shape sigmoid function, resulting in increasing flexibility, smoother learning, better generalization performance and better convergence

Keywords

Adaptive slope sigmoid function; Cascade-Correlation algorithm; Cascade-2 algorithm; Constructive neural networks; Dynamic node creation.

References

[1] T. Y. Kwok, D. Y. Yeung, (1997) “Constructive Algorithms for Structure Learning in feedforward Neural Networks for Regression Problems,” IEEE Transactions on Neural Networks, vol. 8, no. 3, pp 630-645.
[2] T.Y. Kwok, D.Y.Yenug, (1997) “Objective functions for training new hidden units in constructive neural networks,” IEEE Transactions on Neural Networks, vol. 8, no. 5, pp 1131-1148.
[3] J. J. T. Lahnajarvi, M. I. Lehtokangas, and J. P. P. Saarinen, (2002) “Evaluation of constructive neural networks with cascaded architectures,” Neurocomputing, vol. 48, pp 573-607.
[4] L. Ma and K. Khorasani, (2004) “New training strategies for constructive neural networks with application to regression problems,” Neurocomputing, vol. 17, pp 589-609.
[5] S. E. Fahlman and C. Lebiere, (1990) “The cascade correlation learning architecture,” Advances in Neural Information Processing
System 2, D. S. Touretzky, Ed. CA: Morgan Kaufmann, pp 524-277.
[6] T. Ash, (1989) “Dynamic node creation in backpropagation networks,” Connection Science, vol. 1, no. 4, pp 365-375.
[7] L. Prechelt, (1997)"Investigation of the cascor family of learning algorithms,” Neural Networks
10 (5), pp 885-896.
[8] S. E. Fahlman and J. A. Boyan, (1994) “The Cascade 2 Learning Architecture,” Technical Report(forthcoming), CMU-CS-94-100, Carnegie
Mellon University
[9] M. C. Nechyba and Y. Xu, “Neural network approach to control system identification with variable activation functions,” IEEE International Symposium on Intelligent Control, Columbus, Ohio, USA (1994).
[10] J. N. Hwang, S. Shien and S. R. Lay, (1996) “The Cascade – Correlation Learning: A Projection Pursuit Learning Perspective,” In IEEE Transactions on Neural Networks, vol. 7,
no. 2.
[11] T. Yamada, T. Yabuta, (1992) “Remarks on a neural network controller which uses an auto tuning method for nonlinear functions,” IJCNN, Vol. 2, pp 775-780.
[12] Z. Hu and H. Shao, (1992) “The study of neural network adaptive control systems,” control and Decision, vol. 7, pp 361-366.
[13] C. T. Chen and W. D. Chang, (1996), “A feedforward neural network with function shape auto tuning,” Neural Networks, Vol. 9, issue (4), pp 627-641.
[14] S. Xu and M. Zhang, (2001), “A novel adaptive activation function,” In Proc. Int. Joint Conf. Neural Networks, vol. 4, pp 2779-2782.
[15] P. Chandra ,Y. Singh, (2004),“An activation function adapting training algorithm for sigmoidal feedforward networks,” Neurocomputing, pp 429-437.
[16] S. K. Sharma and P. Chandra.,(2010). "An adaptive slope sigmoidal function cascading neural networks algorithm", Emerging Trends in Engineering and Technology (ICETET), 2010 3rd International Conference on IEEE.

Download

pdf

How to Cite

Jaswinder Kaur, Sudhir Kumar Sharma . (2018). Modified Cascade-2 Algorithm with Adaptive Slope Sigmoidal Function. IITM Journal of Management and IT, 9(1), 32-36.

About This Journal

IIT

IITM Journal of Management and IT

Published by Institute of Information Technology & Management

P-ISSN 2349-9826

✓ Citation copied to clipboard

[1] [1] T. Y. Kwok, D. Y. Yeung, (1997) “Constructive Algorithms for Structure Learning in feedforward Neural Networks for Regression Problems,” IEEE Transactions on Neural Networks, vol. 8, no. 3, pp 630-645.

[2] [2] T.Y. Kwok, D.Y.Yenug, (1997) “Objective functions for training new hidden units in constructive neural networks,” IEEE Transactions on Neural Networks, vol. 8, no. 5, pp 1131-1148.

[3] [3] J. J. T. Lahnajarvi, M. I. Lehtokangas, and J. P. P. Saarinen, (2002) “Evaluation of constructive neural networks with cascaded architectures,” Neurocomputing, vol. 48, pp 573-607.

[4] [4] L. Ma and K. Khorasani, (2004) “New training strategies for constructive neural networks with application to regression problems,” Neurocomputing, vol. 17, pp 589-609.

[5] [5] S. E. Fahlman and C. Lebiere, (1990) “The cascade correlation learning architecture,” Advances in Neural Information Processing

[6] System 2, D. S. Touretzky, Ed. CA: Morgan Kaufmann, pp 524-277.

[7] [6] T. Ash, (1989) “Dynamic node creation in backpropagation networks,” Connection Science, vol. 1, no. 4, pp 365-375.

[8] [7] L. Prechelt, (1997)"Investigation of the cascor family of learning algorithms,” Neural Networks

[9] 10 (5), pp 885-896.

[10] [8] S. E. Fahlman and J. A. Boyan, (1994) “The Cascade 2 Learning Architecture,” Technical Report(forthcoming), CMU-CS-94-100, Carnegie

[11] Mellon University

[12] [9] M. C. Nechyba and Y. Xu, “Neural network approach to control system identification with variable activation functions,” IEEE International Symposium on Intelligent Control, Columbus, Ohio, USA (1994).

[13] [10] J. N. Hwang, S. Shien and S. R. Lay, (1996) “The Cascade – Correlation Learning: A Projection Pursuit Learning Perspective,” In IEEE Transactions on Neural Networks, vol. 7,

[14] no. 2.

[15] [11] T. Yamada, T. Yabuta, (1992) “Remarks on a neural network controller which uses an auto tuning method for nonlinear functions,” IJCNN, Vol. 2, pp 775-780.

[16] [12] Z. Hu and H. Shao, (1992) “The study of neural network adaptive control systems,” control and Decision, vol. 7, pp 361-366.

[17] [13] C. T. Chen and W. D. Chang, (1996), “A feedforward neural network with function shape auto tuning,” Neural Networks, Vol. 9, issue (4), pp 627-641.

[18] [14] S. Xu and M. Zhang, (2001), “A novel adaptive activation function,” In Proc. Int. Joint Conf. Neural Networks, vol. 4, pp 2779-2782.

[19] [15] P. Chandra ,Y. Singh, (2004),“An activation function adapting training algorithm for sigmoidal feedforward networks,” Neurocomputing, pp 429-437.

[20] [16] S. K. Sharma and P. Chandra.,(2010). "An adaptive slope sigmoidal function cascading neural networks algorithm", Emerging Trends in Engineering and Technology (ICETET), 2010 3rd International Conference on IEEE.