Comparative analysis of activation functions in neural networks

Simple item page
dc.contributor.authorKamalov, Firuz
dc.contributor.authorNazir, Amril
dc.contributor.authorSafaraliev, Murodbek
dc.contributor.authorCherukuri, Aswani Kumar
dc.contributor.authorZgheib, Rita
dc.date.accessioned2022-05-18T13:23:50Z
dc.date.available2022-05-18T13:23:50Z
dc.date.copyright© 2021
dc.date.issued2021
dc.descriptionThis conference paper is not available at CUD collection. The version of scholarly record of this paper is published in 2021 28th IEEE International Conference on Electronics, Circuits, and Systems (ICECS) (2021), available online at: https://doi.org/10.1109/ICECS53924.2021.9665646
dc.description.abstractAlthough the impact of activations on the accuracy of neural networks has been covered in the literature, there is little discussion about the relationship between the activations and the geometry of neural network model. In this paper, we examine the effects of various activation functions on the geometry of the model within the feature space. In particular, we investigate the relationship between the activations in the hidden and output layers, the geometry of the trained neural network model, and the model performance. We present visualizations of the trained neural network models to help researchers better understand and intuit the effects of activation functions on the models. © 2021 IEEE.
dc.identifier.citationKamalov, F., Nazir, A., Safaraliev, M., Cherukuri, A. K., & Zgheib, R. (2021). Comparative analysis of activation functions in neural networks. 2021 28th IEEE International Conference on Electronics, Circuits, and Systems, ICECS. https://doi.org/10.1109/ICECS53924.2021.9665646
dc.identifier.isbn978-172818281-0
dc.identifier.urihttps://doi.org/10.1109/ICECS53924.2021.9665646
dc.identifier.urihttp://hdl.handle.net/20.500.12519/641
dc.language.isoen_US
dc.publisherInstitute of Electrical and Electronics Engineers Inc.
dc.relationAuthors Affiliations : Kamalov, F., Canadian University Dubai, Department of Electrical Engineering, Dubai, United Arab Emirates; Nazir, A., Zayed University, Department of Information Systems, Abu Dhabi, United Arab Emirates; Safaraliev, M., Ural Federal University, Automated Electrical Systems Department, Yekaterinburg, Russian Federation; Cherukuri, A.K., Vellore Institute of Technology, School of IT Engineering, Vellore, India; Zgheib, R., Canadian University Dubai, Department of Computer Science, Dubai, United Arab Emirates
dc.relation.ispartofseries2021 28th IEEE International Conference on Electronics, Circuits, and Systems (ICECS)
dc.rightsPermission to reuse abstract has been secured from Institute of Electrical and Electronics Engineers Inc.
dc.rights.holderCopyright : © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
dc.rights.urihttps://www.ieee.org/publications/rights/rights-policies.html
dc.subjectactivation function
dc.subjectloss function
dc.subjectneural networks
dc.subjectReLU
dc.subjectsigmoid
dc.titleComparative analysis of activation functions in neural networks
dc.typeConference Paper
dspace.entity.type

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Access Instruction 641.pdf
Size:
56.44 KB
Format:
Adobe Portable Document Format
Description:
Download

Collections

Department of Electrical Engineering