Zeng, M., Li, Z., Saw, J. W., & Chen, B. (2024). Effect of stochastic activation function on reconstruction performance of restricted Boltzmann machines with stochastic magnetic tunnel junctions. Applied Physics Letters, 124(3). https://doi.org/10.1063/5.0171238
Abstract:
Stochastic Magnetic Tunnel Junctions (SMTJs) emerge as a promising candidate for neuromorphic computing. The inherent stochasticity of SMTJs makes them ideal for implementing stochastic synapses or neurons in neuromorphic computing. However, the stochasticity of SMTJs may impair the performance of neuromorphic systems. In this study, we conduct a systematic examination of the influence of three stochastic effects (shift, change of slope, and broadening) on the sigmoid activation function. We further explore the implications of these effects on the reconstruction performance of Restricted Boltzmann Machines (RBMs). We find that the trainability of RBMs is robust against the three stochastic effects. However, reconstruction error is strongly related to the three stochastic effects in SMTJs-based RBMs. Significant reconstruction error is found when the stochastic effect is strong. Last, we identify the correlation of the reconstruction error with each stochastic factor. Our results might help develop more robust neuromorphic systems based on SMTJs.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - Career Development Fund (CDF)
Grant Reference no. : C210812054
Description:
This article may be downloaded for personal use only. Any other use requires prior permission of the author and AIP Publishing. This article appeared in Zeng, M., Li, Z., Saw, J. W., & Chen, B. (2024). Effect of stochastic activation function on reconstruction performance of restricted Boltzmann machines with stochastic magnetic tunnel junctions. Applied Physics Letters, 124(3). https://doi.org/10.1063/5.0171238 and may be found at https://doi.org/10.1063/5.0171238