Mathematics and Statistics Vol. 10(6), pp. 1206 - 1209
DOI: 10.13189/ms.2022.100606
Reprint (PDF) (310Kb)

Step, Ramp, Delta, and Differentiable Activation Functions Obtained Using Percolation Equations

David S. McLachlan ^1,*, Godfrey Sauti ^2
1 School of Physics, University of the Witwatersrand, Johannesburg 2050, South Africa
² Advanced Materials and Processing Branch, NASA Langley Research Center, United States

ABSTRACT

This paper presents two new analytical equations, the Two Exponent Phenomenological Percolation Equation (TEPPE) and the Single Exponent Phenomenological Percolation Equation (SEPPE) which, for the proper choice of parameters, approximate the widely used Heaviside Step Function. The plots of the equations presented in the figures in this paper show some, but by no means all, of the step, ramp, delta, and differentiable activation functions that can be obtained using the percolation equations. By adjusting the parameters these equations can give linear, concave, and convex ramp functions, which are basic signals in systems used in engineering and management. The equations are also Analytic Activation Functions, the form or nature of which can be varied by changing the parameters. Differentiating these functions gives delta functions, the height and width of which depend on the parameters used. The TEPPE and SEPPE and their derivatives are presented in terms of the conductivity () owing to their original use in describing the electrical properties of binary composites, but are applicable to other percolative phenomena. The plots in the figures presented are used to show the response (composite conductivity) for the parameters (higher conductivity component of the composite), (lower conductivity component of the composite) and , the volume fraction of the higher conductivity component in the composite. The additional parameters are the critical volume fraction, , which determines the position of the step or delta function on the axis and one or two exponents , and .

KEYWORDS
Percolation, Step Functions, Delta Functions, Activation Functions, Neural Networks

Cite This Paper in IEEE or APA Citation Styles
(a). IEEE Format:
[1] David S. McLachlan , Godfrey Sauti , "Step, Ramp, Delta, and Differentiable Activation Functions Obtained Using Percolation Equations," Mathematics and Statistics, Vol. 10, No. 6, pp. 1206 - 1209, 2022. DOI: 10.13189/ms.2022.100606.

(b). APA Format:
David S. McLachlan , Godfrey Sauti (2022). Step, Ramp, Delta, and Differentiable Activation Functions Obtained Using Percolation Equations. Mathematics and Statistics, 10(6), 1206 - 1209. DOI: 10.13189/ms.2022.100606.