Modeling the Impedance Behavior of Ionic Conductors (AgPO3)1-x(Ag2SO4)x Glass System Using Artificial Neural Network

The dielectric permittivity and conductivity of (AgPO3)1-x(Ag2SO4)x compound was investigated at different concentrations of (Ag2SO4). The effect of concentration on AC conductivity and permittivity as well as temperature and frequency was investigated in order to model this behavior. Multidimensional mathematical models were as proposed to predict the impedance components ( ZZ ́,ZZ ́ ́ ) and the dielectric permittivity components (εε  ́, εε  ́ ́) of the glass system as a function of temperatures, frequencies and concentrations. Artificial Neural Network (ANN) and nonlinear regression approaches were set as curve fitting techniques in order to construct models based on 1700 points of data. This model can be then used to predict these proprieties at any concentration and therefore helping the product designer to choose the proper mixing and temperature conditions. For ANN, 20, 50, and 100 nodes in a single hidden layer neural network were considered. Although data results of the two approaches showed a good alignment with experimental data, the ANN model with twenty nodes was able to predict the outputs with lower MSE values range from 0.008 to 0.012 for impedance and from 0.006 to 0.008 for dielectric losses than the regression technique. Moreover, R values for the neural network were over 99% in both training and testing of impedance and dielectric permittivity while R values for non-linear regression vary between 73.86% to 94.75%. The proposed ANN model can be of a great help to find the optimal dielectric permittivity and conductivity of (AgPO3)1-x(Ag2SO4)x compound when dealing with a specific application.


Introduction
Neural Networks (NNs) have been extensively used in the literature to perform several artificial intelligence functions, such as pattern classification, object classification, data compression and regression (function approximation). The simplest form of NNs was inspired by human nerve cells and was first used by Pitts and Warren McCulloch in 1943, as a linear threshold gate [1]. A single neuron can perform very complex tasks by connecting to several adjusting neurons [2]. Figure 1 represents a single neuron which should be connected to many other neurons. With respect to Figure 1, each input (x 1 ,x 2 ,….x p ) is multiplied by a corresponding weight (w 1, w 2 ,….w n ), and then the resulting sum of these products is calculated of u. Neuron's output y, as expressed in Equation (1), A step function is then used to convert the resulting sum to either 0 or 1. This is done by comparing the sum to a certain threshold. In detail, the output of a neuron is 1 if the value of u-θ is greater than this threshold and 0 otherwise [2][3].
A more sophisticated neural network model can be developed by changing the previous topology or implementing a refined learning algorithm. Weights can be constructed based on a unidirectional connection (i.e., feed-forward) or two-way connection (i.e., feed-backward). On the other hand, an NN model needs a learning algorithm that optimizes the weights to reduce the error of the predicted output, such as backpropagation, error correction, and Boltzmann and Hebbian rules [2].
In literature, neural networks have been used in several modeling problems. In particular, different authors used this concept in material science in order to predict different material properties. For example, several experiments with different parameters were conducted in the process of granular flow prediction using NN, such as hopper angle and diameter, density, and coefficient of friction [4]. In another study, an artificial neural network was developed to simulate the process and predict the mass discharge rate. The concept of a model based on a radial base function artificial neural network (RBFANN) was generated and applied to predict the reduced glass transition temperature, T rg . It was found that close values are obtained from the simulated model and the experiment [5].
Lee et al. [6] used Artificial Neural Network (ANN) model to predict fatigue life under fluctuated stress of tension and compression with frequencies range from 2.5 to 8Hz. They trained their model until the root mean square error of output became very acceptable. In detail, five runs were conducted on samples of carbon fiber and one run on a sample of glass plastic. In addition to the previous research, Al-Assaf et. al [7] predicted fatigue life of unidirectional fiber/epoxy material under repeated tension and compression load using the ANN approach. Fiber orientation, maximum stress, and stress ratio were considered as input variables. Their ANN model was compared with other methods as well as different types of another artificial neural network, such as PCAN , RBF, and MN. They found that the most accurate network was a modular network (MN). Jarrah et al. [8] used adaptive neuro fuzzy inference system (ANFIS) in another material science research to study the relation between the number of cycles and the failure of unidirectional fiber/epoxy material. The inputs were stress ratio, angle of fiber, and maximum stress. Furthermore, two mathematical approaches (i.e., the linear regression and the computational neural network) were used to predict glass transition temperature for different structures [9]. Computational neural network (CNN) was used for 165 polymers in part 1 structure and 252 polymers in part 2 structure. The authors used a genetic algorithm with CNN to find the optimum values for glass transition temperature. The glass transition temperature for polymer composite material was also predicted using the radial basis function (RBF) model [10]. The RBF model was compared with its linear regression model (adjusted R -square value of 0.9269). Furthermore, the glass transition temperature for 277 polymers structures was studied using a recursive neural network (RNN) as well as quantitative structure-property relationship techniques to predict the physical and chemical properties, and then find the best parameters [11]. The effect of shear strain and the orientation angle of carbon fibers and glass fibers on shear stress was studied by Bezzera et. al [12]. They found that the best mathematical model of predicting non-linear relation between stress and strain is ANN. In spite of different literature of ANN modeling, ANN is not applied to predict the electrical behavior of the glass system (AgPO 3 ) 1-x (Ag 2 SO 4 ) x to the best of the authors knowledge.
An artificial neural network was used to and predict the failure of glass fiber reinforced epoxy composite pipes under several loading conditions [13]. In another study, the glass transition temperature is predicted with 95% accuracy using neural networks [14]. Furthermore, in [15], a three-layer feed-forward (ANN) was used to develop a model for glass-Tedlar photovoltaic thermal (PVT) air collector. The model uses different weather conditions as input to predict thermal energy, electrical energy, overall thermal energy, and overall exergy gain. Several machine learning tools (including ANN) was also used to predict the dissolution rates of aluminosilicate glasses when exposed to various solution with different pH values [16]. Moreover, an algorithm incorporating ANN was employed to estimate output power or performance parameters of a glass-glass (G-G). The estimated parameters were very close to the experimental values when compared by the artificial neural network (ANN) and analytical method alone [17]. A complex nonlinear model using ANN was developed in [18] to identify wear rate and coefficient of friction of dental glass-ceramic. Two different ANN models were presented in [19] to model the acoustic performance of glass fiber. In another study using ANN, the relationships between the bending strength and the mixture quantities were modeled [20].
In this work, different concentrations of glass system were successfully synthesized, and their impedance and dielectric permittivity were recorded. The recorded data was utilized to build different mathematical neural network models and compared them to regression models.

Sample Preparation
In order to prepare the samples of (AgPO 3 ) 1-x (Ag 2 SO 4 ) x , silver sulfate (Ag 2 SO 4 ) in a molar ratio of 0, 0.1, 0.15, 0.2 and 0.25 was added at room temperature to a mixed powder of NH 4 H 2 PO 4 and AgNO 3 . Then, the samples were annealed at a temperature of 225 o C for 30 minutes. Then the annealing temperature was raised to 400 o C and hold for another 2 hours in order to get rid of the byproduct gas. In order to make the reaction, the temperature was raised again to about 600 o C. To ensure the homogeneity, the solution was mixed using a ceramic rod for 3 hours. In the end, the molten mixture was cooled rapidly using water. Both sides of the resulted samples were polished using emery papers and then the two-sides electrodes were pained using a silver paste to make the electrical contacts.

Impedance Measurements
The impedance of all samples was measured using Solarton 1260 impedance analyzer at a frequency range of 50Hz to 1MHz with a temperature range from 25 o C to the glass transition temperature. The real and the imaginary components of the impedance (i.e., ´, ´´) were recorded. In the last step, the permittivity (i.e., ´, ´´) and the conductivity were derived from the impedance data.

Neural Network and Regression
There are different types of neural networks. However, most of them have a common structure, such as the input, the hidden, and the output layers. Optimizing the neural network's weights plays an important role in achieving better prediction. This can be accomplished using several methods that require training the networks by adjusting the neurons' weights to minimize the error. Once the training is completed, the network is ready to predict the outputs corresponding to any inputs.
In this work, Feedforward Neural Network will be used to estimate several electrical properties of (AgPO 3 ) 1-x (Ag 2 SO 4 ) x . Many Neural Network architectures were constructed for this estimation using different learning algorithms. Scaling the data using logarithmic transformation was also considered to improve the performance of the Neural Network. Non-linear regression was also used and considered a three-way interaction between all variables. Regression models are then used to find the optimal coefficient. For all used methods, the data will be divided into two parts: 70% for training and 30% for testing. It is worth mentioning that the regression models (linear and non-linear) are based on certain assumptions to predict the expected outcomes (i.e., best fit equation). On the other hand, the NN has the capability of proper prediction without the assumptions of the relationship between input and output variables.

ANN Approach
A multilayer perceptron (MLP) was used in this work. Although MLP uses a feed-forward structure, it differs from previously discussed neural network models in having an additional hidden layer, as shown in Figure 2. The weights of the network are optimized by using back-propagation learning which is based on a gradient-descent algorithm that tries to minimize the average squared error of the network [21]. On the other hand, the number of hidden layers is assigned by trial and error. Additionally, the number of nodes for each layer depends on the on-hand application [22]. For each node, input is multiplied by its allocated weights ( 1 1, ) then the summation (S) of these terms is calculated as in Equation (2). The result is then transformed/scaled to improve the efficiency of the neural network. Although transformation function can be as simple as any linear function, sigmoid function as presented in Equation (3) is commonly used for several applications. The shape of this function looks like "S" (the large positive sum is transformed to 1 and the negative large sum is transformed to 0). The previous steps are referred to as forwarding propagation. The weights are initially generated randomly as a first training step. The weights are then optimized using a specific learning algorithm. In order to find the appropriate weights for a specific neural network, the back propagation algorithm is used. The objective of this method is to adjust the weights iteratively for each data pattern (r). Feeding all data points into a network to optimize the weights is then repeated for a number of iterations (epochs) until an overall threshold (i.e., the sum of square errors) is met. At a specific output node, the sum of square errors is calculated using the following formula [24]:

Regression Approach
In this study, three independent variables were involved to predict several electrical properties of (AgPO 3 ) 1-x (Ag 2 SO 4 ) x . The regression model is a mathematical fitting technique used to relate several independent variables with a specific response. For example, the multiple nonlinear regression model (NLR) with three independent variables and involving interactions between them is presented in Equation 5 [25].

ANN and NLR Analysis
NN and NLR were used to estimate several electrical properties of (AgPO 3 ) 1-x (Ag 2 SO 4 ) x Glass. A number of neural network architectures were tested at the beginning of the experiments to find the optimal structure. At first, feedforward NNs with a different number of nodes in the hidden layer were tested (10, 20, and 50 nodes) using around 1700 data points (70% training and 30% for testing). The network's weights were optimized using Bayesian regularization as it can provide better results when compared to . We have also chosen a network to fit all four electrical properties at once. Moreover, the networks were built into two different structures. In the first one, the neural network has four outputs: two for impedance and two for dielectric (i.e., impedance (´,´´) and dielectric properties (´,´´)). On the other hand, two neural networks were considered to train the model, each network had two outputs only (fitting impedance and dielectric permittivity). Training the network using the second structure showed a lower mean square error when compared to the first one. The primary trails showed that a single hidden layer network with twenty nodes and two outputs provides more acceptable results when compared to a network with a higher number of nodes in the hidden layer. See Figure 3. Concentration, the inverse of the temperature in kelvin and logarithmic frequency were used in the input layers for both constructed networks, as neural networks perform better with scaled data. The output layer for the first network was Impedance properties (´,´´) while the output layer for the second network was the dielectric permittivity (´,´´).

´´
To determine the square root of mean square error (MSE) using a single hidden layer's network (with 20 nodes). All initial weights are randomly generated. The Bayesian regularization approach was used to adjust weights in every epoch to fit the data and minimize the error between output results and experimental ones (the data was trained for 250 epochs). For impedance training data, it was observed that the error stopped decreasing at around 60 epochs with MSE is 0.008. For testing, however, the best performance of 0.012 at epoch 100, as shown in Figure (4.a) and Figure (4.b). For the dielectric data, training the data achieved an MSE of 0.006 at 250 epochs. Furthermore, an MSE of 0.008 was achieved at epoch 150, as shown in Figure 4.c and Figure 4.d, respectively. The three models (10, 20, and 50 nodes) are trained with a different number of nodes in a single hidden layer. The best model was the 20 nodes model for in terms of impedance and dielectric based on MSE of training and testing. For non-linear regression with three-way interaction, the MSE for training was (0.098 for ´ and 0.214 for ´´) and MSE for testing was (0.046 for ´ and 0.111 for ´´) , while the MSE for training was (0.008 for ´ and 0.004 for ´´) and MSE for testing was (0.008 for ´ and 0.008 for ´´) . For more information, the MSE values for each model are summarized in Table 1.
To check the goodness of the previous models, R 2 values were calculated, as presented in Table 2. R 2 values for the neural network were over 99% in both training and testing of impedance and dielectric permittivity (see Figure 5). On the other hand, R 2 values for non-linear regression vary between 73.86% to 94.75%. Although the best R 2 was achieved for log ´´, the neural network showed a significant improvement in data fitting (i.e., R 2 99.5%).   Figure 6 shows the dependence of the real part permittivity (έ) on frequency and temperature. The values of permittivity decrease with respect to frequency for both samples (i.e., x=0.0 and x=0.15), as shown in Figure (6.a) and Figure (6.b). This behavior of decreasing can be related to the difficulties of the dipoles to reorient themselves quickly at high frequency and, hence, lowering the values of the dielectric permittivity. For such a system, the increase of temperature causes also an increase in the real part of dielectric permittivity. The mobility of the charge carriers will be affected directly by the thermal energy and consequently, the dipoles will orient themselves easily in the direction of the applied electrical field. This leads to achieving higher values of permittivity, as shown in Figures 6.c and Figure 6.d. The data of the neural network and the regression models have been plotted in the previous figures. It is clear that the data generated by the NN coincide strongly with the experimental data.

Experimental Results and Data Fittings
The experimental data showed that the values of the dielectric loss decrease with increasing the frequency, as shown in Figure (7.a) and Figure (7.b). This behavior can be related to the conduction loss caused by the migration of ions over large distances at lower frequencies. At higher frequencies, there is less time for the dipoles to reorient themselves in the direction of the electric field so the conduction losses will be lower at higher frequencies. For the effect of temperature, it was found that the dielectric loss increases with increasing temperature. In detail, the temperature motivates the ions to jump to new sites and hence lose some energy with motion. The effect of the temperature on the dielectric loss for samples x=0.0 and x=0.15 is shown in Figure (   The results of experiments are modeled by two approaches, artificial neural network regression. These models are used to find the non-linear relation between total conductivity and dependent frequency for glass compound (AgPO 3 ) 1-x (Ag 2 SO 4 ) x , at a temperature of 80 o C with different concentrations of Ag 2 SO 4, x= 0, 0.1,0.15,0.2, and 0.25 as illustrated in Figure 8. It is shown that the curve of total conductivity was divided into two regions dc which acts as constant conductivity at low frequencies and ac conductivity which follows power law at high frequencies according to total conductivity [27]: where σ ac = ( 1/´)*( / ) , is based on real part of impedance, A and t are the area of the electrode and the thickness of the sample, respectively. The constant conductivity intersects ac conductivity at a frequency called cutoff frequency (i.e., the frequency of dc equals to ac frequency (ω dc = ω ac ) [28]). So, as mentioned previously, the dc conductivity depends on the frequency at the second region significantly, while the frequency in another region is considered as an independent factor. Moreover, it is clear that total conductivity increases as the concentration of (Ag 2 SO 4 ) increase until reaches the value of x= 0.20 (see Figure 8) [28]. However, three factors (concentration x, temperature T, and frequency F) are used as input factors, while conductivity is considered to be an output response to build the regression technique and ANN models. The most accurate technique to fit the data is ANN-based on the gradient method to predict conductivity while the regression one did not give a higher accuracy based on mean square error.
Experimental results showed a clear increase of conductivity σ ac as temperature increases. This increase is due to the thermally activated process and ions mobility over long distances, while at low temperatures the conduction is mainly based on hopping of charge carriers in localized states [11,[29][30]. Also increasing temperature provides Ag+ ions with thermal energy to disjoint from non-bridging oxygen and enhance them to cross the barriers. However, the behavior of the curves in Figure 9 becomes more linearity at high temperatures for all concentrations. Figure 9 includes ANN modeling which provides high accuracy for prediction conductivity based on MSE comparing to classical regression technique.

Conclusions
(AgPO 3 ) 1-x (Ag 2 SO 4 ) x glass system was prepared with different concentrations of (Ag 2 SO 4 ). Conductivity and dielectric losses have been measured under the effect of different variables, such as concentration, temperature, and frequency. Multi-dimensional mathematical models were utilized to build a relationship between input variables and output responses (conductivity and dielectric losses). Artificial neural network and non-linear regression approaches were prepared to fit the measured data. Three models of ANN with a different number of nodes in a single hidden layer network build, the model with twenty nodes gave lower MSE values range from 0.008 to 0.012 for impedance and from 0.006 to 0.008 for dielectric losses. Moreover, the proposed ANN model provided an excellent agreement with experimental data compared to the non-linear regression model based on MSE. The proposed ANN model can be of great help to find the optimal dielectric permittivity and conductivity of (AgPO 3 ) 1-x (Ag 2 SO 4 ) x compound when dealing with a specific application.