New Possibilities of Application of Artificial Intelligence Methods for High-Precision Solution of Boundary Value Problems

One of the main problems in modern mathematical modeling is to obtain high-precision solutions of boundary value problems. This study proposes a new approach that combines the methods of artificial intelligence and a classical analytical method. The use of the analytical method of fictitious canonic regions is proposed as the basis for obtaining reliable solutions of boundary value problems. The novelty of the approach is in the application of artificial intelligence methods, namely, genetic algorithms, to select the optimal location of fictitious canonic regions, ensuring maximum accuracy. A general genetic algorithm has been developed to solve the problem of determining the global minimum for the choice and location of fictitious canonic regions. For this genetic algorithm, several variants of the function of crossing individuals and mutations are proposed. The approach is applied to solve two test boundary value problems: the stationary heat conduction problem and the elasticity theory problem. The results of solving problems showed the effectiveness of the proposed approach. It took no more than a hundred generations to achieve high precision solutions in the work of the genetic algorithm. Moreover, the error in solving the stationary heat conduction problem was so insignificant that this solution can be considered as precise. Thus, the study showed that the proposed approach, combining the analytical method of fictitious canonic regions and the use of genetic optimization algorithms, allows solving complex boundary-value problems with high accuracy. This approach can be used in mathematical modeling of structures for responsible purposes, where the accuracy and reliability of the results is the main criterion for evaluating the solution. Further development of this approach will make it possible to solve with high accuracy of more complicated 3D problems, as well as problems of other types, for example, thermal elasticity, which are of great importance in the design of engineering structures.


Introduction
Modern methods of engineering structures design include math modeling as the obligatory stage. Math modeling of engineering structures leads to the solving of boundary value problems in the most cases: stress-strain state evaluation; temperature and electro-magnetic fields and so on. The main approaches of solving boundary value problems are numerical methods, namely, finite element and finite difference methods. The existing computer application, based on the numerical methods, allows engineers, who are not experienced in the mathematical physics, to automatically get solutions of boundary value problems for complicated engineering structures. This particular advantage is the reason of widely using the finite element and finite differences methods in the modern engineering practice. Nevertheless, the numerical methods have some drawbacks which will be considered later in the article.
Last decade there is the trend of using artificial intelligence method for solving boundary value problems. This approach is the alternative for classical numerical methods. The main tools of the modeling in the case are artificial neural networks [1][2][3][4][5][6][7][8][9][10][11][12]. The algorithm of solving boundary value problem reduced to the training of the specially designed artificial neural network. The structure of the neural network, neuron activation functions and the training method selected depending on the special 324 New Possibilities of Application of Artificial Intelligence Methods for High-Precision Solution of Boundary Value Problems requirements of the boundary value problem. The choice of the neural network parameters determines the accuracy of the solution, whereas the training method influences the speed of the solution. Development of the general methods of the neural network modeling allows using them for solving boundary value problems with different types of partial differential equations, including nonlinear ones. Compared with the classical grid numerical methods, using artificial neural networks has some advantages: no connected mesh required; easily applied for partial differential equations of any type [1][2][3][4][5][6][7][8][9][10][11][12]. However, in spite of the described successful application, the neural network approaches suffer from the same lack as the classical numerical methods: there is no possibility to estimate robustly the results of the solution. It should be noted here that there exist convergence theorems for most of the numerical methods. But the conditions of the convergence with the mesh increasing are not met in the real engineering problems [13][14][15]. The neural networks methods have no theoretical foundation of the convergence and the precision of the solution estimated, in common, by the error functional value in some solution points [1][2][3][4][5][6].
Our opinion is that such error estimation cannot be considered as robust. The matter is that the solution, got with the neural network approach, satisfies neither differential equations nor boundary conditions. Thus, the error functional value can differ significantly from the value in the points of estimation.
Full and robust error estimation can be done for analytical solutions of the boundary value problem. As the solution is an analytical formula, one can estimate if it satisfies the differential equations and the boundary conditions. The most precise results produced when the analytical method leads to the solution, which precisely satisfies the differential equations [16]. However, such methods can be applied only for some special problems with simple geometry.
Leonid N. Yasnitsky, one of the authors of the article, suggested Fictitious Canonic Region (FCR) method in 1973 [16]. This is analytical-numerical method. The method leads to the solutions which satisfy precisely the differential equations and approximately the boundary conditions. Thus, the method allows robust estimation of the results precision [14]. At the same time, the FCR method can solve problems with complicated geometry.
Sergey L. Gladkiy, another author of the article, developed in 1990-2000-th software applications, based on the FCR method. The applications were used for solving some problems of thermal conductivity, electrostatics, elasticity and thermo-elasticity in 2D and 3D [17]. Table 1 shows the comparison of different methods of solving boundary value problems by several criteria. As one can see from the table, the FCR method is the most suitable for solving problems of building responsible engineering structures when the robust solution results is the main requirement. As mentioned above, the FCR method had been used in engineering practice for some period [17][18][19][20] and some applications had been created based on the method. In spite of this, the method now is not used widely. This is because the only experts, who are experienced in fundamentals of mathematical physics and theory of the FCR method, can use the applications. The main challenge is the choice of fictitious regions and their positioning for complicated geometry of the solving boundary value problem.
We suggest using artificial intelligence approaches, namely, genetic algorithms, for solving the issue. The article describes several cases of the genetic algorithms for selecting the types of the FCRs and their positions. The efficiency of the algorithms demonstrated with the solutions of thermal conduction and elasticity problems.

Trefftz Method
A paper should have a short, straightforward title directed at general readers in no more than 20 words.
Let us state the linear boundary value problem: find ( ) x U function that satisfies the following partial differential equation on its surface S . Here L and B are linear partial differential operators with constant coefficients; ( ) are known functions of coordinates x .
Following the Trefftz method [21], the solution of the boundary value problem (1)-(2) can be found as the sum  (2), for example, minimizing error functional with least squares approximation method. Thus, the solution (3) identically satisfies the original partial differential equation and approximately the boundary conditions. It means that the boundary conditions satisfy exactly the following equation The difference of stated boundary conditions

Fictitious Canonic Region Method
Leonid N. Yasnitsky suggested a methodology of selections of basis functions ) (x n U [16,22]. The methodology is based on the geometric interpretation of the Trefftz method and allows to successfully apply the method. The essence of the methodology will be given with an example. Let us consider a boundary value problem for the domain D with its boundary S (figure 1, a). Along with the D domain we consider some fictitious canonic region V . The original domain embedded in the mind's eye onto the fictitious region ( figure 1, b). As the V region is a canonic one, we can use the separation of variables method (also known as the Fourier method) to get the solution of the differential equation in the form of the following series: can be used as a region V .
This geometric interpretation of the Trefftz mathematical approach allowed formulating and proving the first theorem of convergence [16]. It was also used as the base of the criteria and methodic of basis functions choice. The methods of boundary value problem solutions precision estimation had been suggested and justified. Thus, math approach, built with this theoretical basis, was called Fictitious Canonic Region method [16].
The essence of the FCR choice criteria is the requirement of the solution extending into the region V .
The term «extending» means the possibility to prolongate the solution (as a function which satisfies the differential equation) outside the domain D into the greater region V . The region V here means the minimal domain which includes D and that belongs to the class of regions where the series (5)  Despite the existence of the theoretical justification of basis functions choice and the convergence of the FCR method, the following issues remain unsolved:  Singular points of the function, which is the solution of the boundary value problem, can be found after the function is constructed, i.e. the solution found.  There are, in common, many dispositions of the FCRs, which enforce the extending conditions. There is no criteria of the optimal solution (while it is not found) at the time.


We suggest using methods of artificial intelligence, namely, genetic algorithms, for solving mentioned issues.

Genetic Algorithm for Fictitious Canonic Regions Choice and Disposition
Genetic algorithms are a branch of artificial intelligence and are currently recognized as one of the most effective methods for solving multi-extreme optimization problems. Unlike gradient methods, the optimization process using genetic algorithms is not prone to "getting stuck" in local minima, which is their main advantage. This advantage is due to the fact that genetic algorithms model the natural optimization mechanism that prevails in the plant and animal world. Genetic algorithms originate from Darwin's evolutionary theory. They are based on the idea of using mechanisms of natural selection and genetic inheritance. The optimization problem is formulated in such a way that its solution can be represented in the form of a vector -"chromosomes", whose components are parameters -"genes" that characterize this solution. A number of initial vectors is randomly created -the "initial population". They are evaluated using some criterion of the quality of the solution -the "fitness function". As a result, each vector is assigned a certain value that determines the probability of survival of the individual represented by this chromosome-vector. After that, using the obtained values of the fitness function, "natural selection" is made -"individuals" with a fairly high value of the fitness function survive. Further, genetic operations are performed on the "surviving" individuals -"crossing" and "mutations". As a result, a "new generation" is created. Individuals of a new generation are also evaluated using the fitness function, then breeding is performed again, genetic operations are applied, etc. This is how the "evolutionary process" is simulated, which lasts several life cycles: generations of individuals replace each other until the stop criterion is met the algorithm. Such a criterion may be: finding a global or suboptimal solution; excess of the generation number, allocated for evolution; excess of time for calculation.
There are the following general stages of genetic algorithm: I.
Creating initial population. II. Evaluating fitness function values for every individual in the population. III. Beginning of the evolution loop:  Selecting individuals from the population (selection).  Crossover and mutation.  Evaluating fitness function values for all individuals.  Creating new generation.

IV. End of the evolution loop.
For implementing the genetic algorithm of FCR selection and positioning, the fitness function can be assigned as the following functional: (6) This functional is the integral characteristic of the solution error of the boundary value problem.
Let us introduce the individual term, which can be represented as the following structure as shown on the  K -FCRs' count, integer number.
x, y -coordinates of the FRC center, real values.
N -number of items in the general solution (5) of the FCR, integer number.
Thus, the individual term here means one case of the fictitious canonic regions positioning. The genes of the chromosome are: FCRs' count, types of the FCRs, coordinates of their centers and number of items of general solutions of the canonic regions. The goal of the genetic algorithm is finding such individual for which the fitness function (functional (6)) is minimal in global sense.
Two types of the genetic algorithm have been investigated: continuous and discrete. Continuous algorithm allows the FCR center to be in any point. Thus, the global optimization problem defined on the infinite set of FCRs positioning, practically limited only by the precision of real numbers used in computations. Discrete algorithm builds an uniform grid with some finite steps. The FCR center can be positioning in the node of the grid only. Thus, the global optimization problem is defined on final set of FCRs' positioning.
All individuals in the described genetic algorithm must be correctly defined in the sense of the solving physical problem. For example, it is obvious that the center of a fictitious region "round hole" must not be located inside the original domain of the boundary value problem. The genetic algorithm realizes this verification of the individuals.
The initial population of individuals is generated randomly. The number of the individuals in the population is an input parameter of the algorithm. The number of FCRs, their types, centers and number of items for every FCR generated randomly.
The centers of the FCRs selected according to the specified type of the algorithm -continuous or discrete. If the continuous method is used, the centers of the FCRs can be placed at any point, and besides, normal distribution is used to define both x and y coordinates. If the discrete method is used, the centers placed into the nodes of the grid and the probability for any node is the same.
After the generation of the individuals they all checked for the correctness after. If some individual is not correct, its genes regenerated again.
After evaluating the value of the fitness function using formula (6) all individuals in the population ordered by this value. The individual with the best value takes the leading position, it is not subjected to mutations and necessary selected for the next generation. This satisfies the requirement that the fitness function value for the next generation will be at least not worse as for the current generation.
Two variants of individual selection for crossover have been realized.
In the first variant, each of the individuals is crossed with one of the remaining individuals, randomly selected. Thus, in this case, each individual takes part at least once.
The second variant is called "roulette" algorithm. For each individual, the probability value for crossing is formed. The better the fitness function, the greater the probability of crossing. In accordance with the probability values, random pairs are formed in which descendants appear.
After performing the crossing operation, the obtained individual is checked for correctness. If the individual is not correct for this task, then the crossover operation is overfulfilled.
Five variants of the crossover operator are implemented.
Simple method. Randomly selects the integer K in the range from 1 to M , where M is the number of FCRs in the individual. Next, the first FCR from the first individual is selected into the descendant individual, the remaining FCRs get from the second individual.
Flat method. FCRs, located at the corresponding positions in the chromosomes of individuals, are crossed. The center of the new FCR is randomly selected from the gap formed by the centers of the FCR-parents. The type of the new FCR is randomly selected from the types that the FCR parents had.
Mixed method. FCRs, located at the corresponding positions in the chromosomes of individuals, are crossed. The center of the new FCR is randomly selected from the gap formed by the centers of the FCR-parents. This gap increases due to the coefficient, which is a parameter of the crossing method. The type of the new FCR is randomly selected from the types that the FCR parents had. After the crossing step is completed, the mutation step is performed. Each individual can mutate with some degree of probability, which is the input parameter of the genetic algorithm. With the mutation in an individual, strictly one gene can change. The FCR is randomly selected, which will be susceptible to the mutation. In the selected FCR, the type may change to one of the remaining ones, or one of the coordinates of the center. The coordinate changes in the neighborhood of the original coordinate.
After the mutation operation, the obtained individual is checked for the correctness. If the individual is not correct for this task, then the mutation operation is performed again.
The next step is the selection. At this stage, the check is made for the presence of duplicate individuals. If a duplicate is found, then it is replaced by a new random individual. After no single duplicate remains in the population, the values of the fitness function are calculated for each of the remaining individuals. Individuals in the population are sorted according to the values of the fitness function, and the next generation includes half of the individuals with the best fitness function value. Consider the following problem of stationary heat conduction: it is required to find the temperature distribution in the doubly connected domain shown on figure 3. On the external boundary of the domain the temperature is set to 100 °C, and on the internal 3320 °C.

Stationary Heat Conduction
As fictitious canonical regions, ring regions were used, for each of which the general solution of the two-dimensional heat equation (Laplace's equation) in polar coordinates θ , r has the following form: The optimization problem is stated as the following: select optimal number of the ring FCRs with optimal positioning relative to the domain D, and defining optimal count of items 1 N and 2 N in the series (7). The optimality condition is minimal error of boundary conditions of the boundary value problem solution for selected ring regions. For solving the optimization problem, the genetic algorithm has been launched with the following input parameters:  number of the individuals (solutions of the boundary value problem) in the population -100;  initial count of FCRs in an individual -9;  initial number of items ( 1 N and 2 N ) -20;  mutation probability -5%;  maximal number of generations -100.
After the first 10 generations the error of the boundary conditions reached the value about 0.2%, after 100 generations the error decreased to the value about 0.007%. Taking into account that the function (5) satisfies the partial differential equation identically, it can be stated that the function is practically exact solution of the boundary value problem for the domain D. The solution is presented on the figure 4 as the isolines of the temperature distribution. The positioning of the 9 fictitious ring regions is relative to the domain D found with the genetic algorithm presented on the figure 5.

Theory of Elasticity
Analogously, a problem of theory of elasticity has been solved. The domain for the problem is the same as shown on the figure 3. The boundary conditions for the domain are the following: the outer boundary is fixed rigidly; uniform pressure of 152 MPa (1500 atm) is set on the inner boundary.
Infinitely long hollow cylinders have been used as fictitious canonic regions. The general solution of the equations of the theory of elasticity (Lame's equations) in cylindrical coordinate systems ( )