Knowledge Extraction in Fuzzy Relational Systems Based on Genetic and Neural Approach

In this paper, a problem of MIMO object identification expressed mathematically in terms of fuzzy relational equations is considered. We use the multivariable relational structure based on the modular fuzzy relational equations with the multilevel composition law. The identification problem consists of extraction of an unknown relational matrix and also of parameters of membership functions included in the fuzzy knowledge base, which can be translated as a set of fuzzy IF-THEN rules. In fuzzy relational calculus this type of the problem relates to inverse problem and requires resolution for the composite fuzzy relational equations. The search for solution amounts to solving an optimization problem using the hybrid genetic and neural approach. The genetic algorithm uses all the available experimental information for the optimization, i.e., operates off-line. The essence of the approach is in constructing and training a special neuro-fuzzy network, which allows on-line correction of the extracted relations if the new experimental data is obtained. The resulting solution is linguistically interpreted as a set of possible rules bases. The approach proposed is illustrated by the computer experiment and the example from medical diagnosis.


Introduction
Fuzzy relational calculus [1,2] provides a powerful theoretical background for knowledge extraction from data. Some fuzzy rule base is modelled by a fuzzy relational matrix, discovering the structure of the data set [3 -5]. Fuzzy relational equations, which connect membership functions of input and output variables, are built on the basis of a fuzzy relational matrix and Zadeh's compositional rule of inference [6,7]. The identification problem consists of extraction of an unknown relational matrix, which can be translated as a set of fuzzy IF-THEN rules. In fuzzy relational calculus this type of problem relates to inverse problem resolution for the composite fuzzy relational equations [2].
Inverse problem resolution is of interest to both simplified relational models and multivariable ones. Solvability and approximate solvability conditions of the simplified composite fuzzy relational equations are considered in [2,8,9]. The non-optimizing approach [10] is widely used for fuzzy relational identification. Such adaptive recursive techniques are of interest for the most of on-line applications [11 -13]. Under general conditions, an optimization environment is the convenient tool for fuzzy relational identification [14]. An approach for identification of fuzzy relational models by fuzzy neural networks is proposed in [15 -17].
In the case of multiple variable linguistic model, the inputs -outputs dependency is extended to the multidimensional fuzzy relational structure and requires modularization of the system of fuzzy relational equations [7,18]. The insufficient use of the modular structures is stipulated through the lack of effective algorithms for solving fuzzy relational equations with multilevel composition law. When dealing with the multilevel structure, it is necessary to apply the optimization schemes rather than to make attempts to develop analytical solutions [18 -21].
An approach for fuzzy relational identification expressed mathematically in terms of composite fuzzy relational equations is proposed in [22,23]. We use two main types of relational structures built with the aid of simplified and modular multilevel fuzzy relational equations for MIMO object identification. In [22,23], the genetic algorithm [24,25] as a tool to solve the simplified composite fuzzy relational equations is adapted to identify the relational matrix of rules weights. In [26], we suggest some procedures of numerical solution of the modular multilevel relational equations using genetic algorithms. The procedures envisage Computer Science and Information Technology 2(1): [10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29]2014 11 the optimal solution growing from a set of primary variants by extending the simplified solution set to the case of multidimensional fuzzy relational equations. In this paper, we use the genetic algorithm [26] as a tool to solve the modular composite fuzzy relational equations to identify the relational matrix of terms weights for the given inputs-outputs data set.
Following [22,23], the algorithm for fuzzy relation matrix identification is accomplished in two stages. At the first stage, parameters of membership functions included in the fuzzy knowledge base and elements of the relational matrix are defined using the genetic algorithm. In this case, proximity of linguistic approximation results and experimental data is the criterion of extracted relations quality. At the second stage, the obtained null solution allows us to arrange the genetic search for the complete solution set for the relational matrix. After linguistic interpretation the resulting solution can be represented as a set of possible rules collections, discovering the structure of the given data.
The genetic algorithm [24 -26] uses all the available experimental information for the optimization, i.e., operates off-line and becomes toilful and inefficient if the new experimental data is obtained, i.e., in the on-line mode. In this paper, the process of knowledge extraction is augmented by a hybrid genetic and neural approach to solving composite fuzzy relational equations [23,27]. The essence of the approach is in constructing and training a special neuro-fuzzy network, which allows on-line correction of the extracted relations.
The approach proposed is illustrated by the computer experiment and the example of medical diagnosis.  training data in the form of L pairs of "inputs-outputs" experimental data It is necessary to transfer the available training data into the following system of IF-THEN rule [7] Rule k :

«Multiple Inputs -Multiple Outputs» Object
where ik a ( jk b ) is the fuzzy term describing a variable i x ( j y ) in rule k ; N is the number of rules.

Fuzzy Rules, Relations and Relational Equations
This fuzzy rule base is modelled by the fuzzy relational matrix presented in Table 1. Inputs-outputs interconnection is given by the system of SISO fuzzy relational matrices where il c is the fuzzy term for a variable i x evaluation, n i , 1 = We shall redenote the set of input and output terms-assessments in the following way } ,..., where jp ik v , is the weight of the term ik a in the rule k interpreted as a relation   * is the operation of (  ,  ) [7].
The system of fuzzy relational equations is derived from relation (3): Since the operation  is associated with max-min and the operation  is replaced by min in fuzzy set theory [7], system (4) can be rearranged as: We use a bell-shaped membership function model of variable u to arbitrary term T in the form [23]: where β is a coordinate of function maximum, 1 ) ( = β µ T ; σ is a parameter of concentration-extension.

Optimization Problem for Fuzzy Relations Extraction
Let us impose limitations on the knowledge base (2) volume in the following form: Therefore, knowledge base (1) synthesis is reduced to obtaining the matrix of knowledge base parameters (Table 1). This problem can be formulated as follows. It is necessary to find such a knowledge base parameters matrix, which satisfies the limitations imposed on knowledge base volume and provides the least distance between model and experimental outputs of the object:

Optimization Problem
If 0 R is a solution of the optimization problem (9), then 0 R is the exact solution of the composite system of fuzzy where the experimental input and output matrices 14 Knowledge Extraction In Fuzzy Relational Systems Based on Genetic and Neural Approach are obtained for the given training data.
The problem of solving fuzzy relational equations (10) is formulated as follows [24 -26]. Fuzzy relation matrix and also provides the least distance between model and experimental outputs of the object; that is, the minimum value of the criterion (9).
In the general case, system (10) Here , ( Following [24 -26], formation of the intervals (11) is accomplished by way of solving a multiple optimization problem (9) and it begins with the search for its null solution we have the same value of criterion (9). While searching for upper bounds ( The hybrid genetic and neural approach is proposed for solving optimization problem (9).

Genetic Algorithm for Fuzzy Relations Extraction
The chromosome needed in the genetic algorithm for solving this optimization problem includes the real codes of The multi-crossover operation provides a more accurate adjusting direction for evolving offsprings that allows to systematically reduce the size of the search region. The non-uniform mutation, the action of which depends on the age of the population, provides generation of the non-dominated solutions. We used the roulette wheel selection procedure giving priority to the best solutions. The greater the fitness function of some chromosome the greater is the probability for the given chromosome to yield offsprings. The fitness function is built on the basis of criterion (9). While performing the genetic algorithm the size of the population stays constant. That is why after crossover and mutation operations it is necessary to remove the chromosomes having the worst values of the fitness function from the obtained population [28,29].

Neuro-Fuzzy Network for Fuzzy Relations Extraction
Limitations on the knowledge base (2) volume allow embedding system (10) into the special neuro-fuzzy network, which is able to extract knowledge (See Fig. 1) are used which minimize the criterion η is a parameter of training [30].
The partial derivatives appearing in recurrent relations (12) can be obtained according to the results from [27].

Computer Experiment
The training data in the form of the interval values of input and output variables is presented in Table 2. The experiment methods consist of carrying out the following steps. 1 0 . Given the training data, find the null matrix of fuzzy relations between the input and output fuzzy terms by solving the optimization problem (9). We shall evaluate the quality of the model using the following root mean-squared errors j RMSE : The total number of fuzzy terms for input variables is limited to three. The classes for output variables evaluation are formed as follows:   Table 3 together with the parameters of the knowledge matrix is obtained using the genetic algorithm for the preliminary data. The complete solution set for the fuzzy relation matrix is presented in Table 4, where input 1 x is described by fuzzy terms

Low (L), Average (A), High (H); input 2
x is described by fuzzy terms Low (L), High (H); output 1 y is described by fuzzy terms higher than Low (hL), lower than Average (lA), High (H); output 2 y is described by fuzzy terms Low (L), lower than Average (lA). The obtained solution provides the approximation of the object shown in Fig. 3.
Computer Science and Information Technology 2(1): 10-29, 2014 19 Figure 3. Inputs-outputs model synthesized using the genetic algorithm The matrix of rules (terms combinations) weights, which corresponds to the preliminary solution, is presented in Table 5. The preliminary matrix of rules weights can be linguistically interpreted as (See Table 9):  the set of the four possible rules bases for output 1 y , which differ in the fuzzy terms describing output 1 y in rules 4 and 6 with overlapping weights;  the set of the eight possible rules bases for output 2 y , which differ in the fuzzy terms describing output 2 y in rules 2, 4, 6 with overlapping weights. For the specified data, a neural adjustment of the null solution (Table 3) has yielded the new solution n 0 R presented in Table 6 together with the parameters of the knowledge matrix. The complete solution set for the fuzzy relation matrix is presented in Table 7. The obtained solution provides the approximation of the object shown in Fig. 4. The matrix of rules (terms combinations) weights, which corresponds to the resulting solution, is presented in Table 8. The resulting matrix of rules weights can be linguistically interpreted as the unique set of IF-THEN rules (See Table 9).   The aim is to generate the system of IF-THEN rules for diagnosis of heart diseases. Input parameters are (variation ranges are indicated in parentheses): 1 x -aortic valve size (0.75 -2.5 cm 2 ); 2 x -mitral valve size (1 -2 cm 2 ); 3 x -tricuspid valve size (0.5 -2.7 cm 2 ); 4 x -lung artery pressure (65 -100 mm Hg).
The training data obtained in the Vinnitsa Clinic of Cardiology is represented in Table 10.
In current clinical practice, the valve sizes 3 1 x x ÷ are described by fuzzy terms stenosis (S) and insufficiency (I); pressure 4 x is described by fuzzy terms normal (N) and lung hypertension (H).
The classes for output variables j y evaluation are formed as follows:  The null solution g 0 R presented in Table 11 together with the parameters of the knowledge matrix is obtained using the genetic algorithm for the preliminary data. The obtained null solution  The complete solution set for the fuzzy relation matrix is presented in Table 12. The obtained solution provides the preliminary results of diagnosis presented in Table 16 for 57 patients.
The matrix of rules (terms combinations) weights, which corresponds to the preliminary solution, is presented in Table 13. The preliminary matrix of rules weights can be linguistically interpreted as (See Table 18):  the set of the sixteen possible rules bases for output 1 y , which differ in the fuzzy terms describing output 1 y in rules 13 -16 with overlapping weights;  the set of the two possible rules bases for output 2 y , which differ in the fuzzy terms describing output 2 y in rule 9 with overlapping weights;  the set of the four possible rules bases for output 3 y , which differ in the fuzzy terms describing output 3 y in rules 14 and 16 with overlapping weights;  the set of the sixteen possible rules bases for output 4 y , which differ in the fuzzy terms describing output 4 y in rules 10, 12, 14, 16 with overlapping weights. While interpreting the preliminary results in Table 13, the outputs   For the specified data, a neural adjustment of the null solution (Table 11) has yielded the new solution n 0 R presented in Table 14 together with the parameters of the knowledge matrix.  Table 16 for 57 patients.  The matrix of rules (terms combinations) weights, which corresponds to the resulting solution, is presented in Table 17. The resulting matrix of rules weights can be linguistically interpreted as (See Table 18 For heart diseases diagnosis, we obtain the average accuracy rates of 87% and 95% after genetic and neural training, respectively.

Conclusion
This paper proposes a method based on fuzzy relational equations to identify MIMO systems. In experimental data analysis rules generation combined with solving fuzzy relational equations is a promising technique to restore and identify the relational matrix together with some rules base explanation. The method proposed is focused on generating accurate and interpretable fuzzy rule-based systems. The obtained results depend on the randomness of the training data initialization, e.g., on the generation of the training intervals during the execution. In some cases the model may have the highest rule performance only with the special test and training data partition, which is used to build and test the model.
For the practical applications the initial and specified training intervals are derived directly from the problem. In this paper, the issue of adaptation of the resulting solution, while the samples of experimental data (training intervals) are changing, is considered. The genetically guided global optimization is augmented by more refined gradient-based adaptation mechanisms to provide the invariability of the generated fuzzy rule-based systems. Such an adaptive approach envisages the development of a hybrid genetic and neural algorithm for solving composite fuzzy relational equations. Using our new hybrid approach it is possible to avoid random effects caused by different partitions of training and test data by detecting a representative set of rules bases.